Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/html-all.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@doc-kit/core': minor
'@doc-kit/generator-react': minor
'@node-core/doc-kit': minor
---

`all.html` moves out of the `html` generator into a new Node.js generator, `html-all` (`@node-core/doc-kit`), and `html`'s `generateAllPage` option is removed. `html-all` is spliced between `jsx-ast` and `html`, and adds `all.html`, every module page's content on one page, as a page made of theirs. `html` renders such a page, given with `parts`, the pages it is made of, from their compiled content, one part at a time rather than building every part's elements first. Builds that want `all.html` add the `html-all` target.
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ jobs:
path: node

- name: Build Node.js API docs
run: node packages/cli/bin/cli.mjs generate -t web -i "./node/doc/api/assert.md" -o out --config-file ./e2e/doc-kit.config.mjs
run: node packages/cli/bin/cli.mjs generate -t web -t html-all -i "./node/doc/api/assert.md" -o out --config-file ./e2e/doc-kit.config.mjs

- name: Install Playwright browsers
run: pnpm exec playwright install --with-deps
Expand Down
1 change: 1 addition & 0 deletions docs/generators.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ Outputs consumed by the Node.js project's own build and release processes.
| [`man-page`](./generators/man-page.md) | The `node.1` manual page. |
| [`api-links`](./generators/api-links.md) | A map of API symbols to their source locations. |
| [`addon-verify`](./generators/addon-verify.md) | Extracted addon code samples, arranged for testing. |
| [`html-all`](./generators/html-all.md) | `all.html`, every module page on one page. |

## Pipeline stages

Expand Down
1 change: 1 addition & 0 deletions packages/core/src/generators/index.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export const publicGenerators = {
'json-simple': '@doc-kit/core/json-simple',
'legacy-html': '@node-core/doc-kit-legacy/legacy-html',
'legacy-html-all': '@node-core/doc-kit-legacy/legacy-html-all',
'html-all': '@node-core/doc-kit/html-all',
'man-page': '@node-core/doc-kit/man-page',
'legacy-json': '@node-core/doc-kit-legacy/legacy-json',
'legacy-json-all': '@node-core/doc-kit-legacy/legacy-json-all',
Expand Down
1 change: 1 addition & 0 deletions packages/node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ npm install --save-dev @doc-kit/core @node-core/doc-kit
| `man-page` | The `node.1` manual page, from the CLI options documented in `cli.md`. |
| `api-links` | A mapping of API symbols to their source locations, powering source links on nodejs.org. |
| `addon-verify` | Extracted C++ addon code samples from `addons.md`, arranged for compilation by the tests. |
| `html-all` | `all.html`, every module page's content on one page, rendered by the `html` generator. |

## Usage

Expand Down
2 changes: 2 additions & 0 deletions packages/node/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"./addon-verify": "./src/addon-verify/index.mjs",
"./api-links": "./src/api-links/index.mjs",
"./config": "./src/config/index.mjs",
"./html-all": "./src/html-all/index.mjs",
"./man-page": "./src/man-page/index.mjs",
"./package.json": "./package.json"
},
Expand All @@ -26,6 +27,7 @@
],
"dependencies": {
"@doc-kit/core": "workspace:*",
"@doc-kit/generator-react": "workspace:*",
"estree-util-visit": "^2.0.0",
"unist-util-visit": "^5.2.0"
},
Expand Down
19 changes: 19 additions & 0 deletions packages/node/src/html-all/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# `html-all` Generator

The `html-all` generator adds `all.html`, every module page's content on one
page, to the pages the [`html`](./html.md) generator renders. It builds nothing
itself: it is spliced between `jsx-ast` and `html`, passes the pages on, adds
`all.html`'s metadata to the manifest, and once every module page has passed,
adds a page made of theirs. `html` assembles it from their compiled content,
so no module is built twice.

The module pages are its parts, in the order the sidebar lists them; the
index, section pages and the other synthetic pages are left out.

```sh
npx @doc-kit/cli generate -t web -t html-all -i "doc/api/*.md" -o out
```

## Configuring

It has no options of its own.
81 changes: 81 additions & 0 deletions packages/node/src/html-all/__tests__/generate.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
import assert from 'node:assert/strict';
import { describe, it } from 'node:test';

import { generate } from '../generate.mjs';

/**
* A page as `jsx-ast` gives it.
*
* @param {string} api
* @param {string} name
* @param {object} [extra] - More of its metadata
*/
const createPage = (api, name, extra = {}) => ({
data: { api, heading: { depth: 1, data: { name } }, ...extra },
headings: [{ slug: api }],
content: '<></>',
});

/**
* Reads every batch the generator yields.
*
* @param {AsyncIterable<Array<object>> | Array<object>} input
*/
const run = async input => {
const batches = [];

for await (const batch of generate(input)) {
batches.push(batch);
}

return batches;
};

describe('html-all', () => {
const zlib = createPage('zlib', 'Zlib');
const fs = createPage('fs', 'File system');
const index = createPage('index', 'Index');
const chunk = createPage('fs/promises', 'fs.promises', { chunk: {} });
const notFound = createPage('404', 'Page not found', { synthetic: true });

it('adds all.html right after its last part, in sidebar order', async () => {
async function* input() {
yield [zlib];
yield [{ manifest: [zlib.data, fs.data, index.data, notFound.data] }];
yield [index, fs];
yield [chunk, notFound];
}

const batches = await run(input());

assert.deepEqual(batches[0], [zlib]);

const [{ manifest }] = batches[1];

assert.deepEqual(manifest.slice(0, -1), [
zlib.data,
fs.data,
index.data,
notFound.data,
]);
assert.equal(manifest.at(-1).api, 'all');

const [, , all] = batches[2];

assert.deepEqual(batches[2].slice(0, 2), [index, fs]);
assert.equal(all.data.api, 'all');
assert.deepEqual(all.parts, ['fs', 'zlib']);
assert.deepEqual(all.headings, [{ slug: 'fs' }, { slug: 'zlib' }]);

assert.deepEqual(batches[3], [chunk, notFound]);
});

it('adds all.html after every page given at once, as with one thread', async () => {
const batches = await run([zlib, fs, { manifest: [zlib.data, fs.data] }]);

const pages = batches.flat();

assert.equal(pages.at(-1).data.api, 'all');
assert.deepEqual(pages.at(-1).parts, ['fs', 'zlib']);
});
});
95 changes: 95 additions & 0 deletions packages/node/src/html-all/generate.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
'use strict';

import { toBatches } from '@doc-kit/core/utils/generators.mjs';
import { headingSortFn } from '@doc-kit/generator-react/jsx-ast/utils/getSortedHeadNodes';

// The metadata of `all.html`
const HEAD = {
api: 'all',
path: '/all',
basename: 'all',
synthetic: true,
heading: {
type: 'heading',
depth: 1,
children: [{ type: 'text', value: 'All' }],
data: { name: 'All', text: 'All', slug: 'all' },
},
};

/**
* Whether a page is part of `all.html`: the module pages, minus the index.
* Chunk pages and the other synthetic pages are left out, as the module pages
* carry their content.
*
* @param {import('@doc-kit/core/generators/metadata/types').MetadataEntry} data
* @returns {boolean}
*/
const isPart = data => !data.synthetic && !data.chunk && data.api !== 'index';

/**
* Builds `all.html` from its parts: their content one after the other, and
* their tables of contents, in the order the sidebar lists them.
*
* @param {Array<Pick<import('@doc-kit/generator-react/jsx-ast/types').PageCode, 'data' | 'headings'>>} parts
*/
const buildAllPage = parts => {
const pages = parts.toSorted((a, b) => headingSortFn(a.data, b.data));

return {
data: HEAD,
headings: pages.flatMap(({ headings }) => headings),
parts: pages.map(({ data }) => data.api),
};
};

/**
* Passes `jsx-ast`'s pages on, with `all.html`'s metadata in the manifest, and
* `all.html` itself right after the last of its parts.
*
* @type {import('./types').Generator['generate']}
*/
export async function* generate(input) {
const parts = [];

// The parts still to come, once every page's metadata is known
let missing;
let added = false;

for await (const batch of toBatches(input)) {
const pages = [];

for (const item of batch) {
if ('manifest' in item) {
missing = new Set(item.manifest.filter(isPart).map(({ api }) => api));

for (const { data } of parts) {
missing.delete(data.api);
}

pages.push({ manifest: [...item.manifest, HEAD] });

continue;
}

pages.push(item);

if (isPart(item.data)) {
parts.push({ data: item.data, headings: item.headings });
missing?.delete(item.data.api);
}
}

if (!added && missing?.size === 0) {
pages.push(buildAllPage(parts));
added = true;
}

yield pages;
}

// Without a manifest, once every page has passed
if (!added) {
yield [buildAllPage(parts)];
}
}
26 changes: 26 additions & 0 deletions packages/node/src/html-all/index.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
'use strict';

import { generate } from './generate.mjs';

/**
* Adds `all.html`, every module page's content on one page, to the pages the
* `html` generator renders. It builds nothing itself: it passes `jsx-ast`'s
* pages on to `html`, and, once every module page has passed, a page made of
* theirs, which `html` assembles from their compiled content.
*
* @type {import('./types').Generator}
*/
export default {
name: 'html-all',

description:
'Adds all.html, every module page on one page, to the html generator',

dependsOn: '@doc-kit/generator-react/jsx-ast',

streamsInput: true,

dependent: '@doc-kit/generator-react/html',

generate,
};
8 changes: 8 additions & 0 deletions packages/node/src/html-all/types.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import type { PageCode } from '@doc-kit/generator-react/jsx-ast/types';

type Pages = Array<PageCode | { manifest: Array<PageCode['data']> }>;

export type Generator = GeneratorMetadata<
{},
Generate<Pages | AsyncIterable<Pages>, AsyncGenerator<Pages>>
>;
15 changes: 8 additions & 7 deletions packages/react/src/html/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,13 @@ library once and the client assets once, as soon as every page's metadata is
known, and the worker pool imports, renders, templates, minifies and writes
the pages one at a time as they are compiled, so memory scales with the largest
page rather than with the site. With one thread, each step waits for the one
before it to finish. `all.html` is assembled from the module pages' compiled
content rather than built again from scratch.
before it to finish.

A page can be made of other pages' content: an input item with `parts`, the
`api`s of the pages it is made of, which come before it in the input, and its
`headings`. Its program imports theirs, so they are not built again, and it is
rendered first, as it takes the longest. Node.js's `html-all` generator
(`@node-core/doc-kit`) adds `all.html` this way.

## Configuring

Expand Down Expand Up @@ -50,10 +55,6 @@ content rather than built again from scratch.
JSX-in-MDX. See [`components`](#components). **Default:** `{}`.
- `navigation` {Object} Sidebar groups and navigation bar items. See
[`navigation`](#navigation). **Default:** `{}`.
- `generateAllPage` {boolean} When `true`, writes `all.html`: every module
page's content on one page, in sidebar order, assembled from the module pages
rather than built again. Chunk pages and the index are left out.
**Default:** `true`.
- `bundler` {WebBundler} Adapter that bundles the component library and the
client assets, and compiles page programs. See
[Bundler adapters](#bundler-adapters). **Default:** `createViteBundler()`.
Expand Down Expand Up @@ -488,7 +489,7 @@ export default ({ metadata }) => (
- `headings` {Array} Pre-computed table of contents heading entries.
- `readingTime` {string|undefined} Estimated reading time (e.g. `'5 min read'`).
Only present when the `jsx-ast` generator's `showReadingTime` option is
enabled. `all.html` has none.
enabled. A page made of others has none.
- `children` {ComponentChildren} Processed page content.

The `Layout` component receives the props above. Custom Layout components can use
Expand Down
Loading
Loading