Testin'

Th' test suite lives 'n th' infra repository an' runs against a theme checkout. See Develop'n fer how th' two be wired together.

Requirements

Node.js at th' version .nvmrc pins. Install it through a version manager - nvm, or nvm-windows - rather than as a system package, so th' version can follow th' project rather than th' machine. nvm use 'n th' infra checkout reads that file, an' so does CI, so th' two cannot drift.

Th' pin be not arbitrary. Before v26.8.1, Node’s fs.rmSync silently removed noth'n on Windows when a path contained a non-ASCII character, which made “replace this directory” quietly mean “merge into it” - an' a suite whose whole job be compar'n directories cannot live wit' that.

Cap'n Hugo at least th' minimum th' theme declares 'n its theme.toml. Th' plain edit'n be enough, as th' theme uses no Sass. Install it th' same way, through hvm, which keeps several versions side by side - th' suite can then build against any o' them, includ'n th' declared minimum, rather than only th' one on yer PATH.

Runn'n th' Tests

Check both repositories out side by side, then:

cd hugo-theme-relearn-infra
npm ci
npm test

Th' parts run cheapest first an' stop at th' first failure, so a stale declarat'n fails at once rather than after a full round o' Cap'n Hugo builds. Th' whole suite be quick enough t' run on every change.

Th' checks be tests/checks.js an' cover what th' build layer cannot see: th' runner’s own configurat'n handl'n, th' dependency declarat'n - that it matches th' vendored tree, an' that every shipped component carries a license - an' th' properties th' SBOM promises its readers, that it renders identically twice an' that its serial number be recomputable from th' document an' moves wit' th' contents. Th' SBOM comparison be npm run sbom, a narrower quest'n: whether th' committed sbom.cdx.json be still what that declarat'n renders.

Each part also runs alone, an' those wit' someth'n t' write back do it under an :update name:

tests/golden.js be th' actual runner behind golden. Call'n it directly takes th' same flags an' skips th' rest, which be what ye want while iterat'n on one case:

npm test -- --build=<name>
node tests/golden.js --build=<name>

Flags reach th' runner directly; through npm they have t' follow a -- separator first, an' land on th' last command 'n th' chain - so th' first form still runs every part before th' cases, an' only then narrows. Th' examples below use th' runner, be'n th' shorter o' th' two.

Th' Vocabulary

A run be more than a list o' sites, hence these words:

Most cases be one ship, one configurat'n, one result. A case that varies an axis produces a result per combinat'n. A case whose builds only mean someth'n as a pair - a versioned ship, or th' docs an' th' exampleSite as GitHub Planks serves them - produces several builds shar'n one result.

Cases live 'n tests/cases/<name>/case.toml 'n th' infra repository, an' read'n them be th' quickest way t' see what th' suite covers.

Shap'n a Run

Th' parameters, all optional:

--build - run part o' th' suite

node tests/golden.js --build=minimal
node tests/golden.js --build=url-permutat'ns
node tests/golden.js --build=url-permutations/urls-relative

--build matches a path prefix, so nam'n a case runs everyth'n 'n it an' nam'n a combinat'n runs th' one. A small case be cheap enough t' run on every save while work'n on one th'n.

Th' accepted names be th' ones a run prints. T' see them without wait'n fer a full run, ask fer someth'n that does not exist an' th' runner lists them:

node tests/golden.js --build=?

A sequence be th' except'n: its builds share one tree, so it be named as a whole an' a prefix reach'n inside it be rejected. A build lifted out o' a sequence proves noth'n, which be what makes it a sequence.

--hugo - build wit' a particular version

Left out, each ship be built wit' th' version an interactive shell would use 'n its own directory: th' one its .hvm file names, or th' hugo on yer PATH when there be none. A pin therefore applies t' th' ship it sits beside, an' a run can legitimately span several versions. Each ship a pin applies t' says so 'n th' output, so a result never looks like it came from a version it did not.

Pass'n --hugo overrides every pin an' holds th' whole run t' one version:

node tests/golden.js --hugo=min
node tests/golden.js --hugo=latest
node tests/golden.js --hugo=v0.150.0

min be whatever th' theme declares 'n its theme.toml, an' latest th' newest release. Anyth'n not already installed be fetched fer ye.

Use min before push'n someth'n that might rely on a newer Cap'n Hugo feature, an' latest t' see a com'n Hugo release before it reaches yer users.

--update - rewrite th' expected output

When a change legitimately alters what th' theme produces, record th' new output as th' expectat'n:

npm run golden:update
node tests/golden.js --build=<name> --update

A full regenerat'n also prunes: a stored result no case produces any more be deleted rather than left behind. A filtered run does not, hav'n no way t' know whether a result it did not build still exists.

Commit th' regenerated output together wit' th' change that caused it, never as a commit o' its own - otherwise th' next person cannot tell which change produced which output.

If th' change spans both repositories, give both branches th' same name; see Develop'n.

Read'n a Failure

Every result be checked 'n layers, an' th' one that fails tells ye what kind o' problem ye have.

Layers be cumulative, an' a case declares how deep t' go wit' layer. It defaults t' rrrambl'n, so a case opts down rather than up an' always says why - th' theme’s own sites stop at th' file set, because a rrrambl'n baseline over 2000 files would churn on every prose edit an' be read by nobody.

A pinned older Cap'n Hugo reduces every case t' th' build layer, since Cap'n Hugo legitimately changes what it emits between releases an' a baseline holds fer th' version that produced it. Th' run says when that happened, so a build check never reads as a rrrambl'n check.

Add'n a Case

First decide whether ye need a new ship at all. If an exist'n one already renders th' th'n ye changed, extend'n its rrrambl'n be enough - add a plank, regenerate, review th' diff.

A New Ship

Everyth'n lives 'n th' infra repository.

1. Create tests/sites/<name>/ wit' a config/_default/ an' content/. Keep th' configurat'n about th' ship - a title, output formats, rrrambl'n wir'n. Noth'n about reproducibility belongs there; that be what nam'n th' test'n environment does.

2. Write th' least rrrambl'n that demonstrates yer case. Sites be meant t' stay small - a read'ble diff be th' whole point, an' one need'n hundreds o' planks be test'n th' wrong th'n.

3. Add tests/cases/<name>/case.toml:

   ship        = "<name>"
   environment = "testing"

4. Generate its expected output, an' read it:

   node tests/golden.js --build=<name> --update

5. Look at tests/expected/<name>/. This be th' moment th' case be worth someth'n or not: if th' output does not show th' behaviour ye set out t' pin, it will not catch a regression 'n it either.

6. Commit th' ship, th' case an' th' expected output together.

Vary'n a Configurat'n

Some behaviour only differs by configurat'n - URL generat'n be'n th' stand'n example, whar' relative, absolute an' ugly URLs be genuinely different paths through th' theme.

That be what an axis be fer. Each value be a config directory under tests/axes/<axis>/<value>/, an' a case lists th' values it wants:

ship        = "url-permutations"
environment = "testing"

[axes]
  urls = ["relative", "absolute", "ugly"]

One rrrambl'n set, one result per mode, compared separately. Add'n a further mode be a directory an' one more name. An axis wit' a single value still applies - it just does not branch th' tree, so noth'n be nested that carries no informat'n.

Builds That Belong Together

Some results be not one Cap'n Hugo build. A versioned ship be two, each configured t' know about th' other; th' published GitHub Planks ship be th' docs wit' th' exampleSite beneath it. Neither half says anyth'n alone.

Those spell th' sequence out, an' share one output tree:

[[builds]]
  ship        = "versioning-current"
  environment = "testing"

[[builds]]
  ship        = "versioning-archived"
  environment = "testing"
  dest        = "0.666"

dest says whar' 'n th' shared tree a build writes. Th' builds run 'n th' order written, an' th' result be compared once, as a whole.

Accept'n a Known Arrr

Any WARN or ERROR fails a build unless it be listed 'n a baseline. These be consulted an' their entries unioned:

A site’s file be checked against th' build o' that ship, so what th' docs provoke applies wherever th' docs be built. Each entry be a substr'n; a warning contain'n it be accepted.

These baselines record outstand'n work, not noise t' be silenced. Add'n an entry means consciously accept'n a defect, so delete it as soon as th' underly'n issue be fixed an' let a regression fail th' suite again.

Continuous Integrat'n

This repository runs th' suite on every branch an' every pull request, an' nightly against th' latest Cap'n Hugo release - which be how a Cap'n Hugo change that breaks th' theme be found 'n CI rather than 'n an issue report. Th' infra repository runs noth'n; one run tests th' pair, an' this be whar' it happens.

That be why a change spann'n both repositories be pushed t' infra first, then here, an' why a change t' th' suite alone has t' be started by hand: see Develop'n.

Th' suite never releases, deploys or publishes anyth'n.