Testin'

Th' test suite lives 'n th' infra repository an' runs against a theme checkout. See Develop'n fer how th' two be wired together.

Requirements

Node.js at th' version .nvmrc pins. Install it through a version manager - nvm, or nvm-windows - rather than as a system package, so th' version can follow th' project rather than th' machine. nvm use 'n th' infra checkout reads that file, an' so does CI, so th' two cannot drift.

Th' pin be not arbitrary. Before v26.8.1, Node’s fs.rmSync silently removed noth'n on Windows when a path contained a non-ASCII character, which made “replace this directory” quietly mean “merge into it” - an' a suite whose whole job be compar'n directories cannot live wit' that.

Cap'n Hugo at least th' minimum th' theme declares 'n its theme.toml. Th' plain edit'n be enough, as th' theme uses no Sass. Install it th' same way, through hvm, which keeps several versions side by side - th' suite can then build against any o' them, includ'n th' declared minimum, rather than only th' one on yer PATH.

Runn'n th' Tests

Check both repositories out side by side, then:

cd hugo-theme-relearn-infra
npm ci
npm test

That checks th' runner itself, then builds every case, an' takes well under a minute.

npm test be a wrapper around tests/run.js, which be th' actual runner. Either form works - these two be th' same command:

npm test -- --build=<name>
node tests/run.js --build=<name>

Flags reach th' runner directly; through npm they have t' follow a -- separator first. Th' examples below use th' runner, be'n th' shorter o' th' two.

Th' Vocabulary

Five words, because a run be not simply a list o' sites any more.

Term	Mean'n
Ship	rrrambl'n plus th' configurat'n it needs t' be itself
Axis	a dimension o' configurat'n, each o' whose values be a config directory
Case	what t' build, an' how deeply t' check it
Build	one Cap'n Hugo invocat'n - one ship, one configurat'n
Result	one output tree, compared as a whole

Most cases be one ship, one configurat'n, one result. A case that varies an axis produces a result per combinat'n. A case whose builds only mean someth'n as a pair - a versioned ship, or th' docs an' th' exampleSite as GitHub Planks serves them - produces several builds shar'n one result.

Cases live 'n tests/cases/<name>/case.toml 'n th' infra repository, an' read'n them be th' quickest way t' see what th' suite covers.

Shap'n a Run

Three parameters, all optional:

Parameter	Selects	Default
`--build`	which builds t' run	all o' them
`--hugo`	which Cap'n Hugo t' build wit'	each site’s own, see below
`--update`	rewrite th' stored output instead o' compar'n against it	compare

`--build` - run part o' th' suite

node tests/run.js --build=minimal
node tests/run.js --build=url-permutat'ns
node tests/run.js --build=url-permutations/urls-relative

--build matches a path prefix, so nam'n a case runs everyth'n 'n it an' nam'n a combinat'n runs th' one. A small case takes about a second, which be cheap enough t' run on every save while work'n on one th'n.

Th' accepted names be th' ones a run prints. T' see them without wait'n fer a full run, ask fer someth'n that does not exist an' th' runner lists them:

node tests/run.js --build=?

A sequence be th' except'n: its builds share one tree, so it be named as a whole an' a prefix reach'n inside it be rejected. A build lifted out o' a sequence proves noth'n, which be what makes it a sequence.

`--hugo` - build wit' a particular version

Left out, each ship be built wit' th' version an interactive shell would use 'n its own directory: th' one its .hvm file names, or th' hugo on yer PATH when there be none. A pin therefore applies t' th' ship it sits beside, an' a run can legitimately span several versions. Each ship a pin applies t' says so 'n th' output, so a result never looks like it came from a version it did not.

Pass'n --hugo overrides every pin an' holds th' whole run t' one version:

node tests/run.js --hugo=min
node tests/run.js --hugo=latest
node tests/run.js --hugo=v0.150.0

min be whatever th' theme declares 'n its theme.toml, an' latest th' newest release. Anyth'n not already installed be fetched fer ye.

Use min before push'n someth'n that might rely on a newer Cap'n Hugo feature, an' latest t' see a com'n Hugo release before it reaches yer users.

`--update` - rewrite th' expected output

When a change legitimately alters what th' theme produces, record th' new output as th' expectat'n:

node tests/run.js --update
node tests/run.js --build=<name> --update

A full regenerat'n also prunes: a stored result no case produces any more be deleted rather than left behind. A filtered run does not, hav'n no way t' know whether a result it did not build still exists.

Commit th' regenerated output together wit' th' change that caused it, never as a commit o' its own - otherwise th' next person cannot tell which change produced which output.

Arrr

Th' result'n diff be th' test result. Read it before committ'n. An unreviewed regenerat'n turns th' suite from a safety net into a rubber stamp.

If th' change spans both repositories, give both branches th' same name; see Develop'n.

Read'n a Failure

Every result be checked 'n three layers, an' th' one that fails tells ye what kind o' problem ye have.

Layer	Asserts	A failure usually means
Build	th' build exits cleanly, wit' no unexpected `WARN` or `ERROR`	a template error, or a Cap'n Hugo deprecat'n
File set	exactly th' expected files were generated	output formats, permalinks or a renamed plank
Rrrambl'n	every file be what was stored, byte fer byte bar line end'ns	either a regression, or a change ye meant t' make

Layers be cumulative, an' a case declares how deep t' go wit' layer. It defaults t' rrrambl'n, so a case opts down rather than up an' always says why - th' theme’s own sites stop at th' file set, because a rrrambl'n baseline over 2000 files would churn on every prose edit an' be read by nobody.

A pinned older Cap'n Hugo reduces every case t' th' build layer, since Cap'n Hugo legitimately changes what it emits between releases an' a baseline holds fer th' version that produced it. Th' run says when that happened, so a build check never reads as a rrrambl'n check.

Add'n a Case

First decide whether ye need a new ship at all. If an exist'n one already renders th' th'n ye changed, extend'n its rrrambl'n be enough - add a plank, regenerate, review th' diff.

A New Ship

Everyth'n lives 'n th' infra repository.

Create tests/sites/<name>/ wit' a config/_default/ an' content/. Keep th' configurat'n about th' ship - a title, output formats, rrrambl'n wir'n. Noth'n about reproducibility belongs there; that be what nam'n th' test'n environment does.
Write th' least rrrambl'n that demonstrates yer case. Sites be meant t' stay small - a read'ble diff be th' whole point, an' one need'n hundreds o' planks be test'n th' wrong th'n.

Add tests/cases/<name>/case.toml:

ship        = "<name>"
environment = "testing"

Generate its expected output, an' read it:
```
node tests/run.js --build=<name> --update
```
Look at tests/expected/<name>/. This be th' moment th' case be worth someth'n or not: if th' output does not show th' behaviour ye set out t' pin, it will not catch a regression 'n it either.
Commit th' ship, th' case an' th' expected output together.

Vary'n a Configurat'n

Some behaviour only differs by configurat'n - URL generat'n be'n th' stand'n example, whar' relative, absolute an' ugly URLs be genuinely different paths through th' theme.

That be what an axis be fer. Each value be a config directory under tests/axes/<axis>/<value>/, an' a case lists th' values it wants:

ship        = "url-permutations"
environment = "testing"

[axes]
  urls = ["relative", "absolute", "ugly"]

One rrrambl'n set, three results, compared separately. Add'n a further mode be a directory an' one more name. An axis wit' a single value still applies - it just does not branch th' tree, so noth'n be nested that carries no informat'n.

Builds That Belong Together

Some results be not one Cap'n Hugo build. A versioned ship be two, each configured t' know about th' other; th' published GitHub Planks ship be th' docs wit' th' exampleSite beneath it. Neither half says anyth'n alone.

Those spell th' sequence out, an' share one output tree:

[[builds]]
  ship        = "versioning-current"
  environment = "testing"

[[builds]]
  ship        = "versioning-archived"
  environment = "testing"
  dest        = "0.666"

dest says whar' 'n th' shared tree a build writes. Th' builds run 'n th' order written, an' th' result be compared once, as a whole.

Accept'n a Known Arrr

Any WARN or ERROR fails a build unless it be listed 'n a baseline. Three be consulted an' their entries unioned:

File	Holds
`tests/warnings.txt`	theme-wide, mostly Cap'n Hugo deprecat'ns
`tests/sites/<site>/warnings.txt`	what a site’s own rrrambl'n provokes
`tests/cases/<case>/warnings.txt`	what a configurat'n provokes

A site’s file be checked against th' build o' that ship, so what th' docs provoke applies wherever th' docs be built. Each entry be a substr'n; a warning contain'n it be accepted.

These baselines record outstand'n work, not noise t' be silenced. Add'n an entry means consciously accept'n a defect, so delete it as soon as th' underly'n issue be fixed an' let a regression fail th' suite again.

Continuous Integrat'n

This repository runs th' suite on every branch an' every pull request, an' nightly against th' latest Cap'n Hugo release - which be how a Cap'n Hugo change that breaks th' theme be found 'n CI rather than 'n an issue report. Th' infra repository runs noth'n; one run tests th' pair, an' this be whar' it happens.

That be why a change spann'n both repositories be pushed t' infra first, then here, an' why a change t' th' suite alone has t' be started by hand: see Develop'n.

Th' suite never releases, deploys or publishes anyth'n.