Status

Early, and saying so.

0.8.0

Derived seeds are 53 bits, and lock_format is 5. A 64-bit seed is not exact as a JSON number where numbers are doubles. A plan frozen again gets a different hash and different seeds. A lock already frozen runs as it did. See Freeze and the lock.

The egress proxy image is pinned to a digest, and environment.json records it as proxy_digests. It ran as latest before.

TOUCHSTONE_PROXY_DNS names the resolvers the proxy asks, for networks that block 1.1.1.1 and 8.8.8.8. See Containment.

A container name another run holds is left alone. A second run of the same plan on one host removed the first run's container. It now fails and says the name is taken.

verify checks the recorded size, with the code file_size_mismatch.

A bundle sealed by 0.7.0 verifies as it did. touchstone_version stamps 0.8.0.

$ touchstone version
touchstone 0.8.0

0.7.0

A release of fixes to failure paths. Four change what a command accepts.

run refuses a directory that already holds a run. A second run appended to the first ledger, so one that died part way still sealed as complete. Run again into a new directory.

Item records and estimates refuse NaN and infinity. A non-finite score was written into estimates.json and graded as a figure.

verify fails on a symlink, with the code symlink. A recorded file swapped for a link to the same bytes used to verify.

Docker failing to start a pack is a failed unit. Exit 125 was recorded as the pack's own exit code. The ledger now records unit_failed with Docker's message.

A row that is not a JSON object is left out and counted. The ledger records rows_unreadable and the run reports it as a failure. It used to raise after every unit had finished.

Files are read as UTF-8 on every host, and one that is not UTF-8 is named, with the code not_utf8. A permission or path error is one line with the code os_error.

Also fixed: an empty PLAN.sha256, a ledger line that is not an event, --resamples 0, a --by key named twice, pack output that is not UTF-8, and a .nan plan parameter.

A bundle sealed by 0.6.0 verifies as it did. touchstone_version stamps 0.7.0.

0.6.0

estimates.json carries an uncertainty budget. Each whole-sample figure lists its sampling uncertainty, split into completions and items where the plan ran replicates, and names marking, item selection, item leakage and endpoint identity as unquantified. See Uncertainty and assumptions.

estimates.json records the assumptions behind the estimators. The functional form is tested across replicates with Pearson's chi-square, and unidimensionality and item independence are recorded as not checked.

report prints both. The assumption item is now met on any bundle estimate writes. Item 3.1.4 now needs the budget, so a bundle estimated by 0.5.0 fails it until it is estimated again with --out.

A bundle sealed by 0.5.0 verifies as it did. touchstone_version stamps 0.6.0.

0.5.0

touchstone report states what a bundle holds against NIST AI 800-2, as a PDF. Ten practice items, each met, not met or not applicable, with every figure read from estimates.json and scorecard.json. The PDF is written beside the bundle, and the same bundle sets the same bytes. See Reporting.

environment.json records what a run cost. Wall time per unit, and each pack's per-row cost summed where the pack reports one.

A score card can grade movement with an interval. source: paired_difference pairs items when both bundles share a plan hash and item ids, and otherwise combines the two stored intervals and records why. See Comparing to a prior bundle.

estimate and grade refuse to write into a sealed bundle. A script that re-graded a sealed bundle in place now exits 1 with the code bundle_sealed. Pass --out to write the analysis beside it. See Re-analysis.

validate finds packs/ from the plan's location when the current directory has none, so the quick start works from any directory.

Grade reasons are plain sentences and end on the grade they explain. Under a ceiling that narrows an inconclusive range, the reason names the narrowed range, in the terminal and in the PDF. A ceiling that leaves a range as it was is left out of scorecard.json, so a re-grade can report fewer capped indicators.

A bundle sealed by 0.4.0 verifies as it did, and report reads its grade reasons the same way. touchstone_version stamps 0.5.0.

0.4.0

A plan or a pack manifest carrying a key the contract does not declare now fails to load. Six nested models accepted anything: a packs entry and a systems entry in a plan, and the input_systems, input_schema, strata and network declarations in a manifest. The promise that an unknown key raises rather than being dropped held at the top of a plan and nowhere inside it, which is where most of the keys are. A misspelled replicates ran the pack once and reported a rate with no replicate variance beside it, and a misspelled egress was a pack asking for the network and silently getting none. A pack image whose manifest has such a key now fails at freeze rather than running with the wrong declaration.

validate, verify, estimate and grade take --json. One envelope on stdout, carrying a stable code, the file, and the line and column for each problem, so a CI job or an editor branches on an identifier rather than matching prose. The exit codes are unchanged and the human output is unchanged. An indeterminate grade arrives as a warning and ok stays true. See Machine-readable output.

Three functions in src/ changed shape. plan_check.check, bundle.verify and grade.check return list[Problem] rather than list[str], and each takes an optional path that fills in the position. Anything importing this package rather than running the commands reads .message where it used to read the string.

The four files a person writes have published schemas. Generated from the same pydantic models validate uses and served with this site, so an editor with a YAML language server gives completion, hover text and inline validation that cannot drift from what the tool enforces. --json output has one too. See Editor schemas.

No bundle changes shape, and no hash of an unchanged bundle moves. touchstone_version stamps 0.4.0.

$ touchstone version
touchstone 0.4.0

0.3.0

Intervals over repeated items are wider, and every bundle with replicates above one holds different numbers than 0.2.1 computed from the same rows. Asking an item twice does not give two independent observations of the system, but the rollup counted one row per item-trial and gave that denominator to Wilson. Over 200 items a nominal 95 percent interval held 95 percent of the time at one replicate and 54 percent at twenty, narrowing with every replicate added while the rate it covers stayed where it was. The interval is now computed over items, an outcome with repeated items reports wilson_clustered and carries effective_n and design_effect, and the bootstrap resamples items rather than rows. A run with one replicate per item is unchanged. See Rates and Wilson.

Two smaller corrections travel with it. The printed rate is the interval the bundle stores rather than one recomputed from k and n, which would have disagreed with the file as soon as an interval was widened. And the confident-and-wrong rate counts an item once rather than once per replicate.

Re-read a bundle rather than comparing a 0.2.1 estimate to a 0.3.0 one. The rows are untouched and estimate recomputes from them, so an old bundle can be brought forward.

Still classified Development Status :: 2 - Pre-Alpha on PyPI, which is accurate.

0.2.1

A container the kernel kills for exceeding its memory cap is recorded as out_of_memory from its exit code. State.OOMKilled, which decided it before, is written from an event containerd delivers and on cgroup v2 that event is sometimes never delivered, so a run the harness starved could reach a bundle looking like the pack's own exit code. See Resource limits.

0.2.0

estimate given more than one --by key rolls up each key on its own as well as crossing them, so a card that asks about one dimension per indicator reads its cells off a single run. grade refuses a worst stratum that names no keys where the bundle holds cells sitting inside other cells, rather than ranking a group against part of itself.

0.1.0

The first release that is the code this documentation describes. All seven commands work and are tested doing it.

What is settled

• The bundle format. MANIFEST.json, the hash over the canonical file list, and the refusal to seal an incomplete ledger.
• The item record. One row per item, no scores, pack_id stamped by the harness.
• The statistics. Wilson for rates, seeded BCa for scores, ECE against a declared outcome, between-replicate variance. The arithmetic is pinned by tests.
• Containment. Internal network, proxy sidecar, no TLS termination, declared hosts only.
• The pack contract. manifest.yaml at /app, results to /output, stdout is logs.

What is not settled

The score card format

grade applies whatever ladder the card gives it rather than one built in, and examples/scorecard.yaml shows the shape rather than a rubric anyone should adopt.

The condition vocabulary, the shape of MetricRef, and how per-indicator ceilings are expressed may all change. If you are writing cards now, expect to migrate them.

The package is touchstone-dqi because touchstone was taken on PyPI. DQI is the deployment quality index this is being built to carry, which is separate work and is not published, so nothing here grades against it.

Known gaps

Gap	Notes
example_pack is not published	Every hash in the tutorial is specific to the machine that built the image.
No backend registration	A third-party backend means a fork. A plugin entry point is planned.
One backend	docker only.
No trace viewer	trace_ref points into the bundle; reading it is your text editor's job.
Anchors need manual upgrade	ots upgrade is a step somebody has to remember. See Anchoring.

Bundles sealed before 0.2.1

A bundle from 0.1.0 or 0.2.0 verifies exactly as it did. Two things in one are worth knowing before quoting what it says.

Sealed by	Reads
0.1.0	touchstone_version says 0.0.1. That release bumped pyproject.toml and left the string the code stamps, so a bundle reading 0.0.1 came from 0.0.1 or from 0.1.0.
0.1.0 or 0.2.0	A unit the kernel killed for exceeding its memory cap can carry exit_code: 137 with termination null, which reads as the pack exiting 137 on its own. See Resource limits.

Neither touches a hash, and neither can be corrected in a bundle that is already sealed. Re-running under 0.2.1 is what settles a 137 in an old one.

What is tested

uv run pytest -q

Some of it is worth naming, because it is the kind of testing that decides whether the claims on this site hold:

• The offline claim. CI installs the package with the network switched off and runs it, on every change.
• The published figures. tests/test_estimate_credential.py reproduces the flood-warning numbers from item records, to the last bit of both bounds.
• The bootstrap arithmetic. Pinned to fixed values, so a Python upgrade that moved an interval fails the build rather than the client.
• This documentation. The console blocks in the README are executed, and every command in the pipeline table is checked to exist.

Versioning

No stability guarantee before 1.0. The lock_format and bundle_format integers are how a format change announces itself; a bump changes the hash of an unchanged plan, which is the point.