ADR 0011: Guardrail catalogue source and artifact structure
- Status: Proposed
- Date: 2026-10-03
- Decision owners: architecture and catalogue maintainers
- Approval also needed from: security/platform for registry trust; privacy/legal for provenance and redistribution; the relevant governance authority for culturally sensitive material
- Extends: ADR 0004 (policy bundles and immutable snapshots), ADR 0005 (detector extension model), and ADR 0008 (Indigenous Data Governance)
Context
Groundskeeper needs a catalogue that can grow from a small first-party set into many independently maintained packages without making the evaluator resolve a registry, source tree, or dependency graph during a request. The catalogue will contain:
- generic and jurisdiction-specific PII definitions, including Australian Commonwealth and state/territory variants;
- credential and secret definitions grouped by provider or issuer;
- safety, jailbreak, prompt-injection, and other security checks;
- reusable policy bundles;
- schemas, small conformance fixtures, large or restricted corpora, source provenance, generated indexes and documentation; and
- eventually, third-party definitions and detector implementations.
The existing architecture already separates detector observations from policy decisions, policy source from immutable runtime snapshots, and built-in detector code from future gRPC/WASM extensions. This ADR preserves those boundaries. A catalogue definition describes a concept and the evidence needed to recognize it. A detector implements recognition. A policy decides what to do with a finding. None of those identities are interchangeable.
Decision summary
- Use independently versioned catalogue packages as the ownership, release, licensing, and distribution boundary. Begin with a few coherent packages; do not create one package per rule or one global monolith.
- Organize paths for browsing, never for identity. Domain is the first browsing axis. Geography is used only where meaningful; provider is the equivalent axis for credentials.
- Author one YAML file per coherent definition family by default. Split a family when ownership, provenance, licence, schema, or release cadence differs. A file may contain one item when the item is independently complex.
- Give every object a permanent, globally namespaced ID of the form
<verified-publisher>/<kind>/<name>. Keep Groundskeeper's public dotted finding taxonomy ascategory; do not use source paths or detector-private labels as public IDs. - Keep existing repository authorities. Catalogue definitions live under
catalog/; public policy bundles remain underpolicies/bundles/; canonical schemas remain underapi/schemas/; cross-package and governed evaluation data remain undertest/evaldata/. The catalogue links to these locations rather than cloning them. - Compile and validate off-path. Package releases produce canonical JSON, an exact dependency lock, source/licence inventory, generated index, tests and release evidence. Production activation consumes one flattened, immutable snapshot pinned by OCI digest.
- Publish packages and deployment snapshots as distinct OCI artifacts. SemVer tags aid discovery. Digests identify dependencies and activations. Signatures, provenance, release evidence, and SBOM/licence inventories are OCI referrers.
- Use schema version, package version, item lifecycle, and evidence quality as separate dimensions. Do not add a second independently resolved SemVer to every definition.
- Third parties publish from their own verified namespace. Namespace verification, signatures, and provenance prevent identity collisions and establish origin; conformance and trust policy determine whether Groundskeeper will install the content.
What mature ecosystems teach us
No surveyed ecosystem solves the whole problem. Groundskeeper deliberately combines their strongest properties and avoids their weak catalogue boundaries.
| Ecosystem | Pattern worth adopting | Limitation not to copy |
|---|---|---|
| OpenTelemetry semantic conventions | Typed YAML is authoritative; narrative and registry pages are generated; definitions have stable fully qualified names, stability, area owners, deprecation, and versioned migrations. | One centrally released registry is too coarse for unrelated Groundskeeper publishers and licences. |
| Semgrep | Language/framework/category paths are discoverable; rule tests are colocated and require both true-positive and true-negative cases. | A local rule slug plus registry path is not a sufficiently durable global identity; the hosted registry must not be the only reproducible index. |
| Sigma | A rule has a durable UUID, lifecycle status, dates, references, licence, and typed relationships such as derived, obsolete, merged, and renamed; deprecated rules remain auditable. | UUID-only discovery is poor, and arbitrary extension fields without a closed core profile reduce interoperability. |
| YARA/YARA-X | Rule files can group related rules; tags, metadata, includes, modules, and compiler namespaces support flexible collections. | The language deliberately supplies little catalogue governance: no universal package manifest, lifecycle, provenance, or test layout. |
| Gitleaks | Provider-oriented authoring generates one runtime configuration; each rule has positive and negative examples; generated files are not edited by hand. | One generated global configuration and shallow override precedence do not provide independent package release or strong provenance. |
| Microsoft Presidio | Generic, country-specific, NER, and third-party recognizers are separated; entity, language/country, implementation, context, and confidence are distinct; recognizer tests verify offsets and scores. | Python classes/names and a loading manifest are not stable cross-implementation catalogue contracts. |
| OPA bundles | A package has a manifest, owned roots, language version, tests, signatures, and an all-or-nothing activation unit; schema paths are versioned. | Runtime path-based data roots and unordered multi-bundle merging are unsafe substitutes for explicit, locked composition. |
| OCI/package registries | Package coordinates are namespaced, immutable content is digest-addressed, and signatures/provenance can be attached as referrers. Maven Central demonstrates verified DNS or source-host namespaces. | Tags are mutable discovery handles, not deployment identity; a valid signature alone does not establish safety. |
These comparisons lead to a T-shaped hierarchy: shallow, predictable package and domain structure for broad discovery, with additional depth only for an axis that changes ownership or meaning. Deeply mirroring every taxonomy component in the filesystem would create empty directories and brittle moves without adding governance.
Source tree
The target repository layout is:
catalog/
README.md
namespaces.yaml # first-party namespace claims; registry seed
owners.yaml # package areas and required reviewer roles
packages/
groundskeeper.dev/
pii-core/
package.yaml
definitions/
contact-identifiers.yaml
network-identifiers.yaml
payment-card.yaml
sources.yaml
tests/
contact-identifiers.cases.yaml
README.md
pii-au/
package.yaml
definitions/
national/
tax-identifiers.yaml
health-identifiers.yaml
social-services-identifiers.yaml
business-identifiers.yaml
subdivisions/
au-act/driver-licence.yaml
au-nsw/driver-licence.yaml
au-nt/driver-licence.yaml
au-qld/driver-licence.yaml
au-sa/driver-licence.yaml
au-tas/driver-licence.yaml
au-vic/driver-licence.yaml
au-wa/driver-licence.yaml
sources.yaml
tests/
README.md
secrets/
package.yaml
definitions/
aws.yaml
azure.yaml
github.yaml
stripe.yaml
generic.yaml
sources.yaml
tests/
README.md
security-prompt/
package.yaml
definitions/
prompt-injection.yaml
jailbreak.yaml
tool-abuse.yaml
sources.yaml
tests/
README.md
safety-core/
package.yaml
definitions/
sources.yaml
tests/
README.md
api/
schemas/
catalog/
v1alpha1/
package.schema.json
definition-set.schema.json
source.schema.json
test-case.schema.json
index.schema.json
lock.schema.json
policies/
bundles/ # sole source authority for policy bundles
examples/
test/
conformance/ # execution-mode-neutral detector contract tests
evaldata/
manifests/ # corpus metadata, access and digest manifests
public/ # redistributable corpus partitions only
docs/
catalog/ # generated, committed browsing documentation
decisions/
dist/ # generated, ignored; never hand editedWhy packages, and when to split one
A package is the smallest independently published unit. Everything in one package shares:
- version and changelog;
- maintainers and required approval policy;
- default licence and redistribution posture;
- dependency lock and compatibility range;
- OCI artifact and release cadence.
Create a new package when one of those properties materially diverges. Do not split only because another directory level is possible. For example:
- Start with
pii-au, not nine packages for Australia. Split a state package only if an authority, licence, owner, or change cadence becomes independent. - Start with
secrets, with one family file per provider. Splitsecrets-awswhen provider-specific ownership or release urgency makes the larger package an operational bottleneck. - Keep safety and prompt-security packages separate because their evidence, reviewers, corpora, and model dependencies differ even if both inspect text.
- Never put executable detector binaries in a definition package. A detector package has the separately defined detector manifest and release lifecycle.
This rule gives independent versioning where it has value without recreating the dependency and review overhead of a package per item.
File granularity
The default is one file per coherent family, usually one to roughly ten definitions. Keep variants together when they share the same semantics, validator, sources, maintainers, tests, and release cadence. Examples include the related Australian healthcare identifiers IHI, HPI-I, HPI-O, and HSP-O.
Split definitions when any of the following is true:
- a source, licence, access classification, or approver differs;
- an item has substantial independent rationale or test data;
- reviewers routinely change one subset without understanding the other;
- merge contention becomes material; or
- the family would mix unrelated validators merely to reduce file count.
Do not enforce one-file-per-item mechanically. It creates hundreds of tiny files with duplicated provenance and obscures families. Do not use one file per whole jurisdiction or package either; those become merge and ownership hotspots.
Geography, authority, language, and provider
Paths represent the dominant browsing axis, while metadata represents all axes:
- generic PII is in
pii-core; - country-specific PII uses ISO 3166-1 alpha-2 in package names and metadata;
- subdivisions use ISO 3166-2, lower-case in paths (
au-nsw) and canonical upper-case in metadata (AU-NSW); national/means applicable across Australia. Whether the issuer is a Commonwealth, state/territory, private, or other authority is a separateissuingAuthority.levelfield. This avoids treating “national” and “federal” as competing geographic levels;- language and cultural applicability are explicit metadata, not nested under a country unless the definition is actually jurisdiction-bound;
- credentials use provider/issuer as their browsing axis, not country; and
- a provider, country, language, entity, implementation, and policy consequence remain independent metadata dimensions.
No definition may infer Indigeneity from a name, language, location, embedding, or correlation. Relevant community material requires the authority, access, withdrawal, and review process specified by the Indigenous Data Governance ADR.
Identity and naming
Coordinates and IDs
The following identities are distinct:
| Identity | Example | Purpose |
|---|---|---|
| Publisher namespace | groundskeeper.dev | Verified ownership and collision boundary |
| Package coordinate | groundskeeper.dev/pii-au | Unit of source ownership, SemVer release, and OCI publication |
| Object ID | groundskeeper.dev/definition/privacy.pii.tax_id.au.tfn | Permanent audit/reference identity |
| Public finding category | privacy.pii.tax_id.au.tfn | Stable interoperable taxonomy reported in findings |
| Detector ID | groundskeeper.dev/detector/pii-regex-checksum | Implementation identity from the detector contract |
| Policy/rule ID | groundskeeper.dev/policy-rule/block-validated-tfn | Override and audit target |
| Schema API version | groundskeeper.dev/v1alpha1 | Authoring document contract |
| Package version | 1.4.0 | Human compatibility/discovery version |
| OCI digest | sha256:… | Exact immutable package or snapshot identity |
Object IDs use lower-case ASCII and the grammar:
<verified-publisher>/<kind>/<dot-separated-name>The publisher may be a verified DNS name (example.com) or an approved
source-host namespace (github.com/example). Core kinds are definition,
detector, validator, policy-bundle, policy-rule, schema, and corpus.
Names use lower-case [a-z0-9][a-z0-9_-]* components separated by dots. IDs are
immutable, path-independent, and never reused.
The public category remains the existing hierarchical taxonomy. A third-party
detector may emit a Groundskeeper category without claiming ownership of it. A
new third-party concept uses its fully namespaced object ID and may propose a
separate mapping into the public taxonomy. Detector-private labels such as
AU_TFN are aliases only.
Human-readable IDs are preferable to UUIDs for discovery, but Sigma demonstrates
the value of an identity that survives renames and forks. Groundskeeper obtains
the same collision resistance from the verified publisher prefix and immutability.
Every compiled item also receives a generated content digest. A fork that changes
semantics must mint an ID in the fork's namespace and record derivedFrom; an
exact mirror preserves upstream IDs and digest.
Namespaces for third parties
The public registry must verify DNS control with a TXT challenge or source-host account control before granting publish rights, following Maven Central's namespace model. Publication additionally requires an expected signing identity.
- Losing a domain does not transfer existing IDs automatically.
- Namespace transfer requires an auditable registry action, old/new owner approval where possible, a notice period, and signer rotation.
- An abandoned namespace is frozen, not recycled.
- A package move records old and new coordinates; object IDs remain unchanged.
- Mirrors cannot become authoritative merely by republishing an artifact.
- Private registries may allocate internal namespaces but must include a DNS or organization authority that cannot collide when artefacts are exchanged.
The registry index is a discovery service, not the source of truth for artifact contents. Clients verify coordinate, manifest, digest, namespace claim, expected signer, and provenance independently.
Source documents and manifests
Package manifest
Each package has exactly one package.yaml:
apiVersion: groundskeeper.dev/v1alpha1
kind: CataloguePackage
metadata:
namespace: groundskeeper.dev
name: pii-au
version: 0.3.0
lifecycle: development
title: Australian identifier definitions
description: Generic Australian and state/territory identifier evidence
license: Apache-2.0
owners:
- team: pii-maintainers
requiredReviews:
- privacy
- australian-domain
compatibility:
catalogueSchema: ">=0.1.0 <0.2.0"
hostProtocol: ">=0.1.0 <0.2.0"
imports:
- package: groundskeeper.dev/pii-core
version: 0.2.1
exports:
roots:
- groundskeeper.dev/definition/privacy.pii
content:
definitions:
- definitions/**/*.yaml
sources: sources.yaml
tests:
- tests/**/*.cases.yamlThe compiler rejects unknown core fields. Vendor metadata is allowed only below
extensions.<verified-publisher> so typos in security-relevant fields do not
silently become ignored extensions.
Definition family
A family document contains independently addressable definitions and shared defaults; defaults are expanded during compilation and never act as runtime inheritance:
apiVersion: groundskeeper.dev/v1alpha1
kind: DefinitionSet
metadata:
package: groundskeeper.dev/pii-au
name: tax-identifiers
owners:
- team: pii-maintainers
defaults:
jurisdictions:
- country: AU
languages: [en]
definitions:
- id: groundskeeper.dev/definition/privacy.pii.tax_id.au.tfn
category: privacy.pii.tax_id.au.tfn
title: Australian Tax File Number
lifecycle: development
evidenceStatus: controlled
classification:
- personal-sensitive
- government-related
issuingAuthority:
level: commonwealth
name: Australian Taxation Office
recognition:
validationLevel: context-required
validator: groundskeeper.dev/validator/au-tfn
sources:
- au-ato-tfn-format
- au-oaic-tfn-guidance
tests:
- tests/tax-identifiers.cases.yamlKeep legal/privacy classification orthogonal to structural recognition. A valid ABN, BSB, public professional identifier, or infrastructure identifier is not automatically personal-sensitive or subject to redaction.
Source provenance
sources.yaml deduplicates citations within a package. Every source record has:
- stable package-local source key;
- publisher, title, document version or publication date;
- canonical URL and retrieval date;
- content digest and, where permitted, an archived or vendored snapshot path;
- authority class (
primary,standard,vendor,secondary, orresearch); - claims supported, limitations, and freshness/review date;
- SPDX licence expression where applicable, copyright holder, attribution, redistribution and commercial-use constraints;
- access class (
public,licensed,controlled, orrestricted); and - reviewer and approval reference for controlled algorithms or data.
Authoritative facts and heuristic choices are separate fields. A URL's public access does not imply permission to redistribute its content. Restricted algorithms, standards, model weights, and datasets are referenced by digest and approval metadata but are not copied into a public artifact unless their terms allow it.
Imports and composition
Packages are modules, not inheritance trees:
- Imports are explicit, package-level, exact SemVer references in reviewed source. Floating ranges are not permitted in committed release source.
- Publication resolves every import to an OCI digest and writes
lock.json. - Object references cross a package boundary by stable ID, never relative path.
- The complete import graph must be acyclic. Duplicate IDs, undeclared shadowing, incompatible export roots, ambiguous aliases, and missing references fail the build.
- Shared defaults may be expanded only inside one source family. There is no cross-package YAML merge, implicit inheritance, or last-writer-wins behavior.
extendsis permitted only for a schema-defined semantic refinement and must remain visible in compiled provenance. It is not a general merge operator.- Definitions can declare
requirescapabilities or validators. They cannot load executable code.
Policy bundles remain under policies/bundles/. They reference catalogue object
IDs and package digests and use ADR 0004's typed override rules. Catalogue
packages never embed tenant policy, allowlists, endpoints, credentials, trust
roots, or mandatory operator defaults. Provider-level false-positive knowledge
that is universally true may be detector evidence; tenant-specific suppression
is policy.
Ownership and review boundaries
catalog/owners.yaml is the structured source for generated area documentation
and CODEOWNERS sections. Each package has a primary and backup maintainer group,
required specialist roles, lifecycle (active, inactive, or
needs-maintainer), and escalation contact. Path ownership may narrow a package
for a jurisdiction or provider but cannot waive package-level approval.
Required reviews are determined by the changed claims:
| Change | Required review |
|---|---|
| Core schema, ID, compiler, package/export rule | Catalogue architecture maintainers |
| PII definition or classification | Domain maintainer plus privacy; jurisdiction expert for local claims |
| Controlled algorithm, standard, registry, or dataset | Domain maintainer plus legal/licence/data owner |
| Provider credential pattern | Secrets maintainer plus provider/source evidence reviewer |
| Prompt injection, jailbreak, or security rule | Security maintainer and threat-model reviewer |
| Safety definition/model/corpus | Safety owner plus risk/governance appropriate to the affected population |
| Indigenous names, languages, knowledge, or community material | Indigenous-led authority defined by ADR 0008; ordinary CODEOWNERS approval is insufficient |
| Public policy bundle | Policy owner plus security for weakening/override changes |
| Namespace or signing identity | Registry security/platform owner |
Inactive areas accept fixes, deprecations, and security responses but not feature expansion until ownership is restored, following OpenTelemetry's area model. Authoring, release approval, artifact publication, and production activation are separate roles. A maintainer cannot make an unreviewed artifact trusted merely by publishing it.
Tests and corpora
Colocated package tests
Small, redistributable, deterministic cases live in each package's tests/
directory. Every definition must include inputs that distinguish the intended
implementation from plausible wrong ones:
- true positives and near-miss true negatives;
- boundary lengths, malformed separators, case and Unicode variants;
- invalid and valid checksums when public algorithms exist;
- misleading context and known false positives;
- expected category, exact Unicode code-point span, validation state, confidence/evidence class, and coverage;
- synthetic/nonfunctional credential canaries, never live credentials;
- regression cases for fixed incidents; and
- declared engine/validator compatibility and performance budget.
This adopts Semgrep's same-basename proximity, Gitleaks' positive/negative rule validation, Presidio's offset/confidence checks, and Sigma's benign-baseline testing. Tests must assert semantic output, not only successful parsing.
Evaluation corpora
Large, shared, generated, licensed, private, or governed corpora do not sit next
to source definitions. test/evaldata/manifests/ contains versioned corpus
manifests with:
- globally namespaced corpus ID, version, partitions, and content hashes;
- source/generation method, seed, transformations, labels, and known limits;
- licence, consent/authority, permitted purpose and users;
- redistribution, commercial use, external-judge, cross-border, retention, storage, withdrawal, and deletion constraints;
- access location or retrieval procedure without embedding credentials;
- linked definition/package versions and expected metrics; and
- owner, review date, and incident/change history.
Only redistributable, non-sensitive content belongs in test/evaldata/public/.
Restricted material remains in its approved store; CI receives it only in an
authorized environment. Release evidence records corpus IDs and hashes, not raw
restricted content. Never use real identifiers merely because they are publicly
searchable. Checksum-valid synthetic identifiers are labelled as synthetic and
not guaranteed unissued.
Generated indexes and documentation
The compiler generates, in one deterministic operation:
- a repository index of package coordinates, versions, owners, objects, categories, jurisdictions, providers, languages, lifecycle/evidence status, licences, source freshness, dependencies, replacements and documentation;
- one documentation page per object plus package indexes, support matrices, dependency/replacement graphs, and changelog;
- expanded canonical JSON with defaults resolved;
lock.jsonwith source package version, OCI digest, object IDs/content digests, schema digest, compiler version, and transitive dependencies;- licence/notice inventory and source-provenance summary; and
- release test/coverage report.
docs/catalog/ and the machine-readable public index are committed for browsing
and reviewed through generated diffs. CI regenerates and fails on drift. dist/
is build output and is ignored. Generated files carry a “do not edit” header;
fixes occur in source, schemas, templates, or generator code. Narrative rationale
may be hand-authored in package READMEs, while factual tables are generated from
the catalogue as OpenTelemetry does.
Schema evolution and compatibility
There are four independent lifecycle axes:
apiVersionversions the YAML/JSON document schema.- Package SemVer versions the package's exported semantic contract.
lifecycledescribes an item:development,experimental,stable,deprecated, orunsupported.evidenceStatusdescribes source confidence/access:verified,provisional,controlled,obsolete, orrejected.
Do not give every definition an independently resolved SemVer. The package is the atomic tested unit, and the package digest plus stable object ID exactly identifies an item's semantics. Per-item SemVer would multiply dependency states without making items independently distributable. The generated index includes each item's content digest, introduction/deprecation package versions, and modification history for precise audit.
Before v1, v1alpha1 may change incompatibly and every release must declare
that fact. Once v1 is frozen:
- adding optional fields or new objects is schema/package minor;
- compatible documentation, source-freshness, and false-positive test fixes are patch when they do not alter output;
- expanding recognition, confidence, validation state, default enablement, or evidence in a way consumers observe is at least minor and requires decision diff review;
- removing/renaming a field, category, stable object, or changing its meaning is schema/package major;
- correcting unsafe behavior may disable an item immediately as
unsupported, but does not rewrite an already published artifact; and - released schemas and OCI digests are immutable.
Schemas use JSON Schema 2020-12 at immutable versioned URLs. Core objects are closed except for namespaced extension points. CI validates source against the current schema and prior stable releases for forbidden changes. Migrations record renames, aliases, splits, merges, default changes, and replacement IDs, following OpenTelemetry's schema migrations and Sigma's typed relationships.
Deprecation and replacement
Deprecated items are tombstones, not deleted history. Each records:
deprecatedIn, reason, planned support end, and migration guidance;- typed relations:
replacedBy,renamedFrom,derivedFrom,mergedFrom, orsimilarTo; - historical aliases and source/provenance; and
- whether default bundles still include it.
A stable ID is never reassigned. Default bundles stop selecting a deprecated item
after the documented grace period; compatibility bundles may retain it through a
major support window. unsupported is for unsafe, invalid, or unmaintainable
content and is excluded from new builds. rejected evidence cannot substantiate
a validation claim but remains recorded so folklore does not re-enter later.
Compiled artifacts and OCI distribution
Independently published catalogue package
Each package release compiles to a deterministic tar layer:
manifest.json # coordinate, version, schema, exports, compatibility
catalog.json # expanded definitions, sorted by stable ID
lock.json # exact transitive package and schema digests
sources.json # distributable provenance and freshness metadata
licenses/
LICENSES.json # SPDX expressions, notices, attribution
schemas/
definition-set.schema.json # exact schema needed to inspect this releaseTests and corpora are not loaded by the evaluator. Public test fixtures and release evidence may be a separate evidence artifact that refers to the package digest. Restricted data is never published to OCI merely to make the release self-contained.
Use RFC 8785 JSON Canonicalization Scheme for canonical JSON and hashing rather than a local “sort keys and whitespace” convention. Inputs must satisfy I-JSON; the compiler rejects duplicate keys, non-finite numbers, and values not safely representable by the schema. YAML aliases and merge keys are rejected to keep review and canonicalization unambiguous.
Publish using an OCI image manifest with:
- artifact type
application/vnd.groundskeeper.catalog.package.v1; - config type
application/vnd.groundskeeper.catalog.package.config.v1+json; - package layer type
application/vnd.groundskeeper.catalog.package.layer.v1.tar+gzip; - OCI source, revision, version, title, description, created and licence annotations; and
- repository path
<registry>/groundskeeper/catalog/<publisher>/<package>.
SemVer tags such as 1.4.0 and optional channels such as stable are discovery
handles. Released version tags must be protected, but consumers still resolve and
store the manifest digest. Artifact media types should be registered before the
format is declared stable.
Deployable runtime snapshot
The control plane resolves packages and policy layers, verifies all inputs, runs tests/replay gates, and emits a second OCI artifact:
manifest.json # snapshot format and host compatibility
catalog.json # flattened definition closure
policy.json # fully composed policy and typed overrides
detectors.json # required capabilities and pinned implementations
bindings.json # approved deployment bindings/correlation snapshot
lock.json # every input coordinate, version and digest
provenance.json # source commit/build summary; attestation is a referrerThis snapshot uses artifact type application/vnd.groundskeeper.snapshot.v1.
It is complete, immutable, and self-contained for evaluation. The runtime never
downloads package imports, resolves SemVer, merges policy, reads catalogue YAML,
or queries a registry/database during a request. It verifies the snapshot digest,
expected signer identity, provenance subject, schema/host compatibility, and
capabilities before atomically swapping the whole snapshot. Failure leaves the
last-known-good snapshot active.
Flattening duplicates shared definitions across deployment snapshots, but buys deterministic network-free evaluation, atomic rollback, and simple incident reproduction. Independent source/package versioning is a control-plane concern; it must not create partial runtime activation.
Signatures, provenance, and related artifacts
OCI 1.1 subject and the Referrers API associate these with the package or
snapshot digest:
- Sigstore/cosign signature or approved KMS/PKI signature;
- SLSA provenance whose subject equals the OCI digest and whose materials include source revision, locked packages, schemas, compiler, and relevant generated inputs;
- SPDX SBOM/licence inventory where executable detector content exists;
- catalogue source/licence inventory for definition-only packages;
- release test, corpus-hash, compatibility, and decision-diff attestation; and
- revocation or verification-summary information.
Verification checks expected signer identity and issuer, builder/workflow, source repository, build type, parameters, provenance subject, and digest. “A valid signature exists” is insufficient. Trust roots and accepted publishers are operator configuration, not tenant policy or package content. Signing establishes origin and integrity, not semantic quality, legal permission, or safety.
Discovery
The public index supports filtering without parsing every package:
- kind and public category;
- package, publisher, lifecycle, evidence status and trust tier;
- country, subdivision, issuing-authority level, language and cultural scope;
- provider/issuer, modality, stage and detector capability;
- validation level (shape, context, checksum, reference, registry/service);
- licence, redistribution/access class and source freshness;
- owners, last review, compatibility and deprecation/replacement;
- available policy bundles and corpus/release evidence; and
- OCI coordinate, SemVer and digest.
The index is generated from signed package manifests and can be mirrored. Search ranking and trust are registry concerns; neither changes object identity. Local install tools can build the same index from a set of OCI digests for air-gapped use.
Publication and activation gates
Publication is all-or-nothing:
- Parse YAML safely and validate closed schemas.
- Verify IDs, namespace ownership, export roots, source records and licences.
- Resolve the acyclic import DAG and produce the exact lock.
- Validate references, aliases, lifecycle transitions and compatibility.
- Run package cases, conformance tests, benign baselines and authorized corpus replay; compare old/new findings, spans, confidence and policy outcomes.
- Generate canonical JSON, docs, index, notices and release evidence; fail on generated drift.
- Build the OCI artifact reproducibly, generate provenance, approve, sign and publish without mutating an existing release.
Activation separately verifies trust policy and the complete deployment snapshot, then uses the existing shadow, deterministic canary, atomic promotion and rollback process. Registry availability is not required after an artifact is installed.
Consequences and trade-offs
Benefits
- Packages allow PII jurisdictions, provider secrets, safety domains, and third parties to release independently without package-per-rule overhead.
- Stable namespaced IDs keep audits, overrides, aliases, and deprecations valid across source moves.
- Family files keep shared evidence visible while limiting merge hotspots.
- Explicit provenance and corpus controls support Australian restricted sources, commercial datasets, and culturally governed material without pretending all public URLs are redistributable.
- Generated indexes/docs make a distributed package ecosystem discoverable.
- OCI reuses registry RBAC, replication, content addressing, signatures and referrers while the flattened snapshot keeps the data plane simple.
Costs
- A compiler, namespace registry, schema compatibility checker, documentation generator and release pipeline are required earlier than with loose YAML.
- Package boundaries require judgement and occasional package splits.
- Flattened snapshots duplicate content and are larger than deltas.
- Closed schemas and mandatory provenance make quick ad hoc contributions slower.
- DNS/source-host namespace verification needs transfer, abandonment, and signer operations.
These costs are accepted because silent collisions, untraceable validation claims, runtime dependency resolution, and licence-blind redistribution are substantially more expensive to correct after the catalogue becomes public.
Rejected alternatives
One repository-wide catalogue file
Rejected because it creates merge contention, global ownership, one release cadence, and weak provenance boundaries. A generated aggregate remains useful, but it is not source.
One file and package per item
Rejected as the default because hundreds of tiny packages multiply manifests, versions, signatures, dependency locks and review overhead. Use one item per file only when its complexity or governance merits it.
Directory path as stable ID
Rejected because taxonomy cleanup, package splits, and ownership moves would become breaking changes. Paths are navigation only.
Country-first global hierarchy
Rejected because credentials, safety and prompt security are not principally geographic. Domain-first with optional jurisdiction/provider axes is more stable.
Put definitions, policies, schemas and corpora under one catalog/ tree
Rejected because it would duplicate the established authorities under
policies/, api/schemas/, and test/, and would blur different ownership,
licensing and runtime roles.
Per-item SemVer
Rejected because items are not independently distributed and tested. Package SemVer, immutable item ID, per-item content digest, and OCI digest provide exact auditability with a tractable dependency graph.
UUID-only IDs
Rejected because they are collision-resistant but poor for authoring and discovery. Verified publisher namespaces provide collision resistance while preserving readable identifiers. Optional imported ecosystem UUIDs remain aliases.
Runtime loading of source packages or multiple OCI bundles
Rejected because dependency ordering, partial failure, registry outage, and inconsistent activation would enter the request path. Only the compiled complete snapshot is executable.
Signatures as the trust decision
Rejected because signatures prove possession/identity, not correctness, safety, licensing, or governance approval. Trust policy, provenance verification, conformance and release review are separate gates.
Decisions still requiring review
This ADR recommends defaults but cannot grant the required organizational or legal authority. Reviewers must decide:
- Whether the first public schema is
groundskeeper.dev/v1alpha1and when its compatibility promise begins. - Which initial package split is funded. The recommended seed is
pii-core,pii-au,secrets,security-prompt, andsafety-core. - Whether the permanent ID grammar and core public taxonomy are approved before definitions are published.
- Which team operates namespace verification, transfer, abandonment, registry moderation, signing identity, revocation and trust roots.
- Which OCI registry and compatibility fallback are supported, and whether the proposed media types will be registered.
- Exact CODEOWNERS teams and named privacy, legal, Australian-domain, provider, security, safety and governance approvers.
- The controlled-source decisions already tracked for ATO algorithms, DVS, PAF, G-NAF, standards, retention and redistribution.
- The Indigenous-led authority and workflow required before relevant catalogue definitions or corpora can be created or released.
- Numeric quality, benign-pass, latency and regression gates by risk tier.
- Policy DSL/override approval remains under ADR 0004; this ADR deliberately does not settle tenant weakening or failure posture.
Sources
External facts and upstream formats were reviewed on 2026-10-03. Pin upstream versions when implementation begins; links to moving branches document the surveyed projects, not immutable build inputs.
- OpenTelemetry semantic-conventions repository structure, generated docs, ownership, stability and migration requirements: CONTRIBUTING, attribute registry, and telemetry schemas.
- Semgrep registry contribution, namespace, metadata and positive/negative test requirements: contributing rules and semgrep-rules.
- Sigma rule IDs, statuses, relationships, licences and schema: rule specification and Sigma rules repository.
- YARA rule names, tags, metadata, includes and modules: writing YARA rules.
- Gitleaks provider rule generation, unique IDs, allowlists and configuration extension: Gitleaks repository and configuration and contribution guide.
- Microsoft Presidio recognizer registry, custom recognizers and country-specific implementations: Presidio Analyzer, developing recognizers, and default recognizer manifest.
- OPA bundle layout, manifest, roots, schema, signatures and activation behavior: OPA bundles and bundle manifest schema.
- OCI artifact types, subjects and referrers: OCI Image and Distribution 1.1, image manifest, and Distribution Specification.
- ORAS explanation of attached artifacts and referrer discovery: Attached Artifacts.
- Verified publisher namespace precedent: Maven Central namespace registration.
- Package compatibility: Semantic Versioning 2.0.0.
- Deterministic canonical JSON: RFC 8785 JSON Canonicalization Scheme.
- Signature identity and digest verification: Sigstore cosign verification.
- Provenance subject, builder and expectation verification: SLSA artifact verification and SLSA provenance.
Groundskeeper-specific legal, Australian PII, Indigenous Data Governance, conformance, policy-bundle, detector-extension, and persistence sources remain in the repository research index and their respective design documents.