Skip to content

feat: drift.lock lockfile, refs command, --changed flag#17

Merged
laulauland merged 15 commits intomainfrom
laurynas/lockfile
Apr 8, 2026
Merged

feat: drift.lock lockfile, refs command, --changed flag#17
laulauland merged 15 commits intomainfrom
laurynas/lockfile

Conversation

@laulauland
Copy link
Copy Markdown
Member

@laulauland laulauland commented Apr 8, 2026

Summary

Move all doc-to-code bindings out of embedded YAML frontmatter / HTML comments and into a centralized drift.lock file at the repo root.

  • drift.lock — flat, line-oriented lockfile (doc -> target sig:hex). Parser, serializer, walk-up discovery, query helpers in lockfile.zig
  • drift link writes to drift.lock, strips legacy embedded frontmatter on migration
  • drift check / status / unlink read from drift.lock
  • drift refs — new reverse-lookup command: which docs reference a given file/symbol
  • drift check --changed <path> — scope checking to docs whose targets match a path prefix (for CI)
  • spec → doc rename throughout codebase and docs
  • Arena-oriented memory management (run arena + scratch arena per command)
  • Explicit error sets on all command entry points, clean CLI error messages

Test plan

  • zig build test passes (unit + integration tests for all commands)
  • drift check on this repo passes
  • drift link docs/DESIGN.md round-trips cleanly
  • drift refs src/lockfile.zig returns expected docs
  • drift check --changed src/commands/ scopes correctly
  • Legacy repo with frontmatter anchors: drift link migrates to lockfile and strips frontmatter

Add src/lockfile.zig (file-is-the-struct) with:

Parser: read drift.lock line by line, skip # comments and blank lines,
split on " -> " for spec/rest, split rest on " " for target and key:value
pairs. Return ArrayList of Binding structs { spec, target, metadata[] }.

Serializer: sort bindings lexically by full line, write one per line,
trailing newline. Prepend "# drift.lock — managed by drift, do not edit manually".

Discovery: walk up from cwd checking for drift.lock at each directory.
Return the path to drift.lock and the project root (its parent dir).
Return null if not found (drift link will create it in cwd).

Query helpers:
- bindingsForSpec(spec_path) -> []Binding
- bindingsForTarget(target_path) -> []Binding (reverse lookup)
- addBinding / removeBinding / updateBinding mutations

All paths in drift.lock are relative to the project root (drift.lock's directory).

Acceptance criteria (integration tests):

1. Parse round-trip: write a drift.lock with 3 bindings, comments, and
   blank lines. Parse it, serialize it back. Output matches input
   (comments stripped, lines sorted, trailing newline).

2. Parse metadata: line "a.md -> b.ts sig:abc123 origin:github:x/y"
   produces Binding with target="b.ts", metadata contains sig: and origin:.

3. Discovery walks up: create drift.lock in parent dir, run discovery
   from a nested subdir. Returns the parent dir as project root.

4. Discovery returns null: temp dir with no drift.lock anywhere up
   to filesystem root. Discovery returns null.

5. bindingsForSpec: lockfile with 5 bindings across 2 specs. Query for
   one spec returns only its bindings.

6. bindingsForTarget: lockfile with 3 specs referencing the same target.
   Query returns all 3 spec paths.

7. Serializer sorts: add bindings in reverse order, serialize. Output
   lines are lexically sorted.
Modify src/commands/link.zig:

Targeted mode (drift link spec.md target.ts):
- Compute sig: from current file via symbols.computeContentSig()
- Call lockfile.addBinding() to append/update the line in drift.lock
- If drift.lock doesn't exist, create it at cwd (or git root)
- Print: "added docs/auth.md -> src/auth/login.ts sig:a1b2c3d4e5f6a7b8"

Blanket mode (drift link spec.md):
- Read all bindings for that spec from lockfile
- Recompute sig: for each target
- Write updated lockfile

Legacy migration (on both modes):
- After writing to lockfile, check if spec has drift: frontmatter
  or <!-- drift: ... --> HTML comments
- If so, strip them from the spec file (use existing frontmatter.zig
  parsing to find boundaries, then remove those ranges)
- This is the only migration path — no separate command

Stop writing frontmatter/HTML comments entirely. link.zig should not
import frontmatter.zig for writing, only for legacy detection/stripping.

Acceptance criteria (integration tests):

1. Link creates drift.lock: empty repo, run drift link docs/spec.md src/app.ts.
   drift.lock exists, contains "docs/spec.md -> src/app.ts sig:<hex>".
   spec.md has no drift: frontmatter.

2. Link updates existing binding: link spec to target, modify target,
   link again. drift.lock has one line for that binding (not two),
   sig: value changed.

3. Link with symbol: drift link docs/spec.md src/app.ts#MyFunc.
   drift.lock contains "docs/spec.md -> src/app.ts#MyFunc sig:<hex>".

4. Blanket relink: link spec to 3 targets, modify one target, run
   drift link docs/spec.md (no target arg). All 3 sigs refreshed.

5. Legacy migration — frontmatter: create spec with drift: frontmatter
   containing 2 anchors. Run drift link docs/spec.md. drift.lock has
   2 entries. Spec file no longer contains "drift:" or "files:".

6. Legacy migration — HTML comment: create spec with <!-- drift: ... -->
   block. Run drift link docs/spec.md. drift.lock has entries. Spec
   file no longer contains "<!-- drift:".

7. Idempotent: run drift link docs/spec.md src/app.ts twice without
   changing the target. drift.lock content is identical both times.
Modify src/commands/lint.zig:

Replace spec discovery (scanner.findAndSortSpecs) with lockfile.read().
Build the spec list from lockfile bindings — group by spec path, collect
anchors per spec.

Keep the existing staleness check logic (checkAnchorBySig) — it already
compares sig: hex against recomputed fingerprint. Wire it to read the
sig: value from the lockfile Binding metadata instead of parsing it
from the anchor string's @sig: suffix.

Legacy fallback: if drift.lock doesn't exist, fall back to the current
scanner-based flow (parse frontmatter/comments from specs). This handles
repos that haven't run drift link on the new version yet. Print a warning:
"no drift.lock found, falling back to embedded anchors".

Keep all existing output formatting (text and JSON drift.check.v1).
The Anchor struct in payload/drift_check_v1.zig needs its provenance
populated from lockfile metadata instead of parsed from anchor strings.

Remove scanner.zig imports once lockfile path is working — scanner.zig
becomes dead code after this (keep it for the legacy fallback).

Acceptance criteria (integration tests):

1. Fresh anchor: write drift.lock with valid sig:, run drift check.
   Exit 0, output shows "ok" for that spec.

2. Stale anchor: write drift.lock with outdated sig:, modify the target
   file, run drift check. Exit 1, output shows "STALE".

3. File not found: write drift.lock pointing to nonexistent file.
   Exit 1, output shows "STALE" with "file not found".

4. Symbol not found: write drift.lock with src/app.ts#Missing, where
   Missing doesn't exist in app.ts. Exit 1, "symbol not found".

5. Mixed fresh/stale: spec with 2 anchors, one fresh one stale.
   Output shows both, exit 1.

6. JSON output: run drift check --format json with lockfile bindings.
   Output parses as valid drift.check.v1 JSON, provenance.kind = "sig".

7. Legacy fallback: no drift.lock, spec has frontmatter anchors.
   drift check still works, stderr contains "no drift.lock found".

8. Origin skip: write drift.lock with origin:github:other/repo.
   Run in a repo with different remote. Anchor reported as SKIP.
Modify src/commands/unlink.zig:

Replace frontmatter editing with lockfile mutation:
- Parse spec path and target from CLI args (same as today)
- Call lockfile.removeBinding(spec_path, target) to remove the matching
  line from drift.lock
- Print: "removed docs/auth.md -> src/auth/old-handler.ts from drift.lock"
- If no matching binding exists, print error and exit non-zero

No longer touch the spec file at all. Remove frontmatter.zig imports.

Acceptance criteria (integration tests):

1. Unlink removes binding: drift.lock has 2 bindings for a spec.
   Run drift unlink docs/spec.md src/a.ts. drift.lock has 1 binding,
   the other is gone. Exit 0.

2. Unlink with symbol: drift.lock has "docs/spec.md -> src/a.ts#Foo sig:...".
   Run drift unlink docs/spec.md src/a.ts#Foo. Line removed.

3. Unlink nonexistent: drift.lock has no binding for the given spec+target.
   Exit non-zero, stderr has error message.

4. Spec file untouched: spec.md has content before unlink. After unlink,
   spec.md content is byte-identical.

5. Other bindings preserved: drift.lock has bindings for 2 different specs.
   Unlink one binding from one spec. The other spec's bindings unchanged.
Modify src/commands/status.zig:

Replace scanner-based spec discovery with lockfile.read(). Group bindings
by spec path, list targets under each spec. Same output format as today
but sourced from lockfile:

  docs/auth.md (3 anchors)
    files:
      - src/auth/provider.ts#AuthConfig
      - src/auth/login.ts
      - src/auth/session.ts

JSON output (--format json) should also source from lockfile bindings.

Only show specs that have entries in drift.lock — no scanning for
markdown files without bindings.

Acceptance criteria (integration tests):

1. Lists specs from lockfile: drift.lock has 2 specs with 3 and 1
   anchors. drift status shows both with correct anchor counts.

2. Shows targets without sig: status output lists target paths
   without the sig: metadata (clean display).

3. JSON output: drift status --format json returns valid JSON with
   spec paths and anchor lists sourced from lockfile.

4. Empty lockfile: drift.lock exists but has only comments/blank lines.
   drift status produces no spec output, exit 0.

5. No lockfile: drift.lock doesn't exist. Prints warning, exit 0.
Add src/commands/refs.zig and wire it up in src/main.zig:

CLI: drift refs <path> or drift refs <path#Symbol>
- Add "refs" to clap SubCommand enum in main.zig
- Add dispatch case calling refs.run()

Implementation:
- Read drift.lock via lockfile.read()
- Filter bindings where target matches the argument
  (exact match, not prefix — drift check --changed does prefix)
- Print matching spec paths, one per line, sorted, deduplicated
- Exit 0 regardless of whether matches are found

Support --format json for tool integration (array of spec paths).

Acceptance criteria (integration tests):

1. Single match: drift.lock has docs/auth.md -> src/login.ts.
   drift refs src/login.ts prints "docs/auth.md". Exit 0.

2. Multiple matches: 3 specs reference src/login.ts.
   drift refs src/login.ts prints all 3 spec paths, sorted.

3. Symbol match: drift.lock has docs/auth.md -> src/app.ts#AuthConfig.
   drift refs src/app.ts#AuthConfig prints "docs/auth.md".
   drift refs src/app.ts prints nothing (exact match, not prefix).

4. No matches: drift refs src/nonexistent.ts prints nothing. Exit 0.

5. JSON output: drift refs --format json src/login.ts returns a
   JSON array of spec path strings.

6. Deduplicated: drift.lock has two bindings from the same spec to
   the same target (shouldn't happen, but defensive). Output shows
   the spec path once.
Modify src/commands/lint.zig and src/main.zig:

Add --changed <path> optional arg to the check/lint clap params.

When --changed is provided:
- Read drift.lock
- Filter bindings where target starts with the given path prefix
- Collect the unique spec paths from matching bindings
- Run staleness checks only for those specs (and only the matching
  anchors within each spec, not all anchors)
- Same output format and exit codes as regular drift check

This enables CI pipelines to scope lint to changed files:
  drift check --changed src/auth/

Without --changed, behavior is unchanged (check all bindings).

Acceptance criteria (integration tests):

1. Scoped match: drift.lock has bindings for src/auth/login.ts and
   src/payments/stripe.ts. Run drift check --changed src/auth/.
   Output includes docs referencing src/auth/login.ts but not
   src/payments/stripe.ts.

2. No match: drift check --changed src/nonexistent/. Exit 0,
   no specs reported (nothing to check).

3. Exact file: drift check --changed src/auth/login.ts matches
   that exact target. Spec is checked.

4. Partial anchor check: spec has 2 anchors, one in src/auth/ and
   one in src/payments/. With --changed src/auth/, only the auth
   anchor is checked. The payments anchor is not in the output.

5. Stale scoped: --changed path matches a stale anchor. Exit 1,
   STALE reported. Unmatched stale anchors in other paths do not
   affect the exit code.

6. Works with --format json: drift check --changed src/auth/
   --format json returns valid drift.check.v1 JSON containing only
   the matched specs and anchors.
No internal error names leak to users. Custom parseCommand captures
failed input for 'unknown command' messages. Command modules (lint, link)
declare RunError sets. main.zig renders domain errors with context and
maps system errors to readable messages via fatal/exitWithError.
@laulauland laulauland linked an issue Apr 8, 2026 that may be closed by this pull request
ParserCache eagerly compiled tree-sitter queries alongside parsers.
When a query was invalid (e.g. zig.scm uses node names that don't
match the grammar version), the entire cache entry failed, causing
file-level fingerprinting to return null even though it doesn't need
queries. This made drift check report 'cannot compute fingerprint'
for all zig files while drift link (which uses fresh parsers) worked
fine.

Remove the cache entirely. Parser/query creation is microseconds per
file — negligible for the number of files drift checks. Both link and
check now use the same computeFingerprint path.
Two bugs fixed:

1. lockfile.discover() walked up past .git/.jj boundaries, so
   integration tests in .zig-cache/tmp/ polluted the repo's
   drift.lock with test bindings. Now discovery stops at the
   first .git or .jj directory.

2. ParserCache eagerly compiled tree-sitter queries alongside
   parsers. Invalid queries (zig.scm node names didn't match
   grammar) caused file-level fingerprinting to fail even though
   it doesn't need queries. Removed the cache — parser creation
   is microseconds per file. Both link and check now use the
   same computeFingerprint path.
@laulauland laulauland merged commit 8f0d655 into main Apr 8, 2026
5 checks passed
@laulauland laulauland deleted the laurynas/lockfile branch April 8, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lockfile mode

1 participant