There are good tools for asking questions about code, mapping it, writing specs and reviewing pull requests, and several of them now check an agent's work against a spec. This page explains why I built Leyline anyway, and where the other tools do better.
Leyline is a tool you install beside a coding agent. It maps a codebase into a graph, assists with human understanding, helps the agent plan a change as a spec, coordinates agent reviews against the plan, and then checks the finished code against the spec using the graph and your test results.
Each of those jobs already has a good tool. Sourcegraph Deep Search answers questions about a codebase and cites the files and searches it used. Local graph servers such as GitNexus, codebase-memory-mcp and code-review-graph give an agent a code graph over MCP. OpenSpec (which Leyline builds on), Spec Kit and Kiro turn a request into a spec. CodeRabbit, Greptile and Qodo review pull requests, and CodeScene reads change coupling out of git history more thoroughly than Leyline does.
Several of them now check an agent's work against the spec as well. OpenSpec has /opsx:verify, Spec Kit has /speckit.converge, and Kiro runs property-based tests built from the requirements. So it's fair to ask why Leyline should exist.
The matrix below comes from a research pass against vendor docs and READMEs on 2026-10-07. Hover or tap a cell to see what the tools in that row do in that phase, with a source where I have one.
Every other row has gaps. Leyline covers every phase, but covering a phase doesn't make it the best tool there, and "When another tool is the better choice" below says where it isn't. The case for using it rests on four differences, which the rest of this page takes one at a time: 1) no model judges the work, 2) one map shares decisions between steps, 3) the records stay in your repository, 4) and the tool says what it doesn't know.
/opsx:verify has a language model search the code for evidence of each task and scenario, then report CRITICAL, WARNING and SUGGESTION items that don't block archiving. /speckit.converge has a model read the source against the spec, plan and tasks, and it treats ticks already in the task list as unverified claims, a choice I agree with. Qodo rates each ticket Fully, Partially or Not compliant, and CodeRabbit marks linked issues Addressed, Not addressed or Unclear. All four are a model's verdict, and none of them uses test results.
Kiro is the exception. It turns requirements into property-based tests, links each to its requirement and task, and runs them. It doesn't check whether each task was done against a diff of the code.
A model that judges an agent's work reads the agent's explanation, and agents write fluent explanations easily. Leyline has no language model in it. A task's verdict comes from what changed on the graph between the plan and now. A scenario's verdict comes from its test's result before and after the change, and whether that test ran the changed code. Leyline also lists every edit that falls outside the spec, which I didn't find in any other tool.
I planned a small change called loud-engine on a test repository: make Engine.start return the name in upper case, and add Engine.shout. Then I gave the OpenSpec folder and the test command to a fresh Claude subagent that didn't know Leyline existed. Its report, cut for length:
I've implemented the loud-engine change, and all 6 tests pass.
…
Only the tests call
start(). Nothing inpy/weborcs/does, so no other code had to change.Decision for you:
shout()keeps the name's original case and just adds!, because that's how task 1.2 words it. …Tests:
6 passed in 0.01s. Nothing is committed.
leyline check loud-enginereal output**Yes.** Every task is done, every scenario is proven, and nothing outside the spec changed. … Verdicts: 5 proven. … Tests: 5 of 5 passed before, 6 of 6 after. Next: nothing left to check; the change was done as agreed. Review the diff and commit it.
The report and the verdict agree, and every claim in the report matches the check. Now suppose the agent had also upper-cased the name in Engine.child. A reviewer had proposed that edit before any code was written, and the person had rejected it. I scripted the edit to see what Leyline would say. Every test still passes, so the agent's report could read the same as before:
leyline check loud-engine, after a scripted extra editreal output**Not yet:** 1 edit is outside the spec. … Verdicts: 5 proven. … **Changed, but not in the spec:** - Engine.child (callable) py/src/pkg/core.py Tests: 5 of 5 passed before, 6 of 6 after. … Next: Engine.child changed outside the spec: add a task for each (then `leyline plan loud-engine`) or undo it.
The verdict changed while the report could stay the same, because Leyline never reads the report. The figure below lets you set the inputs yourself. The left panel follows Leyline's rules for one task and one scenario. The right panel is a made-up judge that relies on the report; it's there to show where such a judge's decision comes from.
verdicts.py for one task and one scenario (the real check judged 3 tasks and 2 scenarios). By default "partial", "contradicted" and "inconclusive" block "done as agreed", and "needs a person" is listed without blocking; a project can change that in openspec/leyline.toml. A new function that only the named code calls counts as a helper, not as an edit outside the spec. The right panel is illustrative. It isn't the measured behavior of any product, and a careful model reading the code could do better. It's there to show that the report is an input to a judge and isn't an input to Leyline.Every engineer knows the difference between a pull request's description and its CI run. A well-written description can talk a reviewer into approving a change. It can't make a failing test pass. Asking a model whether an agent did the work is like reviewing the description: the agent's report is one of the things being judged, and agents write convincing reports easily. Leyline's verdict works like CI. It reads what changed in the code and what the tests did, and the report isn't one of its inputs.
A fair objection is that a tool with no language model in it can't be built for agents. It can, because the agent does all the language work. It drafts the spec, explains code, plays the logic and performance reviewers, and writes the code. Leyline gives it facts, in answers kept short enough for its context (about 24,000 characters for most tools, a little more for the few that walk code, with a note on what was cut and how to see the rest), and it ends every step with a Next: line. The chain of Next: lines is the workflow.
The person makes the decisions. Only the person resolves a finding, and a rule an agent adds stays "suggested" until the person confirms it.
Has no language model. Maps the code, gathers evidence and gives verdicts.
Next: lineYour coding agent, plus fresh sessions for review. It does the language work.
You make every decision.
Suppose you use a chat tool to understand the code, a spec tool to plan, a review bot on the pull request and something else to check the result. Each step starts from its own view of the code, and each decision stays in the tool that made it. Leyline keeps one map and one store of findings, so each step starts from what the previous one decided.
Your own change usually starts spec-first, with a plan, and someone else's branch starts at the pull-request review. Both ways in write findings into the same store, and later changes read them back. Scroll through the steps, or use the buttons, to see what each step passes to the next.
tasks.md is looked up, and a name that isn't on the map is read as new code.check compares the code with that snapshot and the tests with the run recorded at plan time. A tick in tasks.md never counts, and the command exits 0 only when the change was done as agreed.leyline pr, which reads the change from the code: what changed, callers of a changed signature that weren't edited, removed code that's still called, and changed code no test reaches. Its findings go in the same store, and with --gate it exits 1 while something the map can show is broken is left, so it can hold a merge in CI the way check holds a change.drift lists code the living specs name that has since gone, moved or changed signature.The clearest example is a finding that comes back in a later change. In the loud-engine plan, Leyline noted that Engine.name is also used by Engine.child. A logic reviewer turned that into a finding. The person rejected it and gave a reason, and Leyline kept the reason as a learning. When a later change raised the same worry, the plan page said so. Step through it below.
Specs are OpenSpec markdown. The page for each change, leyline.md, sits in that change's folder. Learnings are a JSON file with sorted keys, one entry per decision, each holding the reason in the person's own words. A learning is retired automatically once people have accepted at least two of the findings it matched, and more than they rejected. You can read all of it in a diff and review it like code.
The review bots keep this kind of memory elsewhere. CodeRabbit stores learnings on its servers by git organization; they never expire, and admins can edit them. Copilot Memory (in preview) cites code locations, checks each memory against the current branch before using it, and deletes a memory after 28 days without use. Those are reasonable designs for a hosted service. Leyline runs on your machine, needs no account, and is licensed under the GPL, version 3. Its MCP tools read and write only inside the repositories it maps, and it refuses a .leyline/ folder that a repository commits, so a pull request can't hand it a ready-made map.
.gitignore herecheck needs until the change is archived.Every record on the map belongs to one of three layers. Facts come from the source and are rebuilt on every index. A model (in practice, your agent) writes inferred records, with evidence and a confidence, and Leyline flags them as stale when the code behind the evidence changes. Intent is what you state, such as rules and boundaries.
Links between functions come in three grades. A compiler confirmed an exact link, using Roslyn for C# or a SCIP index for other languages. Leyline worked out a heuristic link from the syntax, and made a guess by name alone because only one declaration had that name. The map page draws them solid, dashed and dotted, and answers carry the same grade.
Two of the five verdicts let Leyline decline to answer. "Inconclusive" means the map can't place the code or no result was recorded. "Needs a person" means a task names no code, or a test passed without Leyline being able to show it ran the changed code.
Here is how well the syntax pass holds up against a compiler, from the README. "Right" is precision over the call sites the compiler resolved, and "found" is recall of the compiler's links.
| Language (repository) | Compiler | Hand-written adapter right / found | Generic adapter right / found |
|---|---|---|---|
| TypeScript (Parlance, 160k lines) | scip-typescript | 100% / 98% | 99% / 97% |
| Python (Flask) | scip-python | 99% / 68% | 96% / 87% |
| C# (a private C# and Python project) | Roslyn | 99.7% / 97% | 98% / 83% |
The Parlance row is the editor I wrote about in Validation at a Million Words. The Python recall of 68% is low because the hand-written Python adapter misses calls inside decorators such as @app.route(...).
Without the compiler pass, a linked call isn't necessarily a correct one, so the README also reports a hand check of a random sample of links against the source.
Engine.child to a past decision, under the 0.45 threshold, so it wasn't markedPolly's fluent API has up to twenty overloads per name, which is the hard case for linking by syntax. Its packages couldn't be restored, so the compiler binds only part of the code, and on that part the compiler pass confirmed 4,844 of the syntax links and removed 1,286, nearly all of them extra overloads of the right method.
The third number is about learnings. A finding about trimming the name in Engine.child shares code with the rejected upper-casing finding, but it's a different worry, and it scored 0.44 against a threshold of 0.45. I think leaving it unmarked was the right call. When a past decision might not apply, I'd rather Leyline miss the match than put the wrong reason in front of the person.
The figure below shows what a plan reports for each function in the same small Python package. Pick a function, choose whether the change keeps its signature, and the code the plan names is highlighted. The line style of each link shows how it was found.
leyline plan (0.8.1) on a copy of the same test repository, with one task per function. Every link in this Python package is heuristic or a guess, since no SCIP index was given; poke calls thing.child() on an untyped parameter, so that link is a guess, and the plan says so. Two fixture links (test_start and test_chain asking for the engine fixture) are left out of the drawing to keep it readable. The copy has no git history, so the fourth ring the plan can show, files that usually change together, is empty here.shout upper-cases the name, and the agent raised it as a decision for the person. Leyline reported nothing about it. Finding gaps in the spec is the reviewers' job.pip install leyline-code # Python 3.10+ claude mcp add leyline -- leyline serve # connect Claude Code leyline skills install # into this repo; --to ~ for every project
Then ask in plain words: "Plan adding an export button, with tests," or "Review pull request 123." The agent maps the code when it needs to. Every command also works by hand.
Leyline · Orbitope · GPL-3.0 · claims about other tools are from a research pass on 2026-10-07