Open source · on your infrastructure · formerly Agent Smith

Make workprogrammable.

A ticket goes in. codeably writes the spec, builds it in a sandbox on your machines, checks every acceptance criterion on the branch and opens the pull request. It works inside Azure Boards and Azure Repos, and with GitHub, Jira and GitLab. You answer when it asks.

— stars MIT licensedRuns on your infrastructureCalls your AI provider directlyEvery run shows its cost
A real run, replayedSCRUM-13 · stock transfer between SKUs done · 5 min 55 s
  1. Ticket
  2. Spec
  3. Build
  4. Verify
  5. Review
  6. Check
  7. Deliver
Fetched SCRUM-13 from Jira
Own container: dotnet/sdk:10.0
Spec e337a: transfer stock atomically, two Transfer movements
Every premise still holds
Decided: Transfer writes the ledger itself
5 files committed, within every limit
dotnet build: succeeded
dotnet test: 5 passed, 0 failed
Review: StockLedger.cs:35 repeats a check (DRY)
Fix pass: no edit, the ticket asks for two Transfer movements
Criteria on the branch: 5 of 5 met
Delivered to branch agent-smith/SCRUM-13
5 min 55 s, one finding left open for you

Taken from the run database of a real run on 13 October. Nothing is staged, and the one review finding it left open is part of the story.

Define the tickets together

Work it out is a conversation with an agent that reads your repositories while it answers. It splits the change into tickets in the order they can merge, names what each one requires, and files nothing until you approve.

Example · a payment method across two repos
Work it out · acme-shopreading code
you

Customers in Germany keep asking for SEPA direct debit at checkout. What would that take?

codeably
acme-api/Checkout/PaymentProviderRegistry.csacme-web/src/checkout/PaymentStep.tsxacme-api/Billing/Mandates/

Payment providers register in PaymentProviderRegistry, so the provider itself is small. There is no place to store a signed mandate yet, and the payment step has no IBAN field. I would cut it into three tickets:

1
Store SEPA mandates with signature date
acme-api · requires nothing
4 criteria
2
SEPA provider in the payment registry
acme-api · requires 1
5 criteria
3
IBAN entry and mandate text in checkout
acme-web · requires 2
6 criteria
File 3 work items in Azure BoardsNothing is filed until you click.

It reads before it proposes

While it works you see which files it opens, and every fact in a proposal cites its file and line.

Tickets in merge order

Each proposed ticket says which one it requires. codeably works them in that order, so ticket 3 never builds on code that ticket 2 hasn't merged.

Drop in what you have

Upload a screenshot, a PDF or a spec. Paste a URL, or a Figma link once Figma is connected. Uploads stay with the conversation.

One ticket, every repo, one pull request each

The approved ticket becomes a series of specs. Fresh reviewers check each spec before anything is built. Then every repository gets its own sandbox with its own toolchain, and every acceptance criterion is checked against evidence before a pull request is marked ready for review.

TicketA work item arrives by service hook, polling or a tag, or straight from Work it out.
SpecsThe ticket becomes a series of specs, and each one is attacked by fresh reviewers before any code exists.
BuildEvery repository gets its own sandbox and toolchain. Specs run in their required order.
VerifyEach criterion is met, unmet or not applicable, with the evidence that decided it.
DeliverOne pull request per repo in scope, cross-linked. The work item gets every link, the ticket's page the cost.

Every step, and where the model gets a say

Keep talking on the ticket

When a ticket leaves a real decision open, codeably stops and asks. The question shows up on the work item and on the ticket's page. Your answer resumes the run, or starts the next attempt after a hand-back, and no tokens burn while it waits.

Answer where you already are

Reply in a work item comment in Azure Boards, or in the ticket's conversation in the dashboard. Both land in the same thread.

Every attempt is on the record

The ticket's page lists what started each attempt, what it did and why it stopped, with the cost per spec and the time spent waiting on you.

Done means merged

A ticket counts as done only when its pull request merged. Until then the page says who acts next.

Example · a rate limit with an open question
Tickets · acme-platform · AB#1288waits for you

Rate-limit the public export endpoint

attempts2
specs2 of 3
cost so far$1.92
waiting on you14 min
ConversationSpecsAcceptanceAttemptsCost & compute
codeably · attempt 1 handed back

The ticket says "limit to 100 requests". Per minute per API key, or per client IP? ApiKeyMiddleware already knows the key. Per IP would rely on the X-Forwarded-For header the ingress sets.

you · on the work item

Per API key, per minute. Return 429 with a Retry-After header.

codeably

Taken as an acceptance criterion: "over 100 requests per key per minute return 429 with Retry-After". Attempt 2 is running.

Attempt 1 · stopped with a question$0.84
Attempt 2 · started by your reply · spec 2 of 3$1.08

Review it like a colleague's pull request

Write your comments in Azure Repos as you would for anyone. When you're done, vote Wait for author or comment /agent-smith. The rework reads every open thread, changes the code, runs the checks again and pushes to the same branch.

Example · a reviewer asks for the shared retry helper
Azure Repos · acme-api · PR !412 · AB#1301rework running
Webhooks/WebhookDispatcher.cs
- for (var attempt = 0; attempt < 5; attempt++)
- {
- if (await TrySendAsync(delivery, ct)) return;
- await Task.Delay(TimeSpan.FromSeconds(Math.Pow(2, attempt)), ct);
- }
+ await _retry.ExecuteAsync(() => SendAsync(delivery, ct), ct);
reviewer · line 88

We already have RetryPolicy in Acme.Common. Please use that instead of the hand-rolled loop. It also logs every attempt.

reviewer · line 41

The max delay should come from settings, not a constant.

Wait for authorvote by the reviewer · starts the rework
codeably · attempt 2

Replaced the loop with RetryPolicy.Exponential, read the max delay from Webhooks:MaxRetryDelay. Both threads resolved, all 11 criteria still met. Pushed 3f9c2e1.

Batch your comments

Comments alone start nothing. They are collected, and one vote or one /agent-smith works through all of them together.

Only people who can push

A command counts only from someone with Contribute permission on the repository. Anyone else costs no tokens and starts nothing.

Reviews on request

Opening or pushing a pull request starts nothing. On anyone else's pull request, /as review asks for a review. On codeably's own, /as starts a rework.

Pull request commands and reworks

Why we built it this way

These are the choices behind the software factory, and the reasons for each.

Your code stays on your machine.

codeably does not store any information about your code. It runs on your infrastructure and calls your AI provider directly. The more AI gets used, the more information tends to end up stored somewhere you never intended. Not here.

You see what every run cost.

Every run records its token usage and its cost in dollars. The ticket's page splits it per attempt and per spec, next to the cap you set for the run, so you know what a ticket cost before you merge it.

The why gets written down.

Great developers write great code. They even write documentation. But we have never seen the why being documented. codeably writes the specification it worked from next to the code and keeps a decision log in .agentsmith/decisions/, each entry naming what it chose and why, so the reasoning stays available over the complete lifecycle of the program.

You approve the spec, not a wall of diff.

Reviewing 800 changed lines tells you little about whether the right thing got built. So the moment to say yes or no is earlier: the specification. Approve it in a Work it out conversation and the run executes exactly that. Let it derive one from a plain ticket and the cut is on the ticket before any code exists, where a comment corrects it. Either way you judge the intent, and the delivery account judges the code against it.

One sandbox per repo, with its own toolchain.

In a mono repository, codeably works on UI, backend and workers in one place. With several repositories per project, it creates one sandbox per repository and one pull request for each.

Which languages are supported?

A Python repository gets a Python sandbox, a .NET repository gets your .NET SDK. The language belongs to the sandbox image, not to codeably, so any language your toolchain image can build works.

Who builds codeably

codeably was created by Holger Leichsenring, who leads the project and owns its direction. It is shaped by the teams that run it in production: their tickets, failed runs and review comments turn into the next specs, which is why most of what codeably does today exists. Engineers from those teams are starting to contribute directly. Professional services, onboarding and maintenance contracts come from 45level, which works with the project.

Built by the method it teaches

codeably bootstraps an .agentsmith/ directory into your repo: context, specs, a decision log, a memory of what it learned. That is the product, and it is also how this repository got written, by Holger Leichsenring and one language model. About 750 hours of human time went in, roughly 37 minutes per completed spec.

1,229completed specs
699,551lines of C#
6,908recorded decisions
10,301tests

The tests do the steering

Ten coding principles are enforced as architecture tests: max 120 lines per file, state held in DI, exactly one method allowed to decide whether a run delivered, a gate that has to run after the thing it judges. Every one of them has something that went wrong behind it. When a rule was right and the existing code broke it 187 times, the violations went into a frozen baseline that entries can only leave, so the debt moves one direction and nobody has to schedule a cleanup week.

The model has no way to wave itself through

A blocking commit hook lets a spec commit through once the dashboard's build and tests, the backend build, all 10,301 tests, four CLI dry runs and every pipeline harness preset come back green. CI would have told us about a break afterwards. The hook stops the commit from existing. And the principles.md we built this project under is the same file codeably injects into its own agents at runtime.

Including what didn't work

The skill catalog grew to 95 role definitions and then dropped to 12 in one breaking release, because stronger models did better with bash and a clear goal than with narrow toolsets. The plan generator got retired after evaluations caught it truncating every multi repo plan. A cached token metric read zero for a long stretch while the caching underneath it worked fine. The full account is in the docs, along with how every figure here was counted.

More pipelines than the coding one

code is the headline because it's the one most people show up for. It covers bug fixes, features, refactors and migrations alike. Seven more presets ship in the box: security-scan and api-security-scan (a security master reviewing a codebase or a live API), pr-review (a review of a pull request, posted back as PR comments), legal-analysis (contract review), mad-discussion (multi-agent design discussion when you want to argue something out), init-project (bootstraps the .agentsmith/ directory per component in each repo), and spec-dialog (a design conversation that ends in a ticket or a spec draft). Same orchestrator, different masters, different output shapes.

The masters and skills behind them live in a versioned catalog that ships inside the binary. You can pin another release, or layer your own skills on top without forking it. Pipeline reference is in the docs.

Deploy as a single binary, Docker container, or Kubernetes. The sandbox-agent injects via an init-container into whatever toolchain image your repos use, so you get one carrier image plus the upstream toolchain image you already trust. Slack and Teams work as a front door too, for triggering runs and for the same design conversation the dashboard has.

AI providers · Docker Compose · Kubernetes

Have a look.

agent-smith demo runs the real pipeline against a bundled sample project. One model key, no tracker, no Docker. The command still carries the old name; the CLI rename follows.