Architect is an agent factory. Type a job in one sentence and it builds an AI agent on NVIDIA Nemotron.
The agent only runs if it passed its exam. Change it, by hand or with a coding agent, and it stops with 0 model calls until it passes again or you undo.
Architect runs on Nebius Token Factory, as a desktop app and in the terminal. The main drawback is that the stop is a tripwire, not a lock. Watch the video or try it.
Change it, and it stops
-
Passed. It is your agent.
59 of 100 receipts read correctly. Pass mark: 40 of 100.
A receipt counts as right only when its total, subtotal, tax and item count are all right.
-
Stopped: changed after its exam. 0 model calls.
Changed:
artifact/config.json(model: Nemotron 3 Super to Nemotron 3.5 Lightning).Your app's calls to this agent are refused before any model is called. Stopped means not examined yet. It does not mean worse.
-
Retake its exam, or undo the change.
Retake exam. The changed agent takes the exam again and runs only if it reaches the pass mark. This one read "29 of 100. Pass mark 40. Did not pass." and stayed stopped.
Undo. Puts back the version that passed, exactly as it was. Your app works again at once.
These are the sentences the app shows, with the numbers from my run of the demo agent in the example app, a receipt reader on Nemotron 3 Super whose model I changed by hand to the cheaper Lightning. Its exam uses development receipts, so these scores are not the published results further down.
How an agent gets its exam
Describe a job in plain words, like reading the totals off a receipt. Architect tries Nemotron models from the cheapest up. Each one practices on one set of cases, then takes an exam on cases it has never seen. The cheapest model that reaches the pass mark becomes your agent, and a model that does not pass is never installed.
The published results
The example is a receipt reader for a small expense app. On 348 receipts that no build step saw, plain Super got 180 right to Ultra's 184, too close for this run to say which is more accurate (their 95% intervals overlap). Super's cost per receipt is about 30% of Ultra's. These are the models alone, with the plain prompt and no rule checks.
| Model, plain prompt | Exact on 348 | 95% interval | Per 1,000 receipts |
|---|---|---|---|
| Nemotron 3 UltraUltra | 0.529184/348[0.476, 0.581] | [0.476, 0.581] | $0.92 |
| Nemotron 3 SuperSuper | 0.517180/348[0.465, 0.569] | [0.465, 0.569] | $0.28 |
| Nemotron 3.5 LightningLightning | 0.27094/348[0.226, 0.319] | [0.226, 0.319] | $0.058 |
The receipt reader's plan declares two rule checks from the user's own rules: amounts keep every printed digit, and a subtotal equal to the total is null unless the receipt prints it as a subtotal. Each check is code that runs on every reply. The checks are part of the plan, so each exam tests the model and its checks together, and a pass covers both. With the checks, every model got more exam receipts right. Lightning missed the pass mark by one receipt, and Super shipped again.
| Model | With checks | Without | Gain |
|---|---|---|---|
| Nemotron 3.5 LightningLightning | 39/100 | 27/100 | +12 |
| Nemotron 3 SuperSuper | 65/100 | 51/100 | +14 |
| Nemotron 3 UltraUltra | 71/100 | 49/100 | +22 |
Three later builds of the same plan made the same agents and took the same exam: Lightning got 39, 42 and 39, and at 42 it passed and shipped.
A receipt counts only when its total, subtotal, tax and item count are all exact. The checks were added after a re-run of a pilot on other receipts, where nearly every field Ultra and Lightning got wrong broke a rule the prompt states, most often by giving a subtotal where the receipt prints none. They make no model calls, and each exam cost the same per 1,000 receipts as without them, to the cent.
Forge is Architect's coding agent: it writes a custom harness (the code around the model calls) to try to beat the tuned preset, a built-in strategy tuned on the practice receipts. In 12 sessions with a Nemotron model as its engineer, 5 with Ultra and 7 with Super, none of its harnesses was accepted, so a build on Nemotron ships the tuned preset.
To check the signed results offline, run uv run architect verify --published receipts in a checkout, or read the full results and the evaluation plan.
- Prices: Nebius Token Factory list prices of 2026-10-03.
- No build step saw the delivery receipts.
- Regions: Lightning is served from eu-north1, Super and Ultra from us-central1, and latency depends on the region.
- The models may have seen CORD v2 in pretraining.
- The input is text rebuilt from CORD's word annotations. No OCR ran.
- Every model ran at temperature 0 with thinking off. NVIDIA recommends 1.0 for these models.
Ten change requests, one run each
I wrote down ten ordinary change requests before any model call, such as "Make it faster." Each went once to Architect's own coder on Nemotron 3 Super, against the demo agent shown at the top of the page (Super, 59 of 100, pass mark 40).
- 8
changed the agent. All 8 stopped, and the app's next call made 0 model calls.
- 3
of those were retaken and passed, with 48, 46 and 48 of 100.
- 5
could not be examined: a model the settings do not have together with a changed budget, settings out of range, JSON that no longer parses, and an edit to generated code. They stayed stopped until Undo.
- 2
changed nothing, though the coder reported the work as done.
- 0
took a retake and missed the pass mark.
So the trial does not show the exam catching a change that made the agent worse, and it does not show any change improving it. I didn't measure whether the five edits that couldn't be examined would have broken the app.
By hand, outside the ten, I switched Super to the cheaper Lightning. It scored 29 of 100 on its retake, did not pass, and stayed stopped until Undo. This is one build, one agent and one coder model, on a demo exam of development receipts. The trial's table has every row.
What it does not do
The rule is a tripwire for a change nobody reviewed, not a lock. It stops calls made through the agent's own folder, so a coding agent can still edit the code that calls it, and someone who controls the computer and the signing key can get past it. Stopped means not examined, and passed does not mean better. A retake can examine a change to the task and rules text, the checks, the model and the settings. A change to anything else, such as the pass mark, the budget or the generated code, can only be undone. Architect won't retake an exam on a published set of cases, so the published receipt reader can only be undone. There's no plugin for Claude Code or Codex. The stop is file-based, and I've only recorded it with Claude Code and by hand.
Watch
The video is a reconstruction from recorded runs, not a screen recording. The app's own interface code is fed the numbers and lines those runs recorded, and the pointer presses are re-enacted. The film typesets a few cards itself (the terminal, the two job cards), and the Undo press stands for the undo command I ran. The narration is an AI voice (Bella, ElevenLabs), and the music is "Closer" by Andrew Ev, from Mixkit.
Try it
Install the app, connect Nebius Token Factory with an API key, and describe the agent you want. The app needs nothing else installed and sets up Python the first time it opens.
Or run the terminal agent from a checkout, with git, uv and Node 20.6 or later. On the first run, choose Nebius Token Factory and paste the key.
git clone https://github.com/echoo19/architect-nemotroncd architect-nemotronuv sync --frozen --python 3.12npm ci --no-audit --no-fund --prefix architect/backends/pi-nodeuv run architect
- Walk-through for judgesBuild the demo agent, change it and watch it stop.
- The retake trialThe ten requests, row by row, and what the trial does not show.
- The published resultsThe receipt reader's evaluation and its plan.
- LimitationsWhat the checks on your computer can and cannot show.