███╗ ██╗ ██████╗ ███████╗██╗ ██╗
████╗ ██║██╔═══██╗██╔════╝██║ ██║
██╔██╗ ██║██║ ██║███████╗███████║
██║╚██╗██║██║ ██║╚════██║██╔══██║
██║ ╚████║╚██████╔╝███████║██║ ██║
╚═╝ ╚═══╝ ╚═════╝ ╚══════╝╚═╝ ╚═╝ networked orchestrated science harness
Autonomous research you can verify.
NOSH is a local-first research harness. Model agents propose and carry out research. The noshd daemon runs the commands, measures the results, and decides what counts.
npm install -g nosh-harness
question does early exit cut inference cost ≥ 30% with accuracy drop ≤ 1% of the split?contract frozen · guardrail acc_drop ≤ 0.010 min effect cost ≥ 0.30 baseline cost −0.167 acceptedexp 3.1 T=0.75 cost 0.375 drop 0.004 passexp 3.2 T=0.70 cost 0.455 drop 0.008 passexp 3.3 T=0.60 cost 0.695 drop 0.020 guardrail fail review independent reviewer agrees on 3.2decision promote exp 3.2 · reject 3.3
Models decide. The daemon executes and judges.
Agents never get the final word on their own work. Model prose never changes state by itself.
Agents
- Propose experiments, plan Missions, and write code in their own Git worktrees.
- Make judgments, and return them as one schema-checked JSON envelope per task.
- Ask the daemon to run the commands declared in the research contract. They never get a shell.
- Run as bounded threads that start from compact, hashed Episodes instead of whole transcripts.
- Write a short program to read, check, and edit several files in one step. Each call inside it is still checked and recorded.
Daemon noshd
- Owns scheduling, Git, Jobs, budgets, validation, and acceptance.
- Fills each task envelope with its own facts (IDs, commits, commands, fingerprints) and issues a receipt. The receipt decides acceptance.
- Commits a worker's edits itself, before every run and after the final answer.
- Checkpoints orchestration state after every transition and resumes after a restart.
Pi is the model runtime, Git holds reviewable research content, and SQLite holds operational state. You drive it all from a terminal UI built with OpenTUI.
Everything belongs to a Project.
Inside a Project, work comes in two shapes. A Mission pursues a goal. A Research Direction answers one question. Both write into the same evidence record, and that record becomes the paper.
-
ProjectThe unit of research, roughly one paper
One Git repository plus an approved research contract. The contract sets scope, datasets and licenses, and the test and evaluation commands that workers may run. Each Project has its own database, branches, worktrees, and budgets, and no context leaks between Projects. Change the contract later with
/amend-contract.-
Research chatWhere you work with the Project agent
Discovery happens here. It asks one question at a time and proposes the contract. Afterwards, this is where you ask questions, plan, stage work, and start Missions and Directions.
-
MissionA goal with a finish line
An approved objective with success criteria, budgets, and a final review rubric. A Mission Director plans the work as a graph of Nodes and repairs the plan as results come in. Accepted work merges onto the Mission's own branch,
nosh/mission-<id>, so you merge one branch at the end.-
NodeOne step of the plan
Each Node has a type, and it runs only after the Nodes it depends on are accepted. The type decides who does the work: a librarian-researcher for literature, a reviewer for reviews and approvals, and a general worker for everything else.
literature_review · reproduction · implementation · dataset · analysis · research_direction · autoresearch · integration · synthesis · paper_section · claim_review · approval · final_review-
TaskOne worker doing one bounded job
A single agent session in its own Git worktree, with a token budget. It finishes with one typed JSON envelope. The daemon checks the work, and then a separate reviewer decides whether the Node is accepted.
-
-
-
Research DirectionOne question, answered rigorously
A falsifiable question with a scope, stopping rules, and a frozen evaluation contract, so every result inside it is comparable. A Direction can stand on its own or serve as one Node of a Mission.
-
BaselineThe starting point everything is measured against
The daemon runs the evaluation on the unchanged code, and a reviewer accepts the numbers. The baseline is the root of the experiment tree.
-
AutoresearchA search loop under fixed rules
Runs rounds of experiments against one primary metric. It enforces guardrails, a minimum meaningful effect, and limits on iterations and compute. It can also run on its own, outside a Direction.
-
ExperimentOne measured variant
A change on its own branch with exactly one parent, compared against that parent and the baseline. Once an experiment is evaluated, its code is frozen, and any further change becomes a child experiment. See how an experiment is kept.
-
-
ClosureThe Direction's answer
The Direction closes as supported, refuted, or inconclusive, with a reviewed closure packet. A negative answer still counts as an answer, and it stays in the record.
-
-
EvidenceHow results become a paper
Missions and Directions both write here. Reviewers read these records for themselves instead of trusting a worker's summary.
ArtifactA content-addressed output: a diff, run result, source, or report.EvidenceAn Artifact with a meaning, such as a baseline run, an ablation, a citation, or a review verdict.ClaimA statement linked to the evidence that supports, contradicts, or qualifies it.PaperBuilt from reviewed claims, and exported to LaTeX or PDF.
-
A model can never declare its own result a success.
Every experiment runs under the same frozen contract. Code decides whether it is kept, and an independent reviewer has to agree.
Proposal
An agent proposes a change and the effect it expects.
Implementation
A worker builds it in its own Git worktree.
Evaluation Job
The daemon runs the contract's evaluation command as a supervised Job.
Result
Metrics and guardrails are checked against the frozen contract.
Review
A separate session reads the diff and the run records, not the worker's claims.
Decision
Only reviewed winners are promoted.
promoteholdreject
keep = run.valid
∧ every guardrail passes
∧ improvement ≥ contract.min_effect
∧ independent review agrees
Decided in code by the daemon. The worker that ran the experiment has no say.
spent = input + output
+ cache_writes
+ 0.1 × cache_reads // per task and step
Every task and step has a budget. Overruns are recorded, never silent.
From a repository to a reviewed answer.
- Turn a repository into a Project
- Discovery proposes a research contract with your test and evaluation commands. On approval, NOSH commits just the contract files.
- Run Missions
- Pause, steer, retry, or stop at any time. Every result is reviewed before it counts.
- Answer research questions
- A Direction measures a baseline, runs Autoresearch, and closes as supported, refuted, or inconclusive.
- Search for improvements
- Autoresearch evaluates each experiment under the same contract, enforces guardrails, and promotes only reviewed winners.
- Keep long calls off the critical path
- Tool calls in one turn run in parallel. A slow test run returns a placeholder, and the result wakes the agent when it lands.
- Work through many files in one step
- Agents write a short program that reads files together and keeps only what matters. In our tests, a 13-file task used about a fifth of the tokens.
- Run your commands safely
- Workers never get a shell. They ask the daemon to run commands declared in the contract, each as a supervised Job.
- See what is happening
- Press Ctrl+G in the terminal UI for a live map of Missions, Directions, experiments, workers, and Jobs.
- Use any model account
- Sign in with a ChatGPT or Claude subscription, or use an API key from OpenAI, Anthropic, Gemini, OpenRouter, DeepSeek, Groq, Mistral, xAI, and others.
- Survive restarts
- Closing the terminal UI does not stop the daemon or its Jobs. Orchestration resumes from its last checkpoint.
Runs on your machine. Talks only to your machine.
The daemon, your repository, its runs, and its evidence live on your computer. The control API never leaves loopback.
- loopback only
- The API binds to loopback. A request is trusted only with a loopback
Hostheader and no foreignOrigin, which rejects cross-origin and DNS-rebinding browser requests. - short sessions
- Any other caller needs a 15-minute bearer session, issued only in exchange for the bootstrap capability in your per-user state directory.
- no worker shell
- Workers can only ask the daemon to run commands declared in the contract. Each one runs as a supervised Job.
- sandboxed programs
- Agent-written programs run in a WebAssembly interpreter on a separate thread, with no files, network, or shell of their own. They can call only the tools the task already has.
- credentials
- Pi's runtime stores your model credential. NOSH never shows it, and API keys are masked while you type them.
- scope
- Do not expose
noshdto a LAN or tunnel. It does not protect against malware running as your own user. Read the threat model.
Three commands to your first Project.
The npm package bundles its own Bun for the terminal UI. You don't need pnpm or a build step.
- node22.19 or newer
- gitany recent version
- modelChatGPT Plus/Pro or Claude Pro/Max, or an API key
- osWindows first; WSL and Linux supported
npm install -g nosh-harness nosh setup nosh open
nosh setup checks your tools, connects a model account, starts the daemon, and picks a default model. In the terminal UI, press Ctrl+O to open a repository as a Project. When a new release comes out, nosh tells you, and nosh update installs it. Read the quick start for the full walkthrough.
Run from a source checkout
Building from source needs pnpm 10.28 through Corepack, and Bun 1.3 or newer on your PATH for the terminal UI.
git clone https://github.com/SirNosh/NOSH.git cd NOSH corepack pnpm install --frozen-lockfile corepack pnpm build node apps/cli/dist/main.js setup node apps/cli/dist/main.js open