How a deployment is built.

Every deployment goes through four phases: preparation, model, harness and finalization.

1 Preparation

We work with you to learn three things.

Your hardware
VRAM and compute. The agent is then targeted at that hardware and tested against it. The smallest machine we build for: one GPU with 16 GB of VRAM.
Your use case
Math, physics, chemistry, electrical engineering and computer science, and so on. It decides the toolkits and the agent's workflow.
Your data
Any data of yours the model could be specialized on.

2 Model

We start from a strong open-weight model that fits your hardware. It can then be specialized two ways.

For our framework
A QLoRA fine-tune that makes the model more dependable inside our agent framework: its interface, tools and workflow.
For your field
A fine-tune on domain-specific material.

We can also distill a stronger model into the smaller one, focused on your use case.

We are benchmarking open-weight models now, to know which to start from on which hardware. Fine-tuning and distillation are planned: this is how we will work, not something we have shipped.

Where a fine-tune runs, and whether your data leaves your machine for it: it runs on our own machine, not on a cloud service, so a fine-tune on your data means sending us that data, and one for our framework needs none of it.

A fine-tune leaves the base weights as they are and trains two small matrices beside them (brass). Pick their rank and count what is trained.
Rank
base
4096 x 4096 = 16,777,216, not trained
adapter
2 x 4096 x 16 = 131,072, trained
share
0.78 % of the layer

One layer, 4,096 wide, as an example. The blocks are drawn thicker than scale so they can be seen.

3 Harness

The harness is what turns a local model into a working agent. It has five parts, and it is the part of Herth that is built and working today.

Below is a small model of it that runs in your browser. Follow one unit of work round the loop and watch the record fill. Then change what the model proposes, and see the gate or the check stop it.

How the agent is kept off the network: the agent's process runs in its own network namespace with no interface to the outside, so the operating system refuses every connection it tries and the harness records the attempt as blocked.

  1. Context

    What the model is given at each step.

  2. Model

    Proposes the next step. On this page its turn is scripted.

  3. Gate

    Tool calls are checked before they execute.

  4. Tools

    Toolkits built for your use case.

  5. Verify

    Hallucination checks on what the model produces.

Transparency

You can see what the agent did and how. Every part of the loop writes a row.

The record of one lap of the loop
StepPartWhat it didStatus
01contexttask, 8 values, 3 toolsok
02modelproposes mean(od600)ok
03gatemean is on the listok
04toolsmean(od600) = 0.4275ok
05verifymodel wrote 0.43, tool returned 0.4275ok
06contextresult addedok
lap6 rows, 0 blockeddone

A miniature of the loop, written for this page. No model runs here: what the model proposes is scripted, and you choose the script. The gate, the tool and the check are executed in your browser, on eight sample values.

4 Finalization

The model and the harness become one agent, on your machine.

Quantize
We quantize the model to fit your hardware. The figure works the sum for any model and any machine.
Integrate
The quantized model is integrated with the harness.
Benchmark
Then we benchmark the whole agent on that hardware: the model and the harness together, on the machine it will run on.
Quantization stores each weight in fewer bits, so it can take fewer values. Above, a curve snapped to those values. Below, what the weights then weigh, against the memory they have to fit in (brass).
Bits per weight
27 B
24 GB
levels
2^4 = 16
rms error
1.85 % of the range
weights
27e9 x 4 / 8 = 13.5 GB
memory
24 GB: fits, 10.5 GB to spare

Weights only, and every weight at the same width. Real formats keep some weights at more bits, so a file labeled 4-bit averages more than 4 bits a weight and is larger than this sum. A running model also needs memory for the text it is working on. Leave room for both.

Bring Herth in

Tell us what you work on and what you would hand to an agent first. Every deployment starts with that conversation.