Jev: how my factory decides what to build next

For most of September my agent factory had a habit I didn't like. Given a choice between paid work and fixing itself, it fixed itself. Every time.

That isn't as daft as it sounds. Each repair was real. Each one, on its own, made sense. But one evening at the end of September it opened six repair tickets against four real builds, the real builds queued behind them, and by the morning there were ten open tickets and nothing in a slot. I'd asked for a factory to build things. I had one that mostly maintained itself.

This is how I fixed that, with a model called Jev and a scoring rule that doesn't use a model at all. It's also about a wider idea that has changed how I think about AI costs: most of the decisions in a system like mine are multiple-choice, and you shouldn't pay a writer to tick a box.

Three rungs

I first read about Jev in September, in a write-up by Chris Parsons. Jev is made by a company called TypeSafe. It's what they call a decision model. It can't write a sentence. It can only do three things: pick one option from a list you give it (up to 255), score something against a scale you describe, or answer yes or no as a probability with a confidence figure. Because it never writes, it's extremely cheap. At the published price, input costs a few pence per million tokens (chunks of text) and output is free, and it answers in well under a second.

When I looked at my own system, I realised how many of my AI calls were really one of those three things. Is this message a log entry or a question that needs a reply? Which category does this bank transaction belong to? Is this stop something that needs me, or something the factory can handle? Each of those was waking a full model, with all the context that comes with it, to produce what was effectively one word.

On 22 September I set it down in a ticket as three rungs, with a rule to always use the lowest one that works:

1. A plain script. No model, free, exact. Anything a rule can decide: is the process alive, does the log say the quota ran out, is this label present.
2. A decision model. Almost free, fast, usually right. A judgement a rule can't make, but which ends in a choice, a score or a yes or no.
3. A full model. The writer. Writing, planning, reasoning, building, conversation.

The cascade runs in that order. If the script can't decide, the decision model tries. If its confidence is below a threshold (I started at 0.8), it passes up to the full model. The confidence figure is what makes that hand-off safe.

The sorting clerk

On 30 September I gave Jev two jobs in the factory and a nickname to go with them: the cheap sorting clerk.

The first job came from a frustration with the factory's supervisor. I run an orchestrator, a Claude session that keeps the factory moving. Its rule is that it's the foreman, not the builder: it files tickets, restarts things and tells me what's true, but never edits code. Rules written in prose kept getting bent, so I wanted something structural deciding what the orchestrator may and may not do.

The second job was the hand-offs. The factory moves work through stages: shaping a ticket, building, verifying, reviewing, landing. Too often a stage would pass on something the next stage couldn't use, and the next stage would fail, retry, or ask me a question. My instruction was that nothing moves between stages unless it meets the format the next stage needs. Jev checks each hand-off. Anything malformed goes back to the same stage to be reformatted. That's not a re-run, not a fault, not a question for me.

What the checks found

Between 6 and 9 October the checks logged 410 decisions, and the numbers were sobering.

The biggest group was questions meant for me. When the factory needs a decision, it puts a question on a single phone page I review (I call it my reviewsheet). Before a question reaches that page, Jev checks it against a short list: is it in plain English, does it have one idea, does it come with a recommended answer, has it already been answered? Of 347 questions checked, 25 passed and 322 were sent back.

The reasons were almost all about plain English. 190 failed on simple words. 170 failed for carrying more than one idea. 121 failed on sentence length, and 116 on strings of jargon nouns. 34 had no recommendation at all, and 5 were questions I'd already answered.

That stung a bit, because those were questions that would otherwise have landed on my phone. It also explained a lot about why reviewing the factory's work had felt like hard going.

Drafts for articles and posts got the same treatment against different checks: does it start from the reader's question, is it specific rather than generic, does it sound like a person, is there a clear next step? 47 passed and 12 were sent back, all 12 for not starting from what the reader actually needs to know.

None of this is the model being clever. It's the model being consistent, cheaply, thousands of times, on checks I'd otherwise skip because they're tedious.

Jev System One: what fills the next slot

The bigger problem was still priority. Which ready ticket should the factory start next?

Until October the answer was a fixed rule: a set number of slots reserved for factory work, the rest for everything else, in a fixed project order. That's how six repair tickets ended up ahead of four real builds. On 29 September I added a stopgap: at most one slot may ever hold a repair ticket. On 5 October I replaced the fixed split properly with ticket #302, which I called Jev System One.

Despite the name, it doesn't call a model at all. It borrows the idea of fast, cheap judgement and makes it a fixed formula, run every time a slot comes free. It asks four questions of every ready ticket:

• Importance. What's the commercial revenue impact? A ticket for one of my products counts as commercial because of what the product is. A factory ticket counts as commercial too, but only if it genuinely gets a product to revenue faster.
• Urgency. How many days until its deadline? A deadline written on the ticket is a commitment and always counts as urgent. A deadline it finds by reading my calendar, email or WhatsApp (read-only) counts too, with the evidence noted.
• Confidence. How much of that came from something explicit on the ticket, and how much was a default? This is logged next to the score, never hidden.
• Quadrant. Importance against urgency puts each ticket in one of four boxes: do now, schedule, do cheaply, or don't start.

It's the old Eisenhower matrix (sorting by important and urgent), applied to tickets. The factory fills its slots from the top. The reviewsheet shows me every ticket in that order with its quadrant, score and a one-line reason, so nothing quietly drops off the bottom.

If I need something to jump the queue, I can say "now" and the ticket number. It pushes back once, showing me the ticket's score and what it would displace, and only goes ahead when I confirm. One pushback is enough to stop me pulling rank on a whim, and not so much that it argues with me.

The paragraph that did nothing

The first real test came overnight on 8 to 9 October, and it caught me out.

One ticket, #336, set up a cloud lane for the factory using some free capacity that was due to expire. I'd asked for it to be at the top. It sat 15th of 23, with an importance of 0.52. Someone had added a paragraph to the ticket headed "Commercial value, stated plainly for Jev", explaining why it mattered.

It made no difference, because the scoring doesn't read prose. Importance comes from the project's kind plus two fixed lists of phrases: one for direct revenue (revenue, paying customer, checkout and the like) and one for factory work that gets the commercial products to revenue faster. When the same true claim was rewritten in those terms, the ticket went to 0.75 importance and an overall score of 0.84, joint second.

I made it a rule that night: importance is read from the project type and those two lists, never from prose, and the way to raise a ticket is to say plainly how it speeds up revenue, and only when it's true. Nobody gets to lift a score by editing the formula.

My first reaction was that this was a limitation. I've changed my mind. A system that can be persuaded by a well-written paragraph will end up ranked by whoever writes the best paragraphs, and in a factory full of AI agents that's the AI. I'd rather the rules were dull and visible.

If you want to try this

You don't need my factory, or Jev, to use the idea.

1. List the decisions your AI makes in a week. Mark each one: rule, choice, or writing.
2. Take the rules out of the AI entirely. If a script can decide it, a script should.
3. Use the cheapest thing that can make each choice. Keep the expensive model for writing and reasoning.
4. Check hand-offs, not just final results. Most of my wasted runs started with one step passing the next something it couldn't use.
5. Rank work from fixed facts. What kind of work it is, and when it's due. Not how well it's argued.
6. Cap the maintenance. Decide in advance how much of your capacity can go on fixing the system itself, and watch the ratio.
7. Show the whole queue, in order. If you can see every item ranked, you'll notice when something important is sinking.

What's still open

The priority score is only as good as the deadlines it knows about, and plenty of my work still doesn't carry one. The check on questions is strict, possibly too strict in places, and I'll tune it. And Jev hasn't yet taken over the jobs I first listed for it, like sorting incoming email and messages. That's next.

What I'm sure of is the principle. When I wrote about building the factory, the hard lessons were about trusting what it told me. This one is about trusting what it chooses to do. In both cases the fix was the same: take the judgement out of the clever part and put it somewhere fixed, cheap and visible.

If you've got AI doing work for you and you're not sure it's working on the right things, let's talk.