OpenJev is a live demo that lets you watch a model generate its own probability distribution token by token. It is a small, cheerful experiment, and it is worth explaining in its own terms.
The demo asks a simple question: how long does it take a local model to read its own mind versus write it down? You pick a model size, set it running on your own GPU, and measure the difference between reading probabilities directly and asking the model to write them out as JSON text.
The Setup
The demo’s weights come from Hugging Face and remain in your browser cache. Inputs never leave the page. First load can take several minutes, depending on the model, network and GPU.
MiniCPM5 2B is the default choice, but on a phone or smaller device you can switch to Qwen3 0.6B in the model box. Larger models load slower and may not fit on low-end devices.
The demo uses pinned GGUF builds through wllama, and quantization changes both quality and speed. Native BF16 and TypeSafe are also noted, though the demo leans on a 102-row subset of weights.
Reading vs Writing
The demo runs two paths sequentially on the same loaded model, so they do not contend for one GPU. Direct runs happen first, followed by generation.
Direct readout works by reading the model’s choice logits and normalizing only across the options you supplied. It is a softmax over only the displayed option tokens, not a calibrated confidence score.
Generation asks the model to estimate the same displayed-option distribution and write it as JSON text. You watch every token arrive.
The two paths receive the same decision, but the method differs entirely. One reads probabilities directly; the other asks the model to write them.
What the Demo Measures
The demo times everything with performance.now(). Setup, warmup, prompt preparation, direct execution, first generated token and generation completion are all recorded.
There is no waitlist. You simply try it out.
Conditional probabilities are a key part of the demonstration. Direct scores are a softmax over only the displayed option tokens. They are not calibrated confidence and do not include every answer the model might prefer.
The demo is explicit about its limits. The phone model trades accuracy for size. MiniCPM is the desktop default. The 4B option needs substantially more memory. None of the tiers is claimed to match Jev.
Why It Works
The charm of OpenJev is its transparency. It is a demo about a particular capability, and it treats its own workings as the subject. The fact that you can watch a model generate its own probability distribution token by token is the whole point.
The comparison is clean. Direct readout and generation each follow a distinct path, and the demo lays out both clearly.
| Method | How It Works | What You See |
|---|---|---|
| Direct readout | Reads logits and normalizes across supplied options | Conditional probabilities, no decoded text |
| Generation | Asks the model to write the distribution as JSON text | Every token arrives one at a time |
The demo does not hide its mechanics. It explains that direct scores are a softmax over only the displayed option tokens, and it notes that they are not calibrated confidence. That honesty is part of the appeal.
The Verdict
OpenJev is a small, clever demo that rewards curiosity. It is a live experiment you can run in your browser, and it measures something real: the difference between reading a model’s mind and asking it to write down what it knows.
The demo is not a benchmark in the usual sense. It is a demonstration of a particular capability, and it shows it off with full transparency about its own workings.
It is also a reminder that models are tools, and that their behavior can be observed and measured in real time. Watching a model generate its own probability distribution is a small pleasure, and the demo delivers it without fuss.
The demo is a small, cheerful experiment that earns its attention. It does not pretend otherwise.
Source material: “OpenJev,” openjev.com.
Get the Notebook.
The day's best stories and every fresh verdict, in plain English, in your inbox by seven. One email a day, no more.