Privatemode Researchers Adapt GLM-5.3-Flash to Mimic Jev’s Decision-Making Process

According to Johannes Hötter, VP Growth, and Marko Rosenmüller, PhD, Technical Lead AI, at Privatemode, an LLM can now render typed decisions in a single forward pass, equaling the precision and pace of Jev. The technique transforms GLM-5.3-Flash into a Jev-like decision model without requiring any fine-tuning of the underlying LLM.

In principle, the approach is straightforward, even if the specifics prove otherwise. Rather than requesting that the LLM produce a complete JSON reply, the researchers design prompts that compel the model to choose an index from a numbered list of choices. During one run, the LLM produces probabilities for each option, which the system then reads directly without having to parse a generated token string. This yields a decision paired with a confidence score, ready for high-volume deployment.

How the Prompt Works

The main point is that an LLM never produces text on its own. Instead, it gives a probability distribution across its full vocabulary of tokens when presented with a prompt. During text creation, the token carrying the highest probability gets chosen as the next token in the sequence. This method becomes costly and slow when all you need to do is fill in a handful of fields within a JSON object.

The investigators recognized that the LLM’s task did not require predicting the entire JSON object, since the format of the output was predetermined. They devised prompts that incorporated the current state, the question at hand, and a list of named options, each assigned an index number. The instruction directed the model to respond with a choice_index followed by its corresponding index number.

The prompt closes with a choice_index, which compels the model’s initial output to point to an index within the set of available options. From there, the system examines the probabilities tied to every option index at that spot, adjusts them across the full set of choices, and settles on the one with the highest probability.

Why This Matters for Decision Making

When it comes to resolving support tickets or deciding which team should handle them, most software turns to an LLM for judgment. The same applies to contract review tasks, such as determining whether a clause belongs in the liability section. In both scenarios, the software expects the LLM’s output to be formatted as JSON and restricted to a fixed set of possible answers.

The typical method works well when the directions are correct, yet it moves at a deliberate pace and costs real money. Each choice forces the LLM to compose a complete JSON object from scratch, while a reasoning model might deliberate over hundreds of tokens before settling on an answer. Confidence scores aren’t part of the output unless the request specifically asks for them.

Jev and Laya are specialized decision models built around this design. They take a piece of state along with a list of named options and return both the selected option and a confidence score for each one. The fresh approach is meant to offer the same qualities using an ordinary LLM instead.

What GLM-5.3-Flash Brings

GLM-5.3-Flash ran through Privatemode during the testing phase, and its performance was judged against a benchmark built from public data sets. The researchers demonstrated that the setup matches TypeSafe’s Jev in both decision accuracy and speed.

This arrangement allows for typed judgments on pictures, something Jev cannot do. It stands as an extra feature rather than the primary aim.

The Technical Details

The team employed vLLM, the library behind GLM-5.3-Flash, to carry out the method. They accessed the /chat/completions endpoint while setting continue_final_message and add_generation_prompt to false. These configurations allowed the model to keep going from where the assistant’s reply had already begun at step two rather than opening a fresh one, and they permitted images to be sent along with the text.

Several model-specific details mattered for the implementation:

• allowed_token_ids can be used to limit the LLM’s output vocabulary to only allowed options. It drops every other token to -inf.
• top_logprobs is not enough for step three. It reports the distribution before the restriction is applied, so formatting tokens such as a leading space take up the top slots and some options drop off the list.
• vLLM’s logprob_token_ids solves this by returning the log probability of exactly the token ids you ask for.
• The library gets the token ids from the server, which keeps it simple to use with any model. Sending a prompt to /completions with echo returns its exact tokenization by the model that is actually serving.

The researchers made the code available to everyone, posting it in the edgelesssys/privatemode-decisions repository. That Python library handles token oracle, prompt construction, masking, and renormalization against any vLLM-backed endpoint.

Testing It Yourself

A running playground operates GLM-5.3-Flash with the setup applied on Privatemode, launched straight from your browser window. Choose among several examples, such as a scanned invoice or a question tied to your local time, or compose your own queries and attach images.

A response usually returns as a set of probabilities across its possible answers, often arriving within a few hundred milliseconds. The display gives a likelihood figure for each option along with an indication of certainty.

The system is end-to-end encrypted, so what you type stays private.

What This Means for LLM Deployment

Teams can now use an LLM as a decision engine without having to pay for fine-tuning or suffer the latency of full token generation. The approach supports GLM-5.3-Flash and vLLM, and the library handles tokenization behind the scenes so users don’t need to manage it themselves.

It marks a genuine step forward for software that must reach a large number of conclusions rapidly, particularly when the potential outcomes are fixed from the start. The picture processing comes as an extra benefit, given that Jev has no image-handling capability whatsoever.

Those with an interest in trying it out may launch the playground themselves, or they may bring the library into their own deployment pipeline instead.

Source material: “Turning GLM-5.3-Flash into a Jev-like decision model,” privatemode.ai.

AI · decision model · GLM-5.3-Flash · Jev · LLM