A developer exploring the intersection of language models and computer vision has published a technique that lets any multimodal model classify images using probability scores rather than raw text output. The approach, built around a system called Jev and extended to support vision inputs, turns a language model into a flexible image classifier by reading the probabilities the model assigns to each possible answer.
The Probability Trick Behind the Approach
The core idea is surprisingly simple. Rather than asking a language model to describe an image in free text and then parsing the answer, the technique asks the model to choose between lettered options and then reads the log probabilities the model assigns to each alternative token. The model is given a structured prompt with a state description, a question, and a set of lettered choices. It is asked to answer with the letter of the best option only.
By requesting the log probabilities of the top candidate tokens, the API returns not just the winning answer but the model's confidence in each option. The request parameters are minimal: a single maximum completion token, log probabilities enabled, and a specified number of top log probabilities to return. Forcing the model to generate only one token keeps the response fast, though the input processing still costs time. When the same state prefix is used across multiple questions, the backend can cache it and reduce redundant computation.
The technique itself is not new. It has been documented in OpenAI's logprobs cookbook and used by projects such as OpenJev and SemIf. What makes the approach noteworthy is how easily it extends to vision models.
Adding Images to the Jev Framework
The documented Jev request format covers text and JSON state only. The author, Allan Rasmusson, added an attachments field that accepts image file paths or base64 data URLs. The images are loaded once and sent alongside each question, so the model evaluates every image against every criterion.
He built a working example that captures frames from a webcam, encodes them as base64 JPEGs, and sends them to either a local llama.cpp server or the OpenAI API. The tool asks three questions per frame: is a person visible, are we indoors or outdoors, and how bright is the scene. The results are printed as a table with timestamps, answers, and frame rate.
In testing, running Gemma 4 12B locally on an RTX 3090 produced about one frame per second with three questions per frame. The same pipeline against OpenAI's gpt-6-luna produced about 0.2 frames per second, likely because each question required a separate round trip through the cloud API rather than a single local inference.
What Rasmusson values most is the flexibility. Rather than relying on a specialized computer vision model that expects a fixed set of categories, he can describe any classification condition in plain text and let the model handle it. Changing a criterion means changing a line in the prompt, not retraining or swapping a model.
How the Scoring Works
The implementation normalizes the returned log probabilities into a proper distribution. For each question, the alternatives are mapped to lettered options. The model's log probabilities for each letter are retrieved, and any missing tokens are handled carefully. If the model omitted non-negligible option scores, the script raises an error rather than silently producing an unreliable result.
The scoring logic adapts to the question type. For multiple choice questions, it returns the winning option and the probability distribution across all choices. For boolean questions, it returns the probability that the answer is true. For ordinal score questions, it returns the expected score weighted by the model's probability distribution across the criteria levels.
The script handles an important API difference: llama.cpp uses the Chat Completions endpoint and returns log probabilities in one format, while OpenAI uses the Responses endpoint and returns them in another. The code abstracts both paths so the same scoring logic applies regardless of the backend.
What This Means in Practice
The approach demonstrates that any multimodal language model can function as an image classifier without specialized training or a dedicated vision pipeline. The classification criteria are expressed in natural language, and the model's own probability estimates provide not just a label but a confidence score for each option.
This is not a replacement for specialized computer vision systems, which will always be faster and more efficient for well-defined tasks. But for scenarios where the classification criteria change frequently, are described in plain language, or require combining multiple conditions, the flexibility of using a language model as a vision evaluator is a meaningful advantage.
Rasmusson published the standalone Python script and the setup instructions for running it with either a local llama.cpp server or OpenAI's API. The code is available under an open license, and the author notes that the attachments field was a custom addition to the Jev-style request format, suggesting the approach could be adopted by other projects in the same space.