An inspectable local interface for typed decisions from frozen language models
Yuki Oshio / UPHASH Inc.
Give a language model a state, explicit criteria, and a set of candidates. Inspect the typed decision, the candidate distribution, and the request that produced it.
Import your own JSONL records → Compare Choice, Noul, and Score distributions locally in your browser. No upload or model download.
Research update 5 October 2026 · Research manuscript · Not peer reviewed
The linked PDF was submitted to EACL 2027 System Demonstrations on 6 October 2026. A decision is pending; no acceptance is claimed. The earlier arXiv submission retains its “Mini Jev” title and frozen files; no public arXiv identifier has been verified as of 6 October. Repository and page URLs retain mini-jev so existing links continue to work.
Reviewer quick start
Inspect first. Recompute next. Run live if needed.
The saved Explorer and numerical replay work without model weights. Live inference is a separate, optional setup.
1
Explore saved records
Use a browser to compare two retained conditions for the same item and model. Inspect probabilities, typed values, shared physical calls, and exact source references.
Static saved evidence, not a running model or a new experiment. The browser loads the selected panel.
2
Verify and recompute
Python 3.10+ and the standard library run integrity checks and six frozen analyses. The numerical bundle is 18.06 MB; no pip packages, model, or accelerator are needed.
Fresh isolated source and Python environment on the same Mac; not a second-machine replication.
3
Optionally run live
Build the native helper and download the separate 20.4 GB model. The tested native setup is Apple Silicon/macOS, with 64 GB memory on the measured machine.
Native build and model download are not part of the lightweight review route. No lower-memory minimum is established.
The interface
One model. Three kinds of decision.
InputState + criteriaExplicit candidate meanings
→
Frozen modelCandidate logitsRead at the next-token position
→
Typed outputChoice · Noul · ScoreInspect probabilities and JSON
CChoice
The most likely candidate, returned as its semantic label.
NNoul
The probability assigned to the true candidate within a false/true pair.
SScore
The expected zero-based stage index, assuming equally spaced ordinal stages.
The released native system uses a frozen model with no trained decision head. The public API and browser expose candidate probabilities and typed values; raw candidate token IDs and logits remain in internal engine/research traces. Probabilities are normalized over the supplied candidates; concentration is not the probability of being correct.
127.08 seconds · Saved-Explorer interaction plus a clearly labeled 16-second archival live-interface excerpt · Embedded English captions; no audio · Open the video directly · Try the Explorer · Transcript
The current recording shows actual interaction with saved experiment records; the labeled archival excerpt shows the older live interface. It adds no new model inference. Running a model yourself requires the separate local API. Recorded demonstrations and software checks do not add experimental observations or establish human usability gains.
Earlier recordings · preserved for context
60-second evidence walkthrough · Retained experiment results and an archival interface excerpt, not a new live run · Open directly · Transcript, sources & CC BY-SA licenseEarlier Mini Jev interface recording · About 145 seconds · Real local-model interaction under the former name · Open directly
The older interface recording and poster retain the former name and source branch; later research integrations are not shown. Their English examples illustrate the interface. The original three studies use Japanese data; a separately linked follow-up now includes English five-choice evaluation.
What we measured
A system you can inspect. Evidence you can check.
The contribution is a runnable local workbench and an auditable evaluation of its behavior. Candidate scoring and model-inspection interfaces have prior art, discussed in the paper.
1,350 / 1,350
Matched decision pairs
Direct readout and native one-token generation produced identical labels and candidate logits in every matched pair.
No latency advantage over native one-token generation was established.
26,050
Original three studies
Measured requests from one matched output-mode study and two Japanese-data studies of presentation sensitivity and averaging across three checkpoints.
Requests are not independent questions. The later 2,400-call and 4,800-call follow-ups are separate packages.
56 / 56
Analysis outputs reproduced
CPU-only reanalysis reproduced the derived outputs byte for byte from retained records.
This is analysis replay. Native build and smoke checks used the same Mac and verified caches, not an independent replication.
A separate follow-up adds 4,800 measured calls: 240 Japanese JCommonsenseQA questions and 240 English CommonsenseQA questions, each evaluated by Qwen2.5-1.5B and Phi-4-mini. In the primary Phi-English comparison, entropy+order-TV and entropy alone have identical per-item correctness, 174/240 at 720 calls; fixed-random and first-entropy references score 180/240 and 177/240. Added order-TV benefit remains unestablished. All eight scores, both error targets, and four policies across five budgets are retained in the linked package.
A post-hoc control reuses those same 4,800 responses to compare arithmetic and normalized geometric averaging. Changing only the averaging rule changes 4–8 labels per 240-item stratum, without a consistent accuracy gain. Both rules share the same physical calls. All four strata and paired exploratory intervals are retained; this is no new inference or full AnyJev reproduction.
Scope, limitations, and prior work
The original three studies use Japanese public-data subsets; the latest separate follow-up covers Japanese and English five-choice tasks from two model families. All were run on one Apple Silicon Mac. The 480 follow-up questions are shared by two models, not 960 independent questions. Public-data contamination is unknown, and model family, size, language and dataset effects are not isolated. The separate 2,400-item local regression suite is AI-authored. No human observations or independent second-machine inference have been collected.
Probability averaging requires multiple model calls and does not consistently improve accuracy. The original model comparisons also differ in runtime and precision. Follow-up policy token and latency sums are saved-pool accounting, not measured online speedups.
LogitTrail, formerly Mini Jev, was inspired by TypeSafe's Jev. It does not reproduce Jev's architecture, training, calibration, speed, or protocol. The separate r-ms/mini-jev project is closely related prior work; LogitTrail is the UPHASH project formerly published under the same name. The paper situates candidate scoring and inspection alongside prior work including Ma et al., Hanna and Feng, LMQL, ChainForge, LLM Comparator, Langfuse, and Phoenix. The current revision also distinguishes LLM2Jev, AnyJev, and AudioJev; their full methods are not benchmarked here. Closely related scoring work includes Zhuang et al., Beyond Yes and No and Wang et al., Single-Token Expected-Value Scoring; LogitTrail does not claim to introduce candidate-logit normalization or expected-value scoring.
AI assistance in implementation, analysis, and writing is disclosed in the manuscript. The author is responsible for its content. The earlier arXiv submission retains the Mini Jev title; no public arXiv identifier has been verified as of 6 October. The current manuscript was submitted to EACL 2027 System Demonstrations on 6 October 2026. The decision is pending; no acceptance is claimed.
The frozen source/evidence ZIP covers the application and the original three studies totaling 26,050 measured requests. The 2,400-call Japanese diagnostic and 4,800-call Japanese/English follow-up each have separate packages linked above, as do the manuscript, video, and framework diagnostics. Framework diagnostics are not general performance or usability benchmarks.
Optional live inference
Keep inference on your Mac.
The browser workbench connects to a resident local model through the LogitTrail API. Inspect each candidate and copy the actual request and response.
Model weights and native binaries are not bundled. Lower-memory machines have not been validated. For model-free inspection and CPU-only replay, use the reviewer route above. This October 2 lightweight check did not build a runtime or download a model.
Reference
Cite the manuscript
This citation identifies the current LogitTrail manuscript. The earlier arXiv submission retains its Mini Jev title and separate frozen files. No public arXiv identifier has been verified as of 6 October 2026. The current manuscript is submitted to EACL 2027 System Demonstrations, with a decision pending.
@misc{oshio2026logittrail,
title = {LogitTrail: An Inspectable Local Interface
for Typed Decisions from Frozen Language Models},
author = {Oshio, Yuki},
year = {2026},
note = {Research manuscript; not peer reviewed},
url = {https://uphash-network.github.io/mini-jev/}
}