CLM：Qwen3-8B开源替代Jev，延迟降低4-13倍

Original post: https://www.reddit.com/r/LocalLLaMA/comments/1woscea/contrastive_language_models/

(sorry I felt it wasn't giving CLM the highlight it deserves)

What it is: a new projection head for Qwen3-8B.

At the API and functional interface level, CLM supports everything Jev does—it is not a subset. However, there are important trade-offs in generalization, context scale, and architecture between the two.

1. Functional Parity (Same Primitives)

CLM was specifically engineered as an open-weights, self-hostable alternative to TypeSafe AI's Jev. It implements the exact same "System One" decision interface and supports all three of Jev’s core question primitives:

• Choice: Evaluates a discrete set of candidates and returns a categorical probability distribution.
• Noul: Outputs a calibrated true/false probability for a proposition or guardrail check.
• Score: Scores an input against an ordered rubric or scale.

Code written for the TypeSafe Jev client can be pointed directly at a clm-serve endpoint with drop-in compatibility (from clm import CLMClient, Choice, Noul, Score).

2. Where CLM Outperforms Jev

• Latency and Disaggregated Caching: Jev is a proprietary cloud model that evaluates state and question choices jointly. CLM separates the state head from the action head. If an agent has a persistent set of tools or actions, CLM embeds those actions once and caches them. In benchmarks like interactive browser agents and gaming (T-Rex, Super Mario), CLM is 4× to 13× faster than Jev.
• Open Weights & Fine-Tunability: Jev is a closed API with no user fine-tuning (you can only prompt it via state and question instructions). Because CLM’s heads are tiny open weights (~75 MB), you can fine-tune them on your own agent trajectories.
• Coding Benchmark Verifiers: When fine-tuned on agent trajectories, CLM achieves state-of-the-art verifier performance on Terminal-Bench 2.1 (87.6%) and DeepSWE (81.6%), whereas zero-shot Jev struggled on those exact benchmarks (scoring ~71% on DeepSWE).

3. Where Jev Still Has the Edge (CLM-8B Limitations)

While CLM covers the entire feature surface of Jev, the current CLM-v0.1-8B release trails Jev in a few areas:

• Zero-Shot Broad Knowledge: Jev is backed by a larger, proprietary model On zero-shot open-domain tasks, Jev still holds an edge in edge-case accuracy (e.g., Berkeley Function Calling Leaderboard v4: Jev scored 99.2% vs. CLM-8B’s 95.2%; WikiRacing: Jev 30/30 vs. CLM-8B 26/30).
• Context Budget: Jev accepts requests up to a 64K token context out-of-the-box. CLM-8B was tested and calibrated at 2K to 8K context. While its Qwen3 backbone can accept longer prompts, representations past 8K haven't been calibrated for the reference head.
• Probability Normalization: CLM calculates probabilities via dot products and softmax over the candidates passed in that request Its probabilities are inherently relative to the candidate set provided, whereas Jev’s scoring is calibrated internally against absolute criteria.

Summary

If you are asking if you will lose API features by using CLM instead of Jev: No, you get the full primitive set (Choice, Noul, Score) with massive latency gains and zero API costs. You only sacrifice some zero-shot generalization on niche out-of-domain tasks compared to TypeSafe's hosted service.