Tech.us Tokle-3M tops Open SLM leaderboard for sub-3M models

Tech.us released Tokle-3M, a 2.91-million-parameter open small language model that ranks first among sub-3M models on the Open SLM Leaderboard as of Oct. 1, 2026. The result highlights a training method called Static Pairwise Attention Bias, which the company says helps compact models outperform size-based expectations without adding inference-time parameters.

Why it matters: - Tokle-3M shows that a compact model can beat size-based expectations without adding inference-time complexity. - The result matters for teams that want lower memory use, faster experiments and easier deployment on controlled infrastructure. - Tech.us is using the model to test whether training methods can narrow the gap between small models and larger systems for targeted work.

What happened: - Tech.us released Tokle-3M, an open small language model with 2.91 million parameters. - The model ranks first among models under 3 million parameters on the Open SLM Leaderboard. - As of Oct. 1, 2026, Tokle-3M posts an Intelligence Index score of 8.91. - Tokle-3M also posts the top arithmetic score in its class at 40.80% on ArithMark-3. - The release was announced Oct. 9, 2026.

The details: - Tokle-3M scores 1.80 standard deviations above the Open SLM Leaderboard trend line for performance versus model size. - Tech.us says that is the strongest result in the category. - The company attributes the result to Static Pairwise Attention Bias, or SPAB. - SPAB is a training approach that guides the model during learning and is removed before release. - The released model runs with 2.91 million parameters and does not require the extra SPAB table at inference. - Tokle-3M was evaluated on HellaSwag, ARC-Easy, ARC-Challenge, PIQA and ArithMark-3 using zero-shot normalized accuracy. - The Open SLM Leaderboard’s Intelligence Index combines chance-adjusted results across reasoning, commonsense and arithmetic benchmarks. - Tech.us published full training and evaluation details on its research site. - The model was pre-trained on English datasets with a 512-token context window. - Because Tokle-3M uses rotary position embeddings, the window can be extended at inference without changing the architecture. - Performance beyond 512 tokens has not been evaluated. - Tokle-3M is not instruction-tuned or safety-aligned. - Tech.us is positioning the release as a research model, not a production assistant. - Tokle-3M is available now on Hugging Face with a model card covering training, evaluation and limitations.

Between the lines: - The research is aimed at models in a comparable size range, not at replacing large general-purpose language models. - The result reinforces a broader industry question: where model size matters and where better training can do more of the work. - Small models can be practical for narrow tasks such as text classification, intent detection, device-command interpretation and system-log triage. - Offline, embedded and on-device deployments may benefit most when data must stay on local hardware. - Tech.us wants smaller models that organizations can run on infrastructure they control. - Self-hosted deployment can reduce the need to move sensitive data outside existing environments. - The company sees Tokle-3M as an early technical result showing that training method and compact architecture can materially affect performance.

What's next: - Tech.us plans to keep exploring how focused training, compact architecture and deployment control can make AI more practical for specific business environments. - The company is studying whether techniques from Tokle can inform more focused AI systems for organizations that need tighter control over training, deployment and adaptation. - Researchers and engineers can use the open weights to reproduce the results and build on them.