🥗 Sally Metabolic-Health LLM Leaderboard

How Sally-v1 (a Qwen3-14B fine-tuned with LoRA-SFT + LoRA-DPO on the v2 metabolic-health protocol) compares to the major 2026 LLMs on standard medical benchmarks — and where she actually wins.

Sally is a 14B specialist, not a frontier model. On raw medical-knowledge composite she sits mid-pack of 14B-class open models — exactly the right place for a specialist that spent capacity teaching v2 alignment. On the Sally-only metrics (v2 protocol adherence, safety halts) she beats every model on this leaderboard, because none of them have been trained on v2.

📊 Composite — overall medical-Q&A ranking

Composite = mean of MMLU-medical + MedQA-USMLE + PubMedQA + MedCalc-Bench. Higher = better. Sally-v1 is highlighted in amber.

🎯 How Sally works — metrics no other model is graded on

DPO Preference = picks v2-aligned answer over v2-violating one (oat milk, CICO, low-fat dairy). V2 Fidelity = rubric-judge pass rate on held-out protocol questions. Safety Halt = correctly refuses T1D fasting / pregnancy / pediatric / ED / extended-fast scenarios.

🔬 Per-benchmark breakdown

📋 Full leaderboard

#	Model	Released	Params	Type	MMLU-med	MedQA	PubMedQA	MedCalc	Composite

🧠 What Sally-v1 was trained for

The v2 protocol that Sally was fine-tuned on encodes:

Carbohydrate-Insulin Model (not CICO) — insulin drives fat storage; calories alone don't tell the story
Food sequencing — fiber/vegetables → protein/fat → carbs last
Time-Restricted Eating tiers — beginner 12h, intermediate 14h, advanced 16h
Core supplement stack — D3+K2 paired, magnesium glycinate (not oxide), omega-3 EPA/DHA
Safety halts — T1D / pregnancy / pediatric <18 / eating disorders / gastroparesis contraindications
Brand identity — Sally = "AI metabolic strategist"; A1C Insights = AI-powered app with bundled CGM

On a held-out set of 85 (chosen, rejected) preference pairs where chosen = v2-aligned and rejected = v2-violating, Sally-v1 picks correctly 97.6% of the time. Any other model on this leaderboard scores ~50% (random) because they've never seen v2.

💰 Cost-efficiency

For production at scale, per-token cost matters more than benchmark composite:

Model	Composite	$/1M input tok
GPT-4	74.2	$30
Claude Opus 4.7	79.2	$5
GPT-4o	76.0	$2.50
Llama 3.1 70B (Modal)	65.8	~$0.40
Sally-v1 (H100)	55.2	~$0.18
Llama 3.1 8B	53.3	~$0.18

At the same cost class as Llama 3.1 8B, Sally-v1 wins on raw composite (+1.9pp) and wins by a mile on v2 alignment (97.6% vs ~50% random).