#dpo

A 1.32B-parameter Mixture-of-Experts LLM trained from scratch in PyTorch - custom tokenizer, MLA attention, 18B-token pretraining, SFT and DPO. No pretrained weights, no HuggingFace Trainer.

How often do you correct your coding agent, and on what kind of work? Measures it from your Claude Code transcripts, drafts CLAUDE.md rules from your recurring corrections, and exports them as preference pairs.

Local CLI to turn failed AI agent traces (wrong tool, bad args, over-refusals) from LangSmith/Langfuse into clean SFT/DPO fine-tuning data

A controller that learns how a tool-using agent should spend compute at each step (WIDER, DEEPER, DECOMPOSE, STOP, or ESCALATE to a stronger model) and improves its own policy from logged traces. Judged on cost per solved task with paired statistics; ships trained policies and an offline embed API (pip install, no key needed).

Reproducible QLoRA SFT and DPO experiments for open-weight LLM reasoning, with locked selection, matched controls, and audit-grade provenance.