#kv-cache

Agentic Runtime

Security-First Agentic IDE Coding Harness

Demonstrating an attribution gap in LLM inference telemetry: per-request tracing records the symptom of KV-cache contention and not the cause

后悔承诺门 · 试验台（探针项目，非产品）：Transformer 缺一条「写通道」。第一步已实现——零后训练 / 零厂商协作 / 不碰 KV 的 harness，测四项外部强制机制（参数级风险分级、尾部强制自审、独立审核 Agent、射程锁上下文编辑）及 A–H 组合的离线效果；第二步已规划——协同进化：KV 局部编辑、推理轨迹纠错、后训练。每一步都给出可操作结论，包括证伪。

Slot-pinning, cache-aware proxy and dashboard for local LLM servers (llama.cpp, Gufo)

From-scratch LLM inference engine — continuous batching, paged KV cache, prefix caching, hand-written Triton paged attention, CUDA graphs, OpenAI-compatible API. Benchmarked against vLLM on an A100.

JavaScript/TypeScript SDK for VeloxQuant-MLX: hardware-aware memory optimization and OpenAI-compatible local inference on Apple Silicon.