#quantization

Quantization, sparsity, and behavioral robustness of Qwen LLMs under edge hardware constraints

The 48GB VRAM Local LLM Guide: real benchmarks, model configs, and gotchas from 135+ community reports. Includes runnable benchmark scripts.

An LLM inference engine written from scratch in Python and C++/Metal — tokenizer, forward pass, KV cache, and INT4 quantization implemented by hand. No PyTorch, no llama.cpp.

An LLM inference engine written from scratch in Python and C++/Metal — tokenizer, forward pass, KV cache, and INT4 quantization implemented by hand. No PyTorch, no llama.cpp.

Open-source platform for creating, distributing and running sovereign EU-compliant LLMs. Verticalize any model for your domain, language and brand. AI Act ready.

🥖 Offline sourdough bread baking assistant powered by an on-device micro-LLM on ESP32-S3 with OpenAI-compatible WiFi API

JavaScript/TypeScript SDK for VeloxQuant-MLX: hardware-aware memory optimization and OpenAI-compatible local inference on Apple Silicon.

Quantization calibration is a language choice. Measuring what English-calibrated GPTQ costs Arabic LLMs — and fixing it.

One command runs the best local LLM your GPU can handle: model picked from measured multilingual benchmarks, llama.cpp tuned for AMD, NVIDIA, Intel and Apple GPUs.