Saluki packs Qwen3.8-27B into 7.89GB. Its 88/120 tool score hides lost and gained tasks, parser sensitivity and memory costs beyond the model file.
Local LLMs & Homelabs
30 Articles in this Category
Running open-weights without cloud APIs: VRAM sizing, Ollama, vLLM, GGUF/EXL2 quantization, and Apple Silicon unified memory.
Strata runs 125B Qwen3.8-Flash-Next on 12GB VRAM at 45 tok/s using predictive expert prefetching and io_uring. Here is the PCIe math and setup.
At Open Source Summit Europe 2026, Microsoft Director Ryan Waite warned enterprises against proprietary model capture: do not surrender your organizational learning loops to closed APIs. Here is how to architect infrastructure mobility across open-weight models.
Retailers cut M5 Max Mac Studio prices by $200 as the 512GB M5 Ultra ships at $10,000+. Here is why VRAM residency, not a 26% CPU uplift, dictates the buy.
Microsoft and NVIDIA’s Surface Laptop Ultra pairs an Arm Grace CPU and Blackwell GPU via 300 GB/s NVLink-C2C. Here is what 128GB unified memory and 1 PFLOPS FP4 mean for 120B local agents.
FeSens openTPU runs LFM2.5 at 85.8 tok/s on Kintex-7 FPGA silicon via agent-designed RTL, but off-chip DDR bandwidth caps multi-billion models at 1-4 tok/s.
Janus bundles llama.cpp Vulkan DLLs into a 50MB Go binary for cross-vendor GGUF local LLM inference across AMD, Intel, and Nvidia GPUs without Python or Docker.
Backblaze reports 1.73% annualized failures for Q2 2026. Drive-days, fleet age and sample size explain why that figure is not a home-NAS forecast.
China Telecom’s Xing4.0-29B-A4B activates about 4 billion parameters for each token, but it stores 29 billion parameters…
OrcaSAQ-2 shrinks a 27B coding model to about 12GB. We examine its fidelity results, agent benchmarks, and the practical limits of running it on a 16GB GPU.