The latest AI releases, research, products, and industry updates.
With your permission, we use Google Analytics to understand page visits and whether key workflows succeed. We do not send prompts, files, account identifiers, model identifiers, balances, costs, or raw errors. Privacy Policy
WebLLM is an open-source engine that runs large language model inference directly in web browsers using WebGPU. It supports OpenAI-compatible chat APIs, streaming, JSON mode, workers, service workers, Chrome extensions, browser caching, and custom MLC-format models. Built-in model families include Llama, Phi, Gemma, Mistral, and Qwen.
The author runs Qwen3.6-35B-A3B and Gemma-4-E4B locally on a 48 GB M4 Pro Mac mini through oMLX. Hermes, Tailscale, and several client apps share the same private inference endpoint across a MacBook and iPhone. The setup is intended to cover routine workloads with predictable costs, low latency, offline access, and greater control over sensitive data, while cloud models remain available for more demanding tasks. The article also explains how MoE active parameters, quantization, context memory, and system overhead affect hardware requirements.
Hugging Face has released @huggingface/kernels, a JavaScript library for loading 207 optimized WebGPU kernels from the Hub for browser-based local AI inference. Each kernel includes versioned contracts, WGSL shaders, correctness tests, benchmarks, and documentation. Fleet adds browser-based testing that can contribute private, consented evidence from real GPUs. On an Apple M4, the kernels were 2.57 times faster by geometric mean and 1.90 times faster at the median than ONNX Runtime Web across 809 matching cases.
StartLux-V1.0-27B-Preview ranked second in a Trusted AI MCP benchmark, outperforming several larger models in tool-use and complex task execution. Built on Qwen3.6-27B, it uses targeted post-training and an Auto Research method, and can reportedly run on consumer PCs.