Perplexity Open Sources Lily: A Rust + Metal Inference Engine for Qwen3.6-35B-A3B on Apple Silicon
What changed
Perplexity open sourced Lily, its local inference engine powering Hybrid Compute in Perplexity Computer. Lily is built in Rust and uses custom Apple Metal kernels designed specifically for one large language model, Qwen3.6-35B-A3B, running on Apple Silicon chips like the M5 Max.
Why builders should care
Local inference engines optimized for specific hardware and models cut down costs and latency by avoiding cloud dependence. Lily’s combination of Rust for safety and performance with Metal for GPU acceleration on Apple Silicon demonstrates a practical path for maximizing throughput on Mac hardware. Perplexity reports Lily outperforms MLX-LM on prefill throughput by 1.23 times and on decode throughput by 1.35 times on a 40-core M5 Max with 128 GB RAM. This boosts inference speed while keeping execution local, which can improve data privacy and reduce cloud costs.
The practical takeaway
For teams running large models on Apple Silicon, Lily offers a ready-made, open source solution tuned to a specific 35-billion parameter model. Using Rust with Metal provides a balance of system-level control and GPU compute efficiency suited to Apple’s architecture. This can pressure other inference engines to further optimize for Apple chips and suggests more model-inference pairs will adopt bespoke kernels instead of generic frameworks. Builders focused on macOS or Apple devices gain a performance edge by leveraging Lily without building custom inference stacks from scratch.
What to watch next
The key will be how easily Lily adapts to new models or chip versions beyond Qwen3.6-35B-A3B on M5 Max. Also watch if other vendors adopt custom metal kernels for faster inference on Apple Silicon. Expanded support for more models or broader hardware coverage could make Lily a go-to for local inference in Apple-centric AI deployments. Finally, tracking community contributions to Lily’s open source code will indicate if it becomes a robust, maintainable alternative in the inference landscape.
AI Quick Briefs Editorial Desk