The Verge has launched a new series documenting the experience of running local artificial intelligence models directly on Apple hardware, specifically the Mac Studio. The project aims to determine if the capabilities of on-device AI are compelling enough to justify the significant expense of high-end computers and expensive RAM.
The author acknowledges that while using cloud-based AI services raises privacy concerns regarding personal data, the growing viability of local models offers an alternative. Despite not being an AI expert, the writer intends to share the process of testing models such as Hermes and Qwen, hoping to guide readers through the excitement, overwhelm, and frustration associated with the learning curve.
This initiative highlights Apple’s marketing push regarding the performance of its new desktops for AI workloads, providing a real-world test of those claims for everyday users.
The learning curve is brutal. Hope the article covers quantization tricks that actually work without burning all memory.
I bought one for this exact reason but still can’t justify the RAM costs. Is it really worth it yet?
Finally, someone testing this honestly. My Mac Studio feels slow running Qwen compared to cloud APIs.