Tech • AI • Robotics • Game
I got one... and it's FAST!!!
8/10TL;DR
Apple’s M5 Ultra showed major gains over the M3 Ultra in local AI workloads, with much faster prompt processing, transcription, image and video generation, and even basic agent-driven video editing.
KEY POINTS
Server-style use case
The M5 Ultra was tested not as a desktop workstation but as a dedicated local AI machine in a server-room setup. The focus was on fully offline workloads, with no calls to OpenAI or Anthropic, covering language models, transcription, visual generation, agent tasks and media analysis.
Key hardware changes
Both the M3 Ultra and M5 Ultra in the comparison used 80 GPU cores, but the newer chip adds a Neural Accelerator on each core and raises memory bandwidth from 819 GB/s to 1.2 TB/s. Those two changes appeared to matter more than raw core count in AI inference tests.
Large-prompt language model gains
Using Qwen3.8 27B 4-bit with a 32,000-token prompt, the M5 Ultra delivered a time to first token of 21.66 seconds, versus 82.85 seconds on the M3 Ultra. That made the newer machine about 3.8 times faster at the initial prompt-processing stage, while generation speed rose to 43.3 tokens per second from 28.1, about 1.5 times faster.
Why the speedup matters
Prompt ingestion stresses compute because the model must process the full context at once, while token generation becomes more constrained by memory movement. In this test, the 27B model occupied roughly 15 GB, and those weights had to be repeatedly moved from unified memory for each generated token. The near-50% increase in bandwidth closely matched the observed 1.5x generation gain.
Smaller models widened the gap
With a 14B model at the same 32K context length, the M5 Ultra cut the wait to 14 seconds, compared with 57 seconds on the M3 Ultra. That translated to roughly 4x faster prompt handling and 1.7x faster generation, suggesting the newer system maintains an advantage across model sizes.
Transcription performance
Local transcription with Whisper models was one of the strongest practical results. On a 13.8-minute recording, the M5 Ultra was 1.8x faster on large-v3 and 2.3x faster on turbo. On a 2-hour 5-minute recording, it completed large-v3 in 89.4 seconds and turbo in 24.4 seconds, with the smallest model finishing in about 12 seconds.
Video understanding and archival potential
Running Qwen3-VL 32B 4-bit MLX on video at roughly one frame per second, the M5 Ultra analyzed footage and generated detailed scene descriptions in about 22 seconds, about 2.6x faster than the M3 Ultra. That result points to a practical use case for indexing large personal archives such as old VHS home videos without paying per-minute API fees.
Image and video generation
Local image generation with FLUX.2 klein 4B completed in 2.6 seconds on the M5 Ultra versus 10.2 seconds on the M3 Ultra, a roughly 4x gain. Video generation with LTX-2.5 22B also improved, with the newer machine running about 1.7x faster on a small clip and 2.5x faster on a longer one.
Agent workloads and long context
The system was also used as a backend for Hermes with Qwen and a context window of about 131,000 tokens. While not positioned as the best primary agent model, the local setup successfully retrieved context and handled basic personal-knowledge queries, indicating that large-context local assistants are becoming more viable on high-memory Apple hardware.
Binary decision models and editing tools
In a spam-classification test using the open-source Laya approach on 300 labeled messages, both systems were accurate, but the M5 Ultra was about 1.4x faster. A separate test connected Qwen3.8 27B through LM Studio and DaVinci Resolve’s MCP to perform automated editing actions locally; the process was slow, taking roughly 13 minutes, but it completed a working edit.
Cost and scaling outlook
The tested configuration, with 256 GB of unified memory and 8 TB of storage, was priced at about $14,000. A 512 GB version is expected later, and the machine was framed less as a general-purpose luxury desktop than as a self-owned AI node that could eventually be clustered for larger local models.
CONCLUSION
The M5 Ultra appears to turn high-end Apple silicon into a serious local AI platform, especially for users prioritizing offline inference, transcription and media workflows. Its biggest advantages came from faster prompt handling and memory bandwidth, though the price keeps it firmly in specialized territory.
Ask a question