#apple + #llm

MLX is an array framework for machine learning on Apple silicon, brought to you by Apple machine learning research.

Some key features of MLX include:

• Familiar APIs: MLX has a Python API that closely follows NumPy. MLX also has fully featured C++, C, and Swift APIs, which closely mirror the Python API. MLX has higher-level packages like mlx.nn and mlx.optimizers with APIs that closely follow PyTorch to simplify building more complex models.

• Composable function transformations: MLX supports composable function transformations for automatic differentiation, automatic vectorization, and computation graph optimization.

• Lazy computation: Computations in MLX are lazy. Arrays are only materialized when needed.

• Dynamic graph construction: Computation graphs in MLX are constructed dynamically. Changing the shapes of function arguments does not trigger slow compilations, and debugging is simple and intuitive.

• Multi-device: Operations can run on any of the supported devices (currently the CPU and the GPU).

• Unified memory: A notable difference from MLX and other frameworks is the unified memory model. Arrays in MLX live in shared memory. Operations on MLX arrays can be performed on any of the supported device types without transferring data.

MLX is designed by machine learning researchers for machine learning researchers. The framework is intended to be user-friendly, but still efficient to train and deploy models. The design of the framework itself is also conceptually simple. We intend to make it easy for researchers to extend and improve MLX with the goal of quickly exploring new ideas.

The design of MLX is inspired by frameworks like NumPy, PyTorch, Jax, and ArrayFire.

Apple’s release notes detail that RDMA integrates with the Thunderbolt framework to enable zero-copy data transfers, meaning data moves directly from one device’s memory to another’s without intermediate buffering. This eliminates bottlenecks associated with TCP/IP protocols, which Thunderbolt previously emulated. Insiders note that while Thunderbolt 5 offers peak speeds, real-world performance depends on factors like cable quality and device compatibility—only M4 and later chips fully support this enhanced mode.

Diving deeper into the technical specifics, Apple’s developer documentation explains that RDMA over Thunderbolt is exposed through new APIs in the macOS networking stack. Developers can initialize clusters using Swift or Objective-C calls that negotiate memory mappings directly over the Thunderbolt bus. This is a departure from traditional Ethernet-based RDMA, which relies on Infiniband or RoCE (RDMA over Converged Ethernet), adapting instead to Thunderbolt’s point-to-point topology.

For those building apps, the update introduces protocols for fault-tolerant clustering. If a device drops out—say, due to a disconnected cable—the system can redistribute workloads dynamically, minimizing disruptions. Testing scenarios outlined in the notes suggest latency as low as microseconds for small transfers, rivaling dedicated high-performance computing setups.

Security is paramount in such a powerful feature. Apple’s notes emphasize built-in encryption for RDMA transfers, preventing unauthorized memory access. A separate 9to5Mac report on the update’s patches reveals fixes for kernel vulnerabilities that could have been exploited in clustered environments, ensuring that the feature doesn’t become a vector for attacks.

Looking at adoption, early sentiment on X suggests enthusiasm among AI researchers. One thread discussed collaborative model training, where multiple users contribute compute power via clustered Macs, democratizing access to high-end AI tools. This could disrupt markets dominated by cloud providers, offering cost savings for startups avoiding subscription fees.