Meta is developing a system with a Korean startup that connects 960 AI chips

Meta is developing a new architecture for AI data centers with the small Korean startup Panmnesia. The project uses a high-speed interconnect standard called CXL. This standard connects processors, AI chips and memory in different server racks without a conventional network connection. The system can connect about 960 accelerators in the same computing domain. This allows all the hardware to operate as a unified system that uses shared resources.

When large AI models are trained, hundreds of devices perform different stages of the same computation. When one of them falls behind, the others must wait as well. With Ethernet and InfiniBand connections operating between server racks, data packet processing and software coordination can create variable latency. CXL is expected to make data transfers more stable by combining memory and computing power in a shared environment.

Panmnesia has developed a switch with multiple connection ports, a unit that accelerates data transfers and a management controller for this system. The hardware is divided into subsystems and groups and operates within a unified interconnect architecture. The company has tested the controller and acceleration unit on actual chips. The switch has also been manufactured, and initial samples are being supplied for the development of commercial products.

For comparison, in the NVIDIA GB200 NVL72 system, 1 processor directly controls 2 accelerators through an NVLink-C2C connection. In the new architecture, 1 processor can connect 16 accelerators. That is 8 times the figure for the NVIDIA system. When about 60 such groups are connected, 960 accelerators can operate in a single domain. Latency in data transfers between server racks could fall from microseconds to several hundred nanoseconds. This would reduce latency by about 10 times.

The new architecture could also allow failed hardware to be replaced without shutting down the entire server. However, at 128 GT/s, the electrical-signal-based CXL connection reaches a distance of about 7 meters with 2 signal repeaters. Panmnesia has developed an optical CXL connection that uses light signals for longer distances and has tested an initial hardware prototype. It has not yet been announced when the project will be deployed in actual data centers.