Optical Interconnects Poised to Revolutionize AI Scale-Up Networking

The distinction between scale-out and scale-up networking in data centers is blurring, with optical communication potentially moving beyond long-distance connections to address the critical latency needs within single mega-computers or racks. While scale-out networks have long favored optics for connecting thousands of AI computers, scale-up networks have relied on dense copper interconnects like Nvidia's NVLink for intra-rack communication. However, escalating AI demands are pushing electrical links towards their physical limits, characterized by increased attenuation, power consumption, and heat as bandwidth requirements surge towards terabits per second. This "copper wall" necessitates shorter, thicker cables, complicating rack routing and limiting the number of processors that can be densely packed. Nvidia's NVLink Fusion program, introduced in 2025, signals a potential shift, allowing hyperscalers to build custom AI systems around Nvidia's fabric and now includes photonics partners like Ayar Labs, Marvell Technologies, and Lightmatter. Companies like Ayar Labs are developing optical chiplets designed to sit alongside processors, converting electrical signals to light mere inches from the compute silicon, leveraging advances in hybrid bonding for integration. Lightmatter offers an alternative with a photonic interposer acting as the substrate, enabling processors to be stacked directly on a silicon photonics engine, with ongoing work focused on integrating lasers directly onto silicon for denser fabrics. While Nvidia is adopting a measured approach, integrating optics gradually as the technology matures, the broader industry anticipates high-volume optical scale-up systems by 2028, potentially enabling multi-rack systems to function as single computing domains.