Forward-looking: We’re approaching a point where traditional copper interconnections won’t be able to carry enough data to keep GPUs and other specialized chips fully utilized. The AI market is urgently demanding a next-generation solution to this interconnection bottleneck, and Broadcom appears to be working on an optics-based solution that is closer to the chip itself.
Broadcom is developing new silicon photonics technology aimed at significantly increasing the bandwidth available to GPUs and other AI accelerators. By utilizing co-packaged optics (CPOs), the fabless chip manufacturer aims to integrate optical connectivity components directly into GPUs, enabling higher data rates while simultaneously reducing power requirements.
The company has been working on CPO solutions for several years and showcased its latest advancements at the recent Hot Chips convention. Broadcom’s “optical engine” reportedly delivers a total interconnect bandwidth of 1.6 TB/sec, equivalent to 6.4 Tbit/sec or 800 GB/sec in each direction.
This new connection can provide “error-free” data transfer to a single chiplet, achieving performance levels comparable to Nvidia’s NVLink and other specialized data center solutions. However, Broadcom has not yet incorporated its optical interconnections into a market-ready GPU, such as the A100 or MI250X. Instead, it used a test chip designed to emulate a real GPU for demonstration purposes.
According to Manish Mehta, Broadcom’s vice president of the optical systems division, copper connections start to degrade after just five meters. While optical communications have long been viewed as the solution to this signal degradation issue, they traditionally require much more power than copper-based technologies.
For example, Nvidia estimates that an optics-powered NVL72 system would require an additional 20 kilowatts per rack, on top of the 120 kilowatts the system already consumes.
Broadcom has managed to reduce power consumption with the use of co-packaged optics, which places individual transceivers in direct contact with the GPU. The company utilized TSMC’s chip-on-wafer-on-substrate (CoWoS) packaging technology to bond a pair of high-bandwidth memory stacks to the compute die. The logic and memory components of the chip sit on a silicon interposer, while Broadcom’s optical engine is located on the substrate.
Mehta explained that CPO technology could connect up to 512 individual GPUs across eight racks, allowing the entire setup to function as a single system. In comparison, Nvidia’s NVL72 can achieve similar unified computing capabilities with “just” 72 GPUs, suggesting that Broadcom’s solution could eventually offer a competitive advantage for next-generation AI workloads.
