Back in 2022, before ChatGPT totally revolutionized the world of artificial intelligence, Etched decided to invest heavily in transformers.
With this focus, the startup has developed Sohu, a specialized ASIC chip designed exclusively for transformer models, the architecture powering the likes of ChatGPT, Sora, and Gemini.
Sohu is a one-trick pony: it can’t run machine learning models like Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), or Long Short-Term Memory Networks (LSTMs), but for transformers, Etched says it’s unparalleled and can surpass Nvidia’s flagship B200 GPU in speed by nearly tenfold.
It’s about scalability
Because Sohu is designed exclusively for transformer models, it can avoid the complex and often unnecessary control flow logic that general-purpose GPUs need to handle to support a wide variety of applications.
By focusing only on the computational needs of transformers, Sohu is able to dedicate more of its resources to performing mathematical operations, which are the core tasks in transformer processing.
This streamlined approach allows Sohu to achieve over 90% utilization of its FLOPS capacity, significantly higher than the approximately 30% utilization seen in general-purpose GPUs. This means that Sohu can perform more calculations in a given period, making it much more efficient for transformer-based tasks.
The world has seen a dramatic surge in the use of transformer models, and every major AI lab – from Google to Microsoft – is committed to scaling this technology further. With performance reaching over 500,000 tokens per second in Llama 70B throughput, Sohu is an order of magnitude faster and more cost-effective than next generation GPUs.
Etched believes that the shift towards specialized chips is inevitable and aims to stay ahead of the curve. “The current and next-generation state-of-the-art models are transformers,” the company says. “Tomorrow’s hardware stack will be optimized for transformers. Nvidia’s GB200s have special support for transformers (TransformerEngine). ASICs like Sohu entering the market mark the point of no return.”
Etched reports Sohu production is ramping up, with substantial orders already in place. “We believe in the hardware lottery: the models that win are the ones that can run the fastest and cheapest on hardware. Transformers are powerful, useful, and profitable enough to dominate every major AI compute market before alternatives are ready.”
The company adds, “Transformer killers will need to run on GPUs faster than transformers run on Sohu. If that happens, we’ll build an ASIC for that too!”
