With all of the latest speak about AI, knowledge facilities, and the cloud, we expect it’s useful to keep in mind that processors like GPUs and CPUs solely make about 20% of the upfront value of a server. These are inclined to get essentially the most focus as the selection of processor must be made first, and it’s this determination that drives the choices for all the things else within the server, however they’re solely a chunk of the full value.
Reminiscence is one other 20%. However nonetheless effectively over half the price of a server comes from rather more prosaic merchandise – printed circuit boards (PCBs), passive elements, cables, energy provides, onerous drives and the racks that maintain them – we must always add networking, which may be greater than all the remainder of the elements, however we are going to save that for one more time – who sells all that gear and the place does the worth accrue?
Editor’s Word:
Visitor writer Jonathan Goldberg is the founding father of D2D Advisory, a multi-functional consulting agency. Jonathan has developed progress methods and alliances for firms within the cellular, networking, gaming, and software program industries.
There are two varieties of distributors right here – OEMs and ODMs – we aren’t going to spell out the acronyms as a result of that truly confuses the image. Usually talking – the OEMs personal the manufacturers and the end-customer relationships. The ODMs present sourcing and manufacturing – the bodily manufacturing and meeting of all of the gear. In between the 2, there may be appreciable overlap in areas of design and methods integration. You will need to needless to say the boundaries between OEMs and ODMs is fuzzy, with important forwards and backwards in lots of areas.
A little bit of historical past. This mannequin got here to life throughout the 1990’s as PC makers moved manufacturing from the US to Asia. The PC manufacturers, the OEMs, outsourced to contract producers largely based mostly in Taiwan. These firms manufactured gear in Taiwan and later shifted closely to China. Over time, the contract producers moved up the worth chain, including design capabilities. The contract producers grew to become ODMs after which lots of them spun off separate firms to promote their very own branded merchandise changing into OEMs in their very own proper. This mannequin then percolated into how most high-volume electronics are produced in the present day.
Servers moved at a barely completely different tempo. These provided decrease quantity and better costs, so the OEMs, the model homeowners, held onto design capabilities (and generally manufacturing) for for much longer. For a few years, the OEMs labored with Intel to design a variety of servers. They then offered these to prospects. Whereas they provided varied configurations, these have been largely catalog methods – prospects picked from the choices out there.
The cloud modified all of this.
Most critically, the general public cloud suppliers (a.okay.a. the hyperscalers) got here to dominate the market, not solely concentrating financial energy but additionally technical competence. Over time, the hyperscalers largely lower out the OEMs, working straight with the ODMs to supply the methods they designed themselves.
At this time, the OEM panorama largely consists of HP, Dell and Lenovo. There are lots of of ODMs, however the largest are all based mostly in Taiwan and embrace Compal, Foxconn, Inventec, Quanta and Wistron. The businesses are all very various with dozens of subsidiaries unfold out throughout the availability chain. There are additionally a handful of different ODMs which are inclined to focus on particular niches, such because the meme inventory of the second SuperMicro with their specialty in GPU servers.
How does this work in follow? At this time there’s a divide between the hyperscalers and primarily everybody else. Think about a big company – a financial institution, a quick meals chain, or an automaker – they might nonetheless wish to personal their very own servers, and even knowledge facilities. They’ll work with the OEMs, who will supply them a catalog of methods to select from. The OEMs will then sometimes act because the methods integrator – working with all of the distributors to supply components, assemble the PCBs, then wire all the things to collectively and set up the software program. The OEMs play an essential position right here as they’re those making a whole lot of the acquisition choices.
Against this, the hyperscalers function dozens of information facilities. Their enterprise is predicated on huge economies of scale and if they’ll shave off 5% of the price of a server that results in lots of of tens of millions of {dollars} in financial savings. On high of that they’ve concentrated technical expertise. Put merely, they’ll afford to rent groups to design servers optimized for his or her particular wants. Different massive companies would not have these groups, nor do they actually need them, they’re simply not working on the identical scale. The hyperscalers then go on to the ODMs who acquire all of the elements, assemble the methods and wire them up. Right here, it’s the finish buyer who’s making buy choices for nearly all the elements.
This presents a giant drawback for all of the part distributors. Think about a chip vendor. They should persuade a buyer to purchase their chip, however the buyer doesn’t need a chip, they need a whole working system. Earlier than they comply with any massive orders, the shopper will wish to check out that system and ensure it runs their software program effectively. So the chip vendor has to work with an OEM or ODM to design that system. And these designs prices cash. It takes a workforce of 5-10 folks a month or two to put all the things out, confirm efficiency, and guarantee firmware and software program compatibility. Then somebody has to purchase elements to construct a couple of prototypes.
These prices add up rapidly, simply a couple of hundred thousand per system and sometimes into seven figures. So earlier than the chip vendor can promote a single chip, they’ve to speculate materials quantities. Prospects all need servers which are as near their wants as doable, this implies somebody has to supply a number of variations of the server, and so the prices can balloon. All earlier than anybody is aware of how effectively the platform promote.
This drawback has gotten worse. When it was solely Intel and AMD promoting server CPUs, the availability chain had a constrained determination area, with effectively established suppliers. Now that there are a dozen CPU designers the combinatorics are rather more daunting. Anybody seeking to enter the marketplace for AI accelerators has to cope with all these prices. And for smaller distributors, they need to be very cautious how they place their bets.
Put money into assist for a scorching chip and the rewards may be immense, however put money into the flawed platforms and the returns are large losses. The issue is much more acute in the case of promoting to the hyperscalers. They need so much various prototypes. They’ve rigorous testing cycles which transfer from a dozen methods, to 100 to a couple thousand. They could pay for these (or not), however any firm designing a chip wants much more quantity than that to justify the prices of the check methods not to mention the price of the complete chip.
After all, there are all kinds of initiatives to standardize a lot of this. The Open Compute Venture’s core mission is to standardize the design of servers. And whereas OCP has made some main contributions to the business, we don’t assume anybody would describe it as a typical commonplace. All of that is going to get extra advanced.
The rising diversification of information facilities, from CPU-only, to heterogeneous compute is forcing all of the distributors – not simply the chip designers – to beginning taking over some heavy threat. Many will chase each deal, others will most likely fall again on outdated habits specializing in AMD and Intel and now Nvidia. The good ones will take a portfolio strategy to their enterprise and monitor their decisions in ways in which resemble hedge fund managers or enterprise buyers. We don’t intend to be alarmist, a lot of this can be a pure a part of electronics cyclicality. Over time, the business will discover some new equilibrium, however the subsequent few years are going to be rather more chaotic.
