When a chip shortage is discussed, the conversation usually turns to lithography and wafer capacity. For the accelerators used in artificial intelligence, the tighter constraint sits at the end of the line, in the steps that assemble finished dies into a single package.
Advanced packaging is the process of placing multiple chiplets, memory stacks and interposers into one unit and connecting them with very short, very dense wiring. It is what allows a processor to sit beside high bandwidth memory and behave as though the two were a single device. Without it, the memory bandwidth described in the previous generation of designs is simply unavailable.
Why packaging capacity is hard to expand
A leading edge fabrication plant costs billions and takes years. A packaging line is cheaper in absolute terms but shares the same problems: specialised equipment, scarce engineers and long qualification cycles. Customers must validate that a package survives thermal cycling and mechanical stress before it can ship in volume.
Yields compound the difficulty. If several dies are combined and each has a small chance of being faulty, the probability that a finished package works falls quickly. Manufacturers mitigate this by testing dies before assembly and by building redundancy into the design, but the economics remain unforgiving.
The interposer and the substrate
Two components receive less attention than they deserve. The interposer is the silicon bridge that carries signals between chiplets, and the substrate is the board that connects the package to the system. Both are specialised, both are capacity constrained, and both have been cited in industry reporting as reasons for delivery delays.
Substrates in particular have long lead times. They are made with processes closer to printed circuit board manufacturing than to semiconductor fabrication, and the suppliers able to produce the finest features are few. Expanding that supply requires investment in a part of the industry that historically attracted less attention.
Thermal density and the limits of air cooling
Packing more power into a smaller area concentrates heat. Air cooling can remove a certain amount per square centimetre, and current packages are close to that limit. Liquid cooling, whether direct to chip or immersion, removes more but requires changes to the facility.
This is where hardware design meets data centre planning. A chip that performs well in a laboratory may be impractical in an existing building, and the mismatch is one reason new facilities are being designed around thermal budgets rather than retrofitted.
Who controls the supply chain
The packaging ecosystem is concentrated. A small number of firms handle the most demanding work, and their capacity is booked well in advance. Governments have noticed, and programmes intended to onshore semiconductor manufacturing increasingly include packaging as a component rather than an afterthought.
Standards work matters here too. Bodies such as JEDEC define the interfaces that let components from different suppliers work together, and the pace of that work affects how quickly new combinations can reach production.
What this means for buyers and planners
Lead times for accelerators are shaped as much by packaging as by wafer supply, and forecasts that assume otherwise will be wrong. Buyers should ask suppliers about package level constraints and about the substrate supply behind them, because those answers predict delivery more reliably than wafer start figures.
For researchers, the implication is that architectural ideas which ignore packaging cost are unlikely to ship. A design that requires an exotic interposer or an unusual substrate will face a long path to volume, however elegant the benchmark results.
Testing, reliability and the cost of a bad package
A package that passes initial qualification can still fail in the field, and a failure in a deployed accelerator is expensive for the operator and reputationally costly for the supplier. Testing therefore extends beyond electrical function to thermal cycling, mechanical shock and long duration burn-in.
These tests take time and consume capacity, which competes with production. A manufacturer under pressure to ship has an incentive to shorten qualification, and customers have responded by demanding detailed reliability data before committing to volume orders.
For operators, the practical consequence is that spare parts and replacements are constrained by the same bottleneck. Planning for failure rates without accounting for packaging lead times produces an optimistic maintenance budget and an unwelcome surprise in the first year of operation.
The next few years
Investment is flowing into packaging capacity, and new lines are being qualified. The gap between demand and supply is likely to narrow rather than close, because demand for AI accelerators continues to grow at the same time.
Watch the qualification announcements rather than the investment headlines. Funding a line is different from qualifying it, and the industry’s recent history is full of capacity that arrived later than the schedule promised. The same discipline applies to forecasts: a number attached to a future quarter should be treated as an intention until a supplier confirms it in writing.
Image: Mister rf · CC BY-SA 4.0 · via Wikimedia Commons.