In 1997 the IBM Deep Blue supercomputer defeated world chess champion Garry Kasparov. It was a groundbreaking demonstration of supercomputer know-how and a primary glimpse into how high-performance computing would possibly at some point overtake human–stage intelligence. Within the 10 years that adopted, we started to make use of synthetic intelligence for a lot of sensible duties, resembling facial recognition, language translation, and recommending films and merchandise.
Quick-forward one other decade and a half and synthetic intelligence has superior to the purpose the place it may possibly “synthesize data.” Generative AI, resembling ChatGPT and Steady Diffusion, can compose poems, create art work, diagnose illness, write abstract studies and pc code, and even design built-in circuits that rival these made by people.
Super alternatives lie forward for synthetic intelligence to change into a digital assistant to all human endeavors. ChatGPT is an effective instance of how AI has democratized the usage of high-performance computing, offering advantages to each particular person in society.
All these marvelous AI functions have been resulting from three components: improvements in environment friendly machine-learning algorithms, the provision of huge quantities of information on which to coach neural networks, and progress in energy-efficient computing by way of the development of semiconductor know-how. This final contribution to the generative AI revolution has acquired lower than its fair proportion of credit score, regardless of its ubiquity.
During the last three many years, the most important milestones in AI have been all enabled by the modern semiconductor know-how of the time and would have been not possible with out it. Deep Blue was applied with a mixture of 0.6- and 0.35-micrometer-node chip-manufacturing know-how. The deep neural community that received the ImageNet competitors, kicking off the present period of machine studying, was applied with 40-nanometer know-how. AlphaGo conquered the sport of Go utilizing 28-nm know-how, and the preliminary model of ChatGPT was skilled on computer systems constructed with 5-nm know-how. The newest incarnation of ChatGPT is powered by servers utilizing much more superior 4-nm know-how. Every layer of the pc techniques concerned, from software program and algorithms right down to the structure, circuit design, and system know-how, acts as a multiplier for the efficiency of AI. Nevertheless it’s truthful to say that the foundational transistor-device know-how is what has enabled the development of the layers above.
If the AI revolution is to proceed at its present tempo, it’s going to wish much more from the semiconductor trade. Inside a decade, it can want a 1-trillion-transistor GPU—that’s, a GPU with 10 instances as many gadgets as is typical in the present day.
Advances in semiconductor know-how [top line]—together with new supplies, advances in lithography, new sorts of transistors, and superior packaging—have pushed the event of extra succesful AI techniques [bottom line]
Relentless Progress in AI Mannequin Sizes
The computation and reminiscence entry required for AI coaching have elevated by orders of magnitude previously 5 years. Coaching GPT-3, for instance, requires the equal of greater than 5 billion billion operations per second of computation for a complete day (that’s 5,000 petaflops-days), and three trillion bytes (3 terabytes) of reminiscence capability.
Each the computing energy and the reminiscence entry wanted for brand spanking new generative AI functions proceed to develop quickly. We now must reply a urgent query: How can semiconductor know-how preserve tempo?
From Built-in Gadgets to Built-in Chiplets
Because the invention of the built-in circuit, semiconductor know-how has been about cutting down in characteristic measurement in order that we will cram extra transistors right into a thumbnail-size chip. As we speak, integration has risen one stage greater; we’re going past 2D scaling into 3D system integration. We are actually placing collectively many chips right into a tightly built-in, massively interconnected system. It is a paradigm shift in semiconductor-technology integration.
Within the period of AI, the potential of a system is instantly proportional to the variety of transistors built-in into that system. One of many important limitations is that lithographic chipmaking instruments have been designed to make ICs of not more than about 800 sq. millimeters, what’s referred to as the reticle restrict. However we will now prolong the scale of the built-in system past lithography’s reticle restrict. By attaching a number of chips onto a bigger interposer—a bit of silicon into which interconnects are constructed—we will combine a system that accommodates a a lot bigger variety of gadgets than what is feasible on a single chip. For instance, TSMC’s chip-on-wafer-on-substrate (CoWoS) know-how can accommodate as much as six reticle fields’ price of compute chips, together with a dozen high-bandwidth-memory (HBM) chips.
HBMs are an instance of the opposite key semiconductor know-how that’s more and more vital for AI: the power to combine techniques by stacking chips atop each other, what we at TSMC name system-on-integrated-chips (SoIC). An HBM consists of a stack of vertically interconnected chips of DRAM atop a management logic IC. It makes use of vertical interconnects referred to as through-silicon-vias (TSVs) to get indicators by way of every chip and solder bumps to kind the connections between the reminiscence chips. As we speak, high-performance GPUs use HBMextensively.
Going ahead, 3D SoIC know-how can present a “bumpless different” to the standard HBM know-how of in the present day, delivering far denser vertical interconnection between the stacked chips. Latest advances have proven HBM take a look at buildings with 12 layers of chips stacked utilizing hybrid bonding, a copper-to-copper reference to a better density than solder bumps can present. Bonded at low temperature on high of a bigger base logic chip, this reminiscence system has a complete thickness of simply 600 µm.
With a high-performance computing system composed of a lot of dies working giant AI fashions, high-speed wired communication might rapidly restrict the computation pace. As we speak, optical interconnects are already getting used to attach server racks in knowledge facilities. We’ll quickly want optical interfaces based mostly on silicon photonics which are packaged along with GPUs and CPUs. It will enable the scaling up of energy- and area-efficient bandwidths for direct, optical GPU-to-GPU communication, such that tons of of servers can behave as a single large GPU with a unified reminiscence. Due to the demand from AI functions, silicon photonics will change into one of many semiconductor trade’s most vital enabling applied sciences.
Towards a Trillion Transistor GPU
As famous already, typical GPU chips used for AI coaching have already reached the reticle discipline restrict. And their transistor depend is about 100 billion gadgets. The continuation of the development of accelerating transistor depend would require a number of chips, interconnected with 2.5D or 3D integration, to carry out the computation. The combination of a number of chips, both by CoWoS or SoIC and associated superior packaging applied sciences, permits for a a lot bigger complete transistor depend per system than might be squeezed right into a single chip. We forecast that inside a decade a multichiplet GPU could have greater than 1 trillion transistors.
We’ll must hyperlink all these chiplets collectively in a 3D stack, however thankfully, trade has been in a position to quickly scale down the pitch of vertical interconnects, rising the density of connections. And there’s loads of room for extra. We see no purpose why the interconnect density can’t develop by an order of magnitude, and even past.
Power-Environment friendly Efficiency Development for GPUs
So, how do all these revolutionary {hardware} applied sciences contribute to the efficiency of a system?
We will see the development already in server GPUs if we have a look at the regular enchancment in a metric referred to as energy-efficient efficiency. EEP is a mixed measure of the power effectivity and pace of a system. Over the previous 15 years, the semiconductor trade has elevated energy-efficient efficiency about threefold each two years. We imagine this development will proceed at historic charges. It will likely be pushed by improvements from many sources, together with new supplies, system and integration know-how, excessive ultraviolet (EUV) lithography, circuit design, system structure design, and the co-optimization of all these know-how parts, amongst different issues.
Particularly, the EEP enhance shall be enabled by the superior packaging applied sciences we’ve been discussing right here. Moreover, ideas resembling system-technology co-optimization (STCO), the place the completely different practical components of a GPU are separated onto their very own chiplets and constructed utilizing the perfect performing and most economical applied sciences for every, will change into more and more essential.
A Mead-Conway Second for 3D Built-in Circuits
In 1978, Carver Mead, a professor on the California Institute of Expertise, and Lynn Conway at Xerox PARC invented a computer-aided design methodology for built-in circuits. They used a set of design guidelines to explain chip scaling in order that engineers might simply design very-large-scale integration (VLSI) circuits with out a lot data of course of know-how.
That very same kind of functionality is required for 3D chip design. As we speak, designers must know chip design, system-architecture design, and {hardware} and software program optimization. Producers must know chip know-how, 3D IC know-how, and superior packaging know-how. As we did in 1978, we once more want a standard language to explain these applied sciences in a method that digital design instruments perceive. Such a {hardware} description language offers designers a free hand to work on a 3D IC system design, whatever the underlying know-how. It’s on the way in which: An open-source customary, referred to as 3Dblox, has already been embraced by most of in the present day’s know-how firms and digital design automation (EDA) firms.
The Future Past the Tunnel
Within the period of synthetic intelligence, semiconductor know-how is a key enabler for brand spanking new AI capabilities and functions. A brand new GPU is not restricted by the usual sizes and kind components of the previous. New semiconductor know-how is not restricted to cutting down the next-generation transistors on a two-dimensional aircraft. An built-in AI system might be composed of as many energy-efficient transistors as is sensible, an environment friendly system structure for specialised compute workloads, and an optimized relationship between software program and {hardware}.
For the previous 50 years, semiconductor-technology growth has felt like strolling inside a tunnel. The street forward was clear, as there was a well-defined path. And everybody knew what wanted to be achieved: shrink the transistor.
Now, we now have reached the tip of the tunnel. From right here, semiconductor know-how will get tougher to develop. But, past the tunnel, many extra potentialities lie forward. We’re not sure by the confines of the previous.