In the race to speed up artificial intelligence, Silicon Valley company Cerebras is taking an unusual strategy: Go big.
While a typical computer chip is the size of a fingernail, Cerebras' chip is the size of a dinner plate.
Deep learning, an AI technology that powers voice assistants, self-driving cars and Go champions, relies on complex "neural network" software arranged in layers. Deep learning systems can run on a single computer, but the largest systems are spread across thousands of connected machines, sometimes in large data centers, such as those operated by Google. In a large cluster, up to 48 pizza-box-sized servers slide into one-man-tall racks; The shelves are lined up in rows and fill a building the size of a warehouse. The neural networks in these systems can solve daunting problems, but they also face obvious challenges. A network that proliferates in a cluster is like a brain scattered across a room and connected together. Electrons move fast, but even so, communication across chips is slow and consumes a lot of energy.
Eric Vishria, a general partner at San Francisco venture capital firm Benchmark, first realized the problem when he heard Cerebras Systems, a new computer chip company, speak in the spring of 2016. Benchmark is known for being an early investor in companies like Twitter, Uber, and ebay -- that is, in software, not hardware. The company looks at about 200 start-ups a year and invests in one. "We were playing this game of kissing a thousand frogs," Vishria told me. At the beginning of his speech, he decided to throw the frog back. "I thought, why did I agree to this?" We're not going to invest in hardware, "he recalled thinking. "It's stupid."
Cerebras co-founder Andrew Feldman started with the slide cover to his team slide and caught Vishria's attention: his talent was impressive. Feldman then compared the two types of computer chips. First, he looked at graphics processing units, or Gpus -- chips designed specifically for creating 3D images. Today's machine learning systems rely on these graphics chips for a variety of reasons. Next, he looked at central processing units, or cpus, the general-purpose chips that do most of the work in a typical computer. "The third slide was about 'Gpus,' which are actually bad for deep learning -- they just happen to be a hundred times better than cpus." Cerebras has come up with a new type of chip that is not designed for graphics but specifically designed for artificial intelligence.
Vishria is used to hearing pitches from companies that plan to use deep learning in cybersecurity, medical imaging, chatbots and other applications. After Cerebras's talk, he spoke with engineers at companies funded by Benchmark, including Zillow, Uber, and Stitch Fix; They told him they were having trouble with AI because it took too long to "train" the neural network. Google has started using super-fast "tensor processing units," or Tpus, special chips designed for ARTIFICIAL intelligence. Vishria knew there was a gold rush going on and someone had to make the picks and shovels.
That year, Benchmark and Foundation Capital, another venture Capital firm, led a $27 million round of funding for Cerebras, which has raised nearly $500 million. Other companies are also making so-called artificial intelligence accelerators; Cerebras' competitors groq, Graphcore and Sambanova raised more than $2 billion in capital between them. But Cerebras' approach is unique. Instead of printing dozens of wafers on a large piece of silicon, cutting them off and connecting them to each other, the company has created a giant "wafer level" chip. While a typical computer chip is the size of a fingernail, Cerebras is about the size of a dinner plate and is the largest computer chip in the world.
Even competitors found the feat impressive. "This is new science," Nigel Toon, Graphcore's chief executive and co-founder, told me. "It's an incredible piece of engineering. It's a masterpiece." Meanwhile, another engineer I spoke to described it as a science project -- big for big's sake. In the past, the company has tried and failed to make giant chips; Cerebras' plan amounts to a bet that overcoming engineering challenges is possible and worth it. "To be honest, for me, ignorance is an advantage," Vishria said. "I don't know if I knew how hard it is to do what they do, I would have the courage to invest."
It is easy to take for granted that computers are getting faster and faster. This is often explained by Moore's Law: the pattern established in 1965 by semiconductor pioneer Gordon Moore, according to which the number of transistors on a chip doubles every year or every two years. Of course, Moore's Law isn't really a law, and engineers work tirelessly to shrink transistors while also improving the "architecture" of each chip to create more efficient and powerful designs.
Chip architects have long wondered whether a single, large-scale computer chip might be more efficient than a bunch of smaller chips, just as a city with concentrated resources and dense blocks is more efficient than a suburb. The idea was first tried in the 1960s, when Texas Instruments limited production of chips a few inches wide. But the company's engineers ran into yield problems. On any given silicon wafer, manufacturing defects inevitably endanger a certain number of circuits. If a wafer contains 50 chips, the company can throw away the bad ones and sell the good ones. But if every successful chip depended on the working circuitry of a single wafer, many expensive wafers would be discarded. Texas Instruments found a solution, but the technology and the need weren't there yet.
In the 1980s, an engineer named Gene Amdahl tried again to solve the problem with a company he founded called Trilogy Systems. It became the largest startup in Silicon Valley's history, with about $250 million in funding. To address the yield problem, Trilogy printed redundant components onto the chip. This method increases production but reduces the speed of the chip. Meanwhile, Trilogy is struggling in other ways. Amdahl ran over a motorcyclist with his rolls Royce, causing legal trouble; Its president died of a brain tumor; Heavy rains have delayed construction of factories, rusting air-conditioning systems and collecting dust on chips. In 1984, Trilogy gave up. "I didn't realize how hard it would be," Amdahl's son told The Times.
If Trilogy's technology is successful, it could now be used for deep learning. Instead, Gpus (chips used in video games) are solving scientific problems in national laboratories. Reusing gpus for AI depends on the fact that neural networks, while very complex, rely on a lot of multiplication and addition. When the "neurons" in the network fire each other, they amplify or reduce each other's signals, multiplying them by coefficients called connection weights. An efficient AI processor will compute many activations in parallel; It combines them into a series of numbers called vectors, or grids of numbers called matrices, or higher-dimensional blocks called tensors. Ideally, you want to multiply one matrix or tensor by another at once. Gpus are designed to do something similar:
"The shadow of Trilogy is so big," Feldman told me recently, "that people stop thinking and start saying, 'It's impossible. '" GPU companies, including Nvidia, jumped on the opportunity to customize their chips for deep learning. In 2015, Feldman and a group of computer architects began discussing the idea of bigger chips after they co-founded a computer server maker, Seamicro, which they sold to chip maker AMD for $334 million. They worked on the issue for four months in an office borrowed from a venture capital firm. When they had an outline of a viable solution, they talked to eight companies; Got funding from Benchmark, Foundation Capital, and Eclipse, and started hiring.
Cerebras' first task is to solve the manufacturing problems that plague large chips. The chip was originally a cylindrical ingot of crystalline silicon about a foot in diameter, and the steel ingot was cut into wafers less than a millimeter thick. The circuit is then "printed" onto the wafer through a process called lithography. Uv-sensitive chemicals are carefully deposited on the surface, and then a beam of UV light is projected through a detailed template called a mask. These chemicals react to form circuits.
Normally, the area covered by light projected through the mask becomes a chip. Then the chip moves and the light is projected again. After dozens or hundreds of chips have been printed, they are laser-cut from the wafer. "The easiest way to do it is for your mom to take out a round cookie dough," Feldman said. "She has a cookie mold and she cuts the cookies carefully." The laws of physics and optics make it impossible to make a bigger cookie cutter. As a result, "we developed a technology so that you can communicate through a little dough between two cookies."
In the printing system Cerebras developed in collaboration with TSMC, the company that makes the chip, the edges of the cookies overlap so that their wires are connected. The result is a single "wafer size" wafer, copper colored square and 21cm on each side. (The largest Gpus are slightly less than 3cm in diameter.) Cerebras produced its first chip, Wafer-scale Engine 1, in 2019. Wse-2, introduced this year, uses a denser circuit, with 2.6 trillion transistors packed into 850,000 processing units, or "cores". (Top Gpus have only a few thousand cores, while most cpus have fewer than 10.)
"2.6 trillion transistors is astounding," said Aart de Geus, Chairman and co-CEO of Synopsys. Synopsys provides some software that Cerebras and other chipmakers use to make and validate their chip designs. De Geus says that when designing chips, engineers first have to consider two core questions: "Where does the data come from?" Where is it handled?" When chips were simpler, designers could answer these questions with a pencil on a drawing table; When working with today's more complex chips, enter code that describes the architecture they want to create, then move on to visualization and coding tools. "Think about how the house looks from the roof," de Geus said. "Is the garage near the kitchen? Or is it close to the bedroom? You want it near the kitchen -- otherwise, you'll have to carry groceries through every corner of the house." After designing the floor plan, he explained, "you can use equations to describe what is happening in the room."
The design complexity of chips is mind-boggling. "There are many layers here," de Geus said, with circuits crisscrossed and layered on top of each other, like a major highway overpass. For the Cerebras engineers, working on the scale of a wafer, the complexity is heightened. Synopsys' software helps in the form of artificial intelligence: pattern matching algorithms identify common problems and propose solutions; The optimizer program moves the room to a faster, more efficient arrangement. If too many lanes try to squeeze into a two-block building, the software allows engineers to play Robert Moses and move the block.
In the end, Feldman says, there are several advantages to oversized chip designs. When the cores are on the same chip, they communicate faster: the computer's brain is now concentrated in a single skull, rather than scattered across a room. Larger chips also handle memory better. Typically, a small chip ready to process a file must first obtain the file from a shared memory chip located elsewhere on the circuit board; Only the most commonly used data is cached closer to home. In describing the efficiency of wafer-level chips, Feldman offered an analogy: He asked me to imagine a group of roommates (core) living in a dorm (chip) who wanted to watch a football game (do computing work). In order to watch the game, feldman says, roommates need to store beer in the refrigerator (data is stored in memory); Cerebras keeps a refrigerator in each room so roommates don't have to risk going to the dorm's communal kitchen or Safeway. This has the added benefit of allowing each core to process different data more quickly. "So I can have Bud in my dorm room," Feldman said. "In your dorm, you can have Schlitz."
Finally, Cerebras must overcome yield problems. The company's engineers use Trilogy's trick: redundancy. But here they have an advantage over their predecessors. Trilogy tries to make generic chips with many different components, so wiring around a single failed component might require connecting to a distant replacement. On Cerebras' chip, all the cores are identical. If one biscuit is wrong, the ones around it are just as good.