MMOOZZGG.com

Glossary

What is Moore's law?

In short

Moore’s law is Gordon Moore’s observation that the number of transistors on a chip doubles about every two years.

History of AI · Moore's law

MMOOZZGG.com editorsChecked: 1 min read

For a child

Imagine your building set doubles in size every couple of years, while the box stays the same. The pieces just get smaller and smaller. Something similar happened to computer chips for a long time. That is why the phone in your pocket can do more than the huge machines that once filled a whole room.

In plain words

In 1965 the engineer Gordon Moore noticed that the number of elements on one chip was growing very fast and steadily, and he predicted it would go on. Ten years later he refined the pace: doubling about every two years. The prediction became a target for the whole industry, and others called it a law, although it is an observation, not a law of nature. Thanks to this growth, computers became cheaper and faster, and at some point they had enough power to train large neural networks.

For business

For business, this rule was a hidden ally for a long time: what was expensive to compute today became cheaper in a couple of years. With AI, the math is not direct. The compute used to train large models grows much faster than chips get cheaper, so frontier models stay expensive. But running a ready model on your own server gets cheaper with each new generation of hardware. The practical lesson: do not buy hardware with a huge reserve for years ahead. Calculate the payback over one generation of graphics cards and compare it with the cost of the cloud. Our guide to graphics cards for AI helps with this calculation.

For a specialist

Gordon Moore, then director of research at Fairchild Semiconductor, published “Cramming more components onto integrated circuits” in Electronics magazine on 19 April 1965. From five data points for 1959–1964, he projected a doubling of components every 12 months and about 65,000 components on a chip by 1975. In 1975, at the IEEE International Electron Devices Meeting, Moore revised the pace to doubling every two years; Carver Mead suggested the name “Moore’s law”. Moore later co-founded Intel. For machine learning, the link to training compute matters: by the estimate of Sevilla et al. (2022), before 2010 it doubled about every 20 months, in step with Moore’s law, and in the deep learning era about every 6 months. Parallelism closes the gap: GPU clusters and specialized accelerators.

Example

A company is choosing between buying a server with eight graphics cards and renting in the cloud. The CFO asks the team to calculate the payback over the life of one card generation: in a few years the same task will cost much less on new hardware.

Related terms

Questions and answers

Who stated Moore’s law and when? Gordon Moore described the trend in an article in Electronics magazine on 19 April 1965, when he led research at Fairchild Semiconductor. In 1975 he refined the pace: doubling about every two years.
How is Moore’s law linked to AI? Growing chip power made it possible to train large neural networks. But the compute used to train models has grown much faster than Moore’s law since the early 2010s, so teams make up the gap with more accelerators, not only with new chips.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us