MMOOZZGG.com

Glossary

What is GPU rental?

In short

To rent GPU for AI means paying for access to someone else’s graphics cards in a data center by the hour or month, instead of buying your own server for neural networks.

Infrastructure · GPU rental

MMOOZZGG.com editorsChecked: 2 min read

For a child

Imagine you need a bike for one day to ride in the park. Buying a new one is expensive, and then it will gather dust in the hallway. You can rent a bike, ride it and give it back. People do the same with powerful computers for smart helpers: they take them for a while when there is big work to do, and then give them back.

In plain words

Powerful graphics cards for neural networks are expensive, and sometimes you need them for only a week. So providers put them in their data centers and rent them out by the hour. You choose a configuration, get a remote server and run your program on it. When the work is done, you shut down the server and stop paying. You can rent a whole card, part of one or several servers at once. For example, a team rents a powerful server for the weekend to fine-tune a model on its documents, and on Monday goes back to normal work on laptops.

For business

This format suits a pilot, one-off tasks such as fine-tuning, and load peaks when it is too early to buy hardware. Costs are variable: you pay for hours of work, storage and traffic. With constant round-the-clock load, renting over time costs more than your own server, so count the break-even point in advance. For a Russian company, the location of the hardware matters: with a provider in Russia it is easier to meet Russia’s personal data requirements. The risks: the card model you need may not be free, the provider changes prices, and your data sits on someone else’s infrastructure. Check the contract, the isolation between clients and how data is deleted after the work ends. Local prices and the break-even calculation are on a separate page of the site.

For a specialist

GPU rental belongs to the IaaS (Infrastructure as a Service) model, one of the three service models in the NIST SP 800–145 definition of cloud computing. Formats: a virtual machine with a whole card passed through (passthrough), a dedicated physical server (bare metal), a cluster with a fast interconnect for distributed training, and a share of a card. NVIDIA MIG (Multi-Instance GPU) lets you split one accelerator: starting with the Ampere architecture, a card can be divided into up to seven isolated instances, each with its own multiprocessors, L2 cache banks, memory controllers and DRAM buses, so one tenant does not affect the work and scheduling of another. For inference there are also serverless endpoints billed per second or per token. When choosing, compare video memory size and bandwidth, links between cards, driver and CUDA versions, idle-time billing and the cost of data egress.

Example

An online marketplace decided to test whether an open model can write product descriptions. Instead of buying a server, the team rents one graphics card from a Russian provider for a month, checks the quality on a thousand product pages, and only then decides whether to keep renting or buy hardware.

Related terms

Questions and answers

Which pays off more: renting a GPU or buying one? Renting pays off for irregular load, for a pilot and for one-off tasks. With constant load from morning to night, your own server pays back faster, and the break-even point is counted in hours of work per month.
Can you rent part of a graphics card? Yes. Some server accelerators can be split into isolated parts: NVIDIA MIG technology divides one card into several instances, each with its own memory and compute units.

This suits small models that do not need a whole card.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us