MMOOZZGG.com

Glossary

What is image to image AI?

In short

Image to image AI makes a new picture from a source image and a text prompt: the neural network keeps the original’s composition and changes the style or details.

Images, video, audio · image-to-image

MMOOZZGG.com editorsChecked: 2 min read

For a child

Imagine you drew a little house in pencil, a bit crooked and without colors. You give the sheet to a magic artist and say: “Make it a winter fairy tale.” The artist looks at your drawing and paints a new picture from it: the house stands in the same place, and around it there is snow, lights and snowdrifts. This is how the method works: the old picture shows where everything stands.

In plain words

Ordinary generation starts from empty noise, but here the neural network has a base: your photo, a sketch or an old banner. The program adds a little noise to the source, as if looking at it through cloudy glass, and then redraws the details from the text description. The main setting is called the change strength. A small value leaves the picture almost as it was and changes only colors or texture. A large one keeps only the general outline and draws an almost new image. For example, a photo of a room with bare walls can become the same room after a renovation in Scandinavian style.

For business

This mode is useful when a company already has a visual base and needs to multiply or restyle it fast. Marketing turns one product shot into a series of seasonal versions, a designer turns a packaging sketch into a realistic mockup, and a developer shows an apartment with different finishes. The price depends on the resolution, the number of iterations and where the model runs: in a cloud service that charges per image or on your own server with a graphics card. The risks: the model can distort a logo, text and the product’s shape, and uploading other people’s photos raises a question of rights. Before the launch, check the model’s license and agree on who approves the result. A good start is one product category, counting how many designer hours it saves.

For a specialist

Image-to-image (img2img) in a broad sense means translating an image from one domain to another: a sketch into a photo, a segmentation map into a scene, a black-and-white shot into a color one. Isola et al. (2016) solved this task with a conditional adversarial network (conditional GAN, pix2pix), which learns both the mapping and the loss function. For diffusion models, the standard technique is described in SDEdit (Meng et al., 2021): noise is added to the input image up to an intermediate step, and then the reverse process runs, in image generators usually with a text condition. The noise level sets the trade-off between realism and faithfulness to the source. In the diffusers library it is called strength, 0.8 by default; at 1 the source is in effect ignored, and the number of real denoising steps grows with strength. Related methods: inpainting (change by mask), ControlNet (structural conditions instead of pixels) and DDIM inversion for precise editing.

Example

An online clothing store uploads a photo of a jacket on a white background and gets the same product on an autumn city street and in the mountains. A designer checks that the cut and color did not change and picks frames for the product page.

Related terms

Questions and answers

How is image-to-image different from text-to-image? In text-to-image, the neural network draws only from a description and starts from pure noise. In image-to-image mode it has a source image, so the composition, angle and proportions stay, and the text says what to change.
Can I upload a product photo without spoiling it? Yes, if you set a low change strength and check the result by eye. To keep the shape exactly, it is safer to mask only the background or add a structural condition, such as the product’s outline.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us