MMOOZZGG.com

Glossary

What is a dataset?

In short

A dataset is a set of data collected and formatted to train or test a model: a table, a folder of pictures, or a body of texts in which every example is described in the same way.

AI basics · dataset

MMOOZZGG.com editorsChecked: 1 min read

For a child

You have an album of animal stickers, and under each one it says: lion, giraffe, elephant. If you give this album to a computer, it can learn to recognize animals. That is a dataset: a neat collection of examples with labels. The more different pictures it has, the more surely the program recognizes new animals.

In plain words

It is not just a pile of files but an ordered collection. Each example in it is described the same way: a photo with the label “cat”, or a table row with an apartment’s price, area, and district. The collection is usually split into parts: the model learns on one and is tested on another. Scientists and companies publish ready datasets openly, and companies build their own from work systems. If the examples contain errors or bias, the trained program will repeat them.

For business

Your own labeled history is often the main asset of an AI project: you can change the algorithm, but a competitor cannot buy your requests, contracts, or labeled defect photos. The cost is made up of export, cleaning, labeling, and quality checks; your own experts or contractors do the labeling. Check the rights in advance: can you use these records for training, do they contain personal data without consent, and who owns the labeling result? Open datasets also have licenses, and some of them forbid commercial use. A pilot shows how many examples are enough for a first result, before you buy hardware.

For a specialist

A dataset is a collection of examples with one schema: features, a target variable or labels, and metadata. For evaluation it is split into train, validation, and test sets; with little data, cross-validation is used. A classic example is ImageNet, an image database organized by the WordNet hierarchy. J. Deng, L. Fei-Fei, and co-authors presented it at CVPR 2009, and the project’s website now lists more than 14 million images. On part of it, A. Krizhevsky and co-authors trained the convolutional network described at NIPS 2012. Important properties are representativeness, class balance, label quality, and no duplicates between the splits. A dataset is documented with a datasheet (datasheet for datasets) following T. Gebru and co-authors (2018), with its license stated; it is stored as CSV, Parquet, or JSON Lines.

Example

A window maker built a dataset from photos of finished products on which craftsmen marked scratches and chips. A camera on the packing line was trained on these images.

Related terms

Questions and answers

What is a dataset in simple words? It is a collection of examples for training a program, where all records are formatted the same way. For example, a thousand product photos, each with a label of what it shows.
Where can you get a dataset to train a neural network? Universities and research communities publish open datasets on their websites and in model catalogs. For company tasks, your own records are more useful: you export them from work systems, clean them, and label them.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us