Glossary
What is a dataset?
In short
A dataset is a set of data collected and formatted to train or test a model: a table, a folder of pictures, or a body of texts in which every example is described in the same way.
AI basics · dataset
MMOOZZGG.com editorsChecked: 1 min read
For a child
You have an album of animal stickers, and under each one it says: lion, giraffe, elephant. If you give this album to a computer, it can learn to recognize animals. That is a dataset: a neat collection of examples with labels. The more different pictures it has, the more surely the program recognizes new animals.
In plain words
It is not just a pile of files but an ordered collection. Each example in it is described the same way: a photo with the label “cat”, or a table row with an apartment’s price, area, and district. The collection is usually split into parts: the model learns on one and is tested on another. Scientists and companies publish ready datasets openly, and companies build their own from work systems. If the examples contain errors or bias, the trained program will repeat them.
For business
Your own labeled history is often the main asset of an AI project: you can change the algorithm, but a competitor cannot buy your requests, contracts, or labeled defect photos. The cost is made up of export, cleaning, labeling, and quality checks; your own experts or contractors do the labeling. Check the rights in advance: can you use these records for training, do they contain personal data without consent, and who owns the labeling result? Open datasets also have licenses, and some of them forbid commercial use. A pilot shows how many examples are enough for a first result, before you buy hardware.
For a specialist
A dataset is a collection of examples with one schema: features, a target variable or labels, and metadata. For evaluation it is split into train, validation, and test sets; with little data, cross-validation is used. A classic example is ImageNet, an image database organized by the WordNet hierarchy. J. Deng, L. Fei-Fei, and co-authors presented it at CVPR 2009, and the project’s website now lists more than 14 million images. On part of it, A. Krizhevsky and co-authors trained the convolutional network described at NIPS 2012. Important properties are representativeness, class balance, label quality, and no duplicates between the splits. A dataset is documented with a datasheet (datasheet for datasets) following T. Gebru and co-authors (2018), with its license stated; it is stored as CSV, Parquet, or JSON Lines.
Example
A window maker built a dataset from photos of finished products on which craftsmen marked scratches and chips. A camera on the packing line was trained on these images.
Related terms
Questions and answers
Sources
- ImageNet, About ImageNet; J. Deng et al., ImageNet: A Large-Scale Hierarchical Image Database, CVPR 2009image-net.org
- A. Krizhevsky, I. Sutskever, G. E. Hinton, ImageNet Classification with Deep Convolutional Neural Networks, NIPS 2012papers.nips.cc
- T. Gebru et al., Datasheets for Datasets, 2018arxiv.org
Where companies use it
Not sure whether your company needs this?
AI solutions map