Glossary
Who was Claude Shannon
In short
Claude Shannon was an engineer and mathematician who created information theory in 1948 and became one of the authors of the proposal for the Dartmouth summer workshop on artificial intelligence.
People · Claude Shannon
MMOOZZGG.com editorsChecked: 2 min read
For a child
How do you pass a secret to a friend across a noisy lunchroom? You can agree in advance: one clap means “yes”, two claps mean “no”. Claude Shannon worked out how to count how many such answers you need to guess any message. One answer became a unit of information, like one block in a tower. Thanks to his ideas, cartoons on a tablet and grandma’s voice on the phone reach you without errors, even when the connection is bad.
In plain words
Shannon worked at Bell Labs, the research center of a telephone company. In 1948 he published the paper “A Mathematical Theory of Communication” and showed that information can be measured, like length or weight. The unit is called a bit: the answer to a question with two equal options. He also proved that data can be sent over a noisy channel almost without errors if it is coded correctly and stays below the channel’s speed limit. This led to archives, mobile networks and video compression. One more of his ideas matters for AI: text can be described by the probabilities of the next letter or word.
For business
Shannon’s theory works in every work tool of a company: in mobile networks, archives, video calls and cloud storage. It follows that any message has a compression limit and any channel has a speed limit. For AI projects, the practical lesson is this. Duplicates and empty documents bloat the knowledge base and the processing bill but add no information. Noise in the data, such as typos in product cards or scraps of correspondence, lowers the quality of a model’s answers just as interference on a line spoils a conversation. So before launch, teams check how many repeats, junk and gaps the data holds. This review is part of the data preparation that a rollout usually starts with.
For a specialist
The paper “A Mathematical Theory of Communication” appeared in the Bell System Technical Journal (vol. 27, July and October 1948). Shannon described a communication scheme of a source, a transmitter, a noisy channel, a receiver and a destination, and introduced entropy as a measure of the source’s uncertainty. He called the unit for a base-2 logarithm a bit, noting that John Tukey suggested the word. The paper contains the source coding theorem and the channel capacity theorem: below this limit, the probability of error can be made as small as you like. For language modeling, the section on approximations to English text matters: Shannon generated letter sequences from the frequencies of pairs and triples (digram, trigram), and word sequences from the frequencies of single words and pairs, that is, he built Markov n-gram models. The loss function of language models, cross-entropy, rests on the same measure. In the Dartmouth project proposal of 31 August 1955, he is listed as a mathematician at Bell Telephone Laboratories, the author of the statistical theory of information and of work on machines that learn.
Example
A clinic chain wants AI to transcribe calls to its front desk. Recordings from the line arrive heavily compressed, and some words get lost. The project lead first checks the recording quality and only then chooses a speech recognition model.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map