Information in Sentences
How can we measure the amount of information that some text has in it, since we can’t just use the length (for example, a long repetitive document might have less information than a short informative one even though it has more characters in it).
Being able to have a good guess at what is coming next is the basis of working out how to reduce the amount of data used for sending text, photos and videos (this is called compression, and is the underlying theory behind showing videos over the internet and storing lots of photos or songs on a mobile phone.
This kind of activity is used by computer scientists to measure the amount of "information" in a document. By doing this activity, you will be able to understand that computer scientists measure information by how surprising a message is.
Telling you something that you know already — for example, when a person who always takes the bus to work says "I took a bus to work today" — doesn’t give you any information, because it isn’t surprising. If that person said instead, "I teleported to work today" that would be surprising, and would therefore tell us a lot of information. The number of yes/no guesses indicates how much information there is in the text.