Information theory is a branch of mathematics based on probability theory and statistical theory. The founder of information theory Claude Shannon created the “mathematical theory of communication” and the successors of Shannon thoroughly developed firm constructions for description of communication processes with data compression and transmission.