One major challenge in modeling, understanding, and predicting complex systems is quantifying the associated uncertainty due to the intrinsic turbulent nature or the external stochastic forcing. Information theory is a powerful tool for assessing the uncertainty, model error, and prediction skills of the underlying complex systems. It also facilitates the development of valuable criteria for building statistically accurate reduced-order models and measuring the predictability limit of nature. This chapter includes an introduction to information theory. The main focus will be on the properties and the applications of three widely used information measurements that are directly applicable to quantifying the uncertainty and model error. They are Shannon’s entropy, relative entropy, and mutual information. The maximum entropy principle, which provides an unbiased criterion for developing approximate PDFs and coarse-grained statistical models, will also be presented with concrete examples. In addition, the relationship between the information criteria and the path-wise measurements will be developed, which illustrates the necessity of adopting the former in studying many complex turbulent systems.