Abstract
This introductory chapter provides a basic review of the Shannon entropy and of some important related quantities like the joint entropy, the conditional entropy, the mutual information and the relative entropy. We also discuss the Fisher information, the fundamental property of concavity, the basic elements of the maximum entropy approach and the definition of entropy in the Quantum case. We close this chapter with the axioms which determine the Shannon entropy and a brief description of other information measures.
Keywords: Shannon Entropy, Mutual Information, Relative Entropy, Fisher Information, von Neumann Entropy.
Related Books
Advances in Sampling Theory-Ratio Method of Estimation
Advances in Time Series Forecasting
Advances in Time Series Forecasting
Exponential Data Fitting and its Applications
Predictive Analytics Using Statistics and Big Data: Concepts and Modeling
Probability Theory for Fuzzy Quantum Spaces with Statistical Applications
Reliability Calculations with the Stochastic Finite Element
Recent Advances in Lifetime and Reliability Models
Introductory Statistical Procedures with SPSS