EC9027 INFORMATION THEORY lecture notes | for ECE - VII semester (elective) | by R.Anirudhan

EC9027 INFORMATION THEORY lecture notes | for ECE - VII semester (elective) | 




syllabus :

UNIT I QUANTITATIVE STUDY OF INFORMATION 
Basic inequalities, Entropy, Kullback-Leibler distance, Mutual information, Bounds on
entropy, Fisher information , Cramer Rao inequality, Second law of thermodynamics ,
Sufficient statistic , Entropy rates of a Stochastic process .

UNIT II CAPACITY OF NOISELESS CHANNEL 
Fundamental theorem for a noiseless channel ,Data compression , Kraft inequality ,
Shannon-Fano codes , Huffman codes , Asymptotic equipartition , Rate distortion
theory .

UNIT II I CHANNEL CAPACITY 
Properties of channel capacity , Jointly typical sequences , Channel Coding
Theorem, converse to channel coding theorem, Joint source channel coding theorem

UNIT IV DIFFERENTIAL ENTROPY AND GAUSSIAN CHANNEL 
AEP for continuous random variables, relationship between continuous and discrete
entropy, properties of differential entropy, Gaussian channel definitions, converse to
coding theorem for Gaussian channel, channels with colored noise, Gaussian
channels with feedback

UNIT V NETWORK INFORMATION THEORY 
Gaussian multiple user channels , Multiple access channel , Encoding of correlated
sources , Broadcast channel , Relay channel , Source coding and rate distortion with
side information , General multi-terminal networks
Previous
Next Post »