problem 1)a) What do you mean by information? What are its units? How does it relate to the entropy?

b) Suppose we have ten messages of probabilities: P(m₁) = 0.49, P(m₂) = 0.14,P(m₃) = 0.14, P(m₄) = 0.07, P(m₅) = 0.07, P(m₆)=.04, P(m7) = 0.02, P(m₈) = 0.02, P(m₉) = 0.005, P(m_l0) = 0.005. Determine the Shanon Fano code for the set of messages. Determine coding efficiency and redundancy.

problem 2) A transmitter has the alphabet consisting of five letters (x1 x2, x3, x4, x5) and receiver has the alphabet of four letters (y1, y2, y3, y4). The probabilities are P (x₁) = 0.2 5, P(x₂) = 0. 4, P(x₃) = 0.1 5, P(x₄) = 0.15, P(x₅) = 0.05 and

P(Y/X) =

         Y₁         Y₂        y₃       y₄
X₁      1          0          0          0
X₂    0.25     0.75       0          0
X₃      0        1/3       2/3        0
X₄      0          0         1/3      2/3
X₅      0          0           1          0

Compute entropies {H(Y), H(X, Y), H(X), H(X/Y), H(Y/X)} and I(X, Y).

problem 3)a) State and describe Sampling theorem in detail. describe how is it useful in communication systems?

b) describe Shanon’s Hartley theorem based on channel capacity. How does channel capacity modify if bandwidth is increased to infinity? describe the orthogonal signalling performance on the basis of theorems.

problem 4) Describe the significance of the following:

a) Companding.

b) Prediction filters.

c) Adaptive filters.

d) Equalization