?X??x1x2?3.1 设信源????通过一干扰信道，接收符号为Y = { y1, y2 }，信道转移矩??P(X)??0.60.4??51???阵为?66?，求：
13???44?(1) 信源X中事件x1和事件x2分别包含的自信息量；
(2) 收到消息yj (j=1,2)后，获得的关于xi (i=1,2)的信息量； (3) 信源X和信宿Y的信息熵；
(4) 信道疑义度H(X/Y)和噪声熵H(Y/X)； (5) 接收到信息Y后获得的平均互信息量。
解： 1)
I(x1)??log2p(x1)??log20.6?0.737 bitI(x2)??log2p(x2)??log20.4?1.322 bit2)
51p(y1)?p(x1)p(y1/x1)?p(x2)p(y1/x2)?0.6??0.4??0.66413p(y2)?p(x1)p(y2/x1)?p(x2)p(y2/x2)?0.6??0.4??0.464p(y1/x1)5/6I(x1;y1)?log2?log2?0.474 bitp(y1)0.6p(y2/x1)1/6I(x1;y2)?log2?log2??1.263 bitp(y2)0.4I(x2;y1)?log2I(x2;y2)?log23)
p(y1/x2)1/4?log2??1.263 bitp(y1)0.6p(y2/x2)3/4?log2?0.907 bitp(y2)0.4H(X)???p(xi)logp(xi)??(0.6log0.6?0.4log0.4)log210?0.971 bit/symboliH(Y)???p(yj)logp(yj)??(0.6log0.6?0.4log0.4)log210?0.971 bit/symbolj
4)
H(Y/X)????p(xi)p(yj/xi)logp(yj/xi)ij55111133 ??(0.6?log?0.6?log?0.4?log?0.4?log)?log21066664444 ?0.715 bit/symbol?H(X)?H(Y/X)?H(Y)?H(X/Y)?H(X/Y)?H(X)?H(Y/X)?H(Y) ?0.971?0.715?0.971?0.715 bit/symbol
· 17 ·
5)
I(X;Y)?H(X)?H(X/Y)?0.971?0.715?0.256 bit/symbol
?21???3.2 设二元对称信道的传递矩阵为?33?
12???33?(1) 若P(0) = 3/4, P(1) = 1/4，求H(X), H(X/Y), H(Y/X)和I(X;Y)； (2) 求该信道的信道容量及其达到信道容量时的输入概率分布；
解： 1)
3311H(X)???p(xi)??(?log2??log2)?0.811 bit/symbol4444iH(Y/X)????p(xi)p(yj/xi)logp(yj/xi)ij322311111122 ??(?lg??lg??lg??lg)?log210433433433433 ?0.918 bit/symbol3211p(y1)?p(x1y1)?p(x2y1)?p(x1)p(y1/x1)?p(x2)p(y1/x2)?????0.5833
43433112p(y2)?p(x1y2)?p(x2y2)?p(x1)p(y2/x1)?p(x2)p(y2/x2)?????0.41674343H(Y)???p(yj)??(0.5833?log20.5833?0.4167?log20.4167)?0.980 bit/symboljI(X;Y)?H(X)?H(X/Y)?H(Y)?H(Y/X)H(X/Y)?H(X)?H(Y)?H(Y/X)?0.811?0.980?0.918?0.749 bit/symbolI(X;Y)?H(X)?H(X/Y)??0.811?0.749?0.062 bit/symbol 2)
1122C?maxI(X;Y)?log2m?Hmi?log22?(lg?lg)?log210?0.082 bit/symbol3333
1p(xi)?23.3 设有一批电阻，按阻值分70%是2KΩ，30%是5 KΩ；按瓦分64%是0.125W，其余是0.25W。现已知2 KΩ阻值的电阻中80%是0.125W，问通过测量阻值可以得到的关于瓦数的平均信息量是多少？
解：
对本题建立数学模型如下：
?X阻值??x1?2??x2?5????Y瓦数??y1?1/8? ??????0.7??0.3??P(X)???P(Y)??0.64p(y1/x1)?0.8,p(y2/x1)?0.2求：I(X;Y)以下是求解过程：
y2?1/4??0.36?
· 18 ·
p(x1y1)?p(x1)p(y1/x1)?0.7?0.8?0.56p(x1y2)?p(x1)p(y2/x1)?0.7?0.2?0.14?p(y1)?p(x1y1)?p(x2y1)?p(x2y1)?p(y1)?p(x1y1)?0.64?0.56?0.08?p(y2)?p(x1y2)?p(x2y2)?p(x2y2)?p(y2)?p(x1y2)?0.36?0.14?0.22H(X)???p(xi)???0.7?log20.7?0.3?log20.3??0.881 bit/symboli
H(Y)???p(yj)???0.64?log20.64?0.36?log20.36??0.943 bit/symboljH(XY)????p(xiyj)logp(xiyj)ij ???0.56?log20.56?0.14?log20.14?0.08?log20.08?0.22?log20.22? ?1.638 bit/symbolI(X;Y)?H(X)?H(Y)?H(XY)?0.881?0.943?1.638?0.186 bit/symbol
3.4 若X, Y, Z是三个随机变量，试证明
(1) I(X;YZ) = I(X;Y) + I(X;Z/Y) = I(X;Z) + I(X;Y/Z)；
证明：
I(X;YZ)????p(xiyjzp(xi/yjzk)k)logijkp(xi) ????p(xp(xi/yjzk)p(xi/yj)iyjzk)logijkp(xi)p(xi/yj) ????p(xp(xi/yj)p(xi/yjzk)iyjzk)logp(x?p(xiyjzk)logijki)???ijkp(xi/yj) ?I(X;Y)?I(X;Z/Y)I(X;YZ)????p(xp(xi/yjzk)iyjzk)logijkp(xi) ????p(xiyjzk)logp(xi/yjzk)p(xi/zk)ijkp(xi)p(xi/zk) ????p(xp(xi/zk)iyjzk)log?ijkp(x???p(xlogp(xi/yjzk)iyjzk)i)ijkp(xi/zk) ?I(X;Z)?I(X;Y/Z)
(2) I(X;Y/Z) = I(Y;X/Z) = H(X/Z) – H(X/YZ)；
证明：
I(X;Y/Z)????p(xxi/yjzk)iyjzk)logp(ijkp(xi/zk)
????p(xi/yjzk)p(yjzk)iyjzk)logp(xijkp(xi/zk)p(yjzk) 19 ··
????p(xiyjzk)logijkp(xiyjzk)p(xi/zk)p(zk)p(yj/zk)p(xiyjzk)p(xizk)p(yj/zk)p(xiyjzk)p(xizk)p(yj/zk)p(yj/xizk)p(yj/zk) ????p(xiyjzk)logijk ????p(xiyjzk)logijk ????p(xiyjzk)logijk ?I(Y;X/Z)p(xi/yjzk)p(xi/zk)ijkI(X;Y/Z)????p(xiyjzk)logijkijk ?????p(xiyjzk)logp(xi/zk)????p(xiyjzk)logp(xi/yjzk)?? ??????p(xiyjzk)?logp(xi/zk)?H(X/YZ)ik?j? ????p(xizk)logp(xi/zk)?H(X/YZ)ik
?H(X/Z)?H(X/YZ)
(3) I(X;Y/Z) ≥0，当且仅当(X, Y, Z)是马氏链时等式成立。
证明：
?I(X;Y/Z)????p(xiyjzk)logijkp(xi/yjzk)p(xi/zk)p(xi/zk)p(xi/yjzk)??I(X;Y/Z)????p(xiyjzk)logijk?p(xi/zk)?? ????p(xiyjzk)?1?log2e??ijk?p(xi/yjzk)???p(xi/zk)? ????p(xiyjzk)????p(xiyjzk)?log2e?ijk?p(xi/yjzk)ijk??????? ?????p(yjzk)?p(xi/zk)?1?log2e?i????jk??? ???p(xi/zk)?1?log2e?i? ?0?I(X;Y/Z)?0 当
p(xi/zk)?1?0时等式成立
p(xi/yjzk)· 20 ·
?p(xi/zk)?p(xi/yjzk)?p(yjzk)p(xi/zk)?p(xi/yjzk)p(yjzk)?p(zk)p(yj/zk)p(xi/zk)?p(xiyjzk)?p(yj/zk)p(xi/zk)?p(xiyjzk)/p(zk)?p(yj/zk)p(xi/zk)?p(xiyj/zk)所以等式成立的条件是X, Y, Z是马氏链
3.5若三个随机变量，有如下关系：Z = X + Y，其中X和Y相互独立，试证明：
(1) I(X;Z) = H(Z) - H(Y)； (2) I(XY;Z) = H(Z)； (3) I(X;YZ) = H(X)； (4) I(Y;Z/X) = H(Y)；
(5) I(X;Y/Z) = H(X/Z) = H(Y/Z)。
解： 1)
?Z?X?Y?p(z/x)???p(yj) (zk?xi)?Yki)?p(zk?xi?0 (zk?xi)?YH(Z/X)????p(xizk)log2p(zk/xi)ik ???p(x?/x?i)??p(zk/xi)log2p(zki)i?k??
???p(x?p(y?i)??j)log2p(yj)?i?j? ?H(Y)?I(X;Z)?H(Z)?H(Z/X)?H(Z)?H(Y) 2)
?Z?X?Y?p(z)????1 (xi?yj)?zkk/xiyj??0 (xi?yj)?zkH(Z/XY)?????p(xiyjzk)log2p(zk/xiyj)ijk
????p(x?x?iyj)??p(zk/xiyj)log2p(zk/iyj)ij?k?? ?0?I(XY;Z)?H(Z)?H(Z/XY)?H(Z)?0?H(Z) 3)
21 ··