2002 Copyright EE Lab508
1.15两个离散随机变量X和Y，其和为Z＝X＋Y，若X和Y统计独立，求证： (1) H(X)≤H(Z), H(Y)≤H(Z) (2) H(XY)≥H(Z) 证明：
设X、Y的信源空间为：?Y b1 b2 ... bs?X a1 a2 ... ar[X?P]:? [Y?P]:??P(X) p1 p2 ... pr ?P(Y) q1 q2 ... qs 又X,Y统计独立trsrs?H(Z)???pzklogpzk???k?1ti?1?j?1si?1p(ai?bj)logp(ai?bj) ???i?1sss?(pj?1i?qj)log(pi?qj)?H(XY)
又H(Z) ＝?r?k?1spzklogpzk??(?(pilog(pi?qj))??j?1ri?1spiqj)?log(sj??i?1j?1piqj)? ? ???i?1r?j?1sj?1??qj?1j?1log(pi?qj) ? ???qi?1jlog(pi?qj)?-?qjlog(qj)第二章 单符号离散信道
?X a1 a2 2.1设信源[X?P]:? 通过一信道，信道的输出随机变量Y的符号集
P(X) 0.7 0.3? b1 b2Y:{b1,b2},信道的矩阵：
[P]?a1?5/6?a2?1/41/6?
?3/4?试求：
(1) 信源X中的符号?1和?2分别含有的自信息量；
(2) 收到消息Y＝b1，Y＝b2后，获得关于?1、?2的互交信息量：I(?1;b1)、I(?1;b2)、I(?2;b1)、
I(?2;b2)； (3) 信源X和信宿Y的信息熵；
(4) 信道疑义度H(X/Y)和噪声熵H(Y/X)；
(5) 接收到消息Y后获得的平均互交信息量I(X;Y)。 解:
?H.F.
2002 Copyright EE Lab508
(1) I(a1)??logp(a1)??log0.7?0.5415 bit I(a2)??logp(a21)??log0.3?1.737 bit(2) I(a1;b1)?log I(a1;b2)?log I(a2;b1)?log I(a2;b2)?log2p(b1a1)p(b1)p(b2a1)p(b2)?log5/60.7?5/6?0.3?1/41/60.7?1/6?0.3?3/41/4?0.34 bit?log??1.036 bitp(b1a2)p(b1)p(b2a2)p(b2)?log0.7?5/6?0.3?1/43/40.7?1/6?0.3?3/47912041120??0.766 bit?log?1.134 bit(3)由上:p(b1)? p(b2)?2?i?12p(ai)p(b1ai)?p(ai)p(b2ai)??i?1?H(X)???p(ai)logp(ai)??(0.7log0.7?0.3log0.3)?0.881 bit/symblei?12 H(Y)???p(bj)logp(bj)??(j?12279120log791202?241120log41120)?0.926 bit/symble(4)H(YX)???j?1i?1p(aibj)logp(bjai)???j?1i?1p(ai)p(bjai)logp(bjai)?0.698 bit/symble 又I(X;Y)?H(Y)?H(YX)?H(X)?H(XY) ?H(XY)?H(X)?H(YX)?H(Y)?0.881?0.698?0.926?0.653 bit/symble(5)?I(X;Y)?H(Y)?H(YX)?0.926?0.698?0.228 bit/symble2.2某二进制对称信道，其信道矩阵是：
0 10?0.98[P]??1?0.020.02?
?0.98?设该信道以1500个二进制符号/秒的速度传输输入符号。现有一消息序列共有14000个二进制符号，并设在这消息中p(0)= p(1)=0.5。问从消息传输的角度来考虑，10秒钟内能否将这消息序列无失真的传送完。 解:
由于二进制对称信道输入等概信源?I(X;Y)?C?1?H(?)?1??log??(1??)log(1??)?1?0.02log0.02?0.98log0.98?0.859 bit/symble?信道在10秒钟内传送14000个二进制符号最大码率Ct?C?14000symble/10s?1201.98 bit/s而输入信源码率为1500bit/s,超过了信道所能提供的最大码率,故不可能无失真传输.为:?H.F.
2002 Copyright EE Lab508
2.3有两个二元随机变量X和Y，它们的联合概率为P[X=0,Y=0]=1/8，P[X=0,Y=1]=3/8，P[X=1,Y=1]=1/8，P[X=1,Y=0]=3/8。定义另一随机变量Z=XY，试计算： (1) H(X),H(Y),H(Z),H(XZ),H(YZ),H(XYZ);
(2) H(X/Y),H(Y/X),H(X/Z),H(Z/X),H(Y/Z),H(Z/Y),H(X/YZ),H(Y/XZ),H(Z/XY); (3) I(X;Y),I(X;Z),I(Y;Z),I(X;Y/Z),I(Y;Z/X),I(X;Z/Y)。 解:
(1)由题意: X的分布:p(X?0)? Y的分布:p(Y?0)?1818??3838??1212;p(X?1)?1818??38838???1212.;p(Z?1)?3818.18;.;p(Y?1)?18 Z?XY的分布为:X的分布:p(Z?0)?且p(X?0,Z?0)?p(X?0)? p(Y?0,Z?0)?p(Y?0)??H(X)??( H(Y)??(12log1212??1212log12121212?38?378;p(X?0,Z?1)?0;p(X?1,Z?0)?38;p(X?1,Z?1)?18;;p(Y?0,Z?1)?0;p(Y?1,Z?0)?;p(Y?1,Z?1)?)?1 bit/symble; 12loglog)?1 bit/symble7711 H(Z)??(log?log)?0.544 bit/symble888822H(XZ)????p(xizk)logp(xizk)i?1k?1 ??(pxz(00)logpxz(00)?pxz(10)logpxz(10)?pxz(01)logpxz(01)?pxz(11)logpxz(11))133311??13 ??(?)log(?)?log?0?log?1.406bit/symble?88?888888??由上面X、Y、Z的概率分布:H(YZ)?H(XZ)?1.406bit/symble
?H.F.
2002 Copyright EE Lab508
(2)p(X?0Y?0)?pxy(00)?pxy(01)py(1)2pxy(00)py(0)?1/81/2?14;pxy(10)?14pxy(10)py(0)?3/81/2?34;pxy(01)??23/81/2?34;pxy(11)?pxy(11)py(1)?1/81/2?.?H(XY)???i?1?j?1p(xiyj)logp(xiyj)??pxy(00)logpxy(00)?pxy(01)logpxy(01)?pxy(10)logpxy(10)?pxy(11)logpxy(11)??(18?log14?38?log34?38?log34?18?log14)?0.811bit/symble???I(X;Y)?H(X)?H(XY)?H(Y)?H(YX)且H(X)?H(Y)?H(YX)?H(XY)?0.811bit/symble同理:2222H(XZ)????p(xizk)logp(xizk)????p(xizk)logp(xizk)p(zk)i?1k?1i?1k?1???pxz(00)logpxz(00)?pxz(01)logpxz(01)?pxz(10)logpxz(10)?pxz(11)logpxz(11)???(12?log1/27/82?0?238?log3/87/8?18?log1/81/82)?0.862 bit/symble2H(ZX)????p(zkxi)logp(zkxi)????p(zkxi)logp(zkxi)p(xi)k?1i?1k?1i?1???pzx(00)logpzx(00)?pzx(01)logpzx(01)?pzx(10)logpzx(10)?pzx(11)logpzx(11)???(12?log1/21/2?0?38?log3/81/2?18?log1/81/2)?0.406 bit/symble由X、Y、Z的概率: H(YZ)?H(XZ)?0.862 bit/symble H(ZY)?H(ZX)?0.406 bit/symble?pxyz(001)?pxyz(101)?pxyz(011)?pxyz(110)?0222222?H(XYZ)???i?1??j?1k?1p(xiyjzk)logp(xiyjzk)???i?1??j?1k?1p(xiyjzk)logp(xiyjzk)p(yjzk)?pxyz(111)logpxyz(111)pyz(11))??(pxyz(000)logpxyz(000)pyz(00)?pxyz(010)logpxyz(010)pyz(10)?pxyz(100)logpxyz(100)pyz(00)11/833/833/811/8??(log?log?log?log)?0.406 bit/symble81/283/881/281/8H(YXZ)?H(XYZ)?0.406 bit/symble222222?H(ZXY)???i?1??j?1k?1p(xiyjzk)logp(zkxiyj)???i?1??j?1k?1p(xiyjzk)logp(xiyjzk)p(xiyj)?pxyz(111)logpxyz(111)pxy(11))??(pxyz(000)logpxyz(000)pxy(00)?pxyz(010)logpxyz(010)pxy(01)?pxyz(100)logpxyz(100)pxy(10)11/833/833/811/8??(log?log?log?log)?0 bit/symble81/883/883/881/8?H.F.
2002 Copyright EE Lab508
(3)由上:I(X;Y)?H(X)?H(XY)?1?0.811?0.189 bit/symble I(X;Z)?H(X)?H(XZ)?1?0.862?0.138 bit/symble I(Y;Z)?H(Y)?H(YZ)?1?0.862?0.138 bit/symble I(X;YZ)?H(XZ)?H(XYZ)?0.862?0.406?0.456 bit/symble I(Y;ZX)?H(YX)?H(YXZ)?0.811?0.406?0.405 bit/symble I(X;ZY)?H(XY)?H(XYZ)?0.811?0.406?0.405 bit/symble
2.4已知信源X的信源空间为
?X: a1 a2 a3 a4 [X?P]:??P(X): 0.1 0.3 0.2 0.4某信道的信道矩阵为：
b1 b2 b3 b4
a1?0.2?a20.6?a3?0.5?a4?0.10.30.20.20.30.10.10.10.40.4??0.1? 0.2??0.2?试求：
(1)“输入?3，输出b2的概率”；
(2)“输出b4的概率”；
(3)“收到b3条件下推测输入?2”的概率。 解：
(1)p(a3;b2)?p(a3)p(b2a3)?0.2?0.2?0.0444(2)p(b4)?(3)p(b3)??i?14p(aib4)?p(aib3)??i?14p(ai)p(b4ai) ?0.1?0.4?0.3?0.1?0.2?0.2?0.4?0.2?0.19p(ai)p(b3ai)?0.1?0.1?0.3?0.1?0.2?0.1?0.4?0.4?0.22?0.3?0.10.22?0.136?i?1?i?1 p(a2b3)?p(a2)p(b3a2)p(b3)
2.5已知从符号B中获取关于符号A的信息量是1比特，当符号A的先验概率P(A)为下列各值时，分别计算收到B后测A的后验概率应是多少。 (1) P(A)=10-2； (2) P(A)=1/32； (3) P(A)=0.5。
?H.F.