??pb2?b2x(x)?xy)dy?(xy)dy?2?a1?x2f???f(?2x2a?b2?b22x2aa2,x?a同理??2py(y)?f(xy)dx?(xy)dx?2????a2?a2y2b?a2?a22b2y?b1?y2fb2,y?b??Hac(X)???px(x)logpx(x)dx?ln???e
??Hc(Y)???py(y)logpy(y)dy?ln?b??e????Hc(XY)??(xy)logf(xy)dxdy?ln(?ab)??????fI(X;Y)?Hc(X)?Hc(X)?Hc(XY)?ln?e#
2.25 设X和Y为连续随机变量，且X的概率密度为
q(x)?1x2/4?22??e?
条件概率密度为
p(y|x)?(1x)2/3?223??)e?(y?1
其中－∞ X~N(0,2?2),?2x?2?2,??Hc(X)???q(x)logq(x)dx?12log(4?e?2) 0 16 ????Hc(Y/X)??????????????q(x)p(y/x)logp(y/x)dxdy1?????????2??e?x2/4?211?(y?x)2/3?2?(y?x)2/3?211()e2log()e2dxdy3??3??1log(3?e?2)2??w(y)????q(x)p(y/x)dx??2???1?e?x2/4?21?(y?x)2/3?211?y2/4?22()edx?e3??2??Hc(Y)?1log(4?e?2)21log(4/3)21Hc(X/Y)?Hc(X)?I(X;Y)?log(3?e?2)2I(X;Y)?Hc(Y)?Hc(Y/X)? 2.27 设x为[0，∞]上分布的连续随机变量，且满足求实现最大微分熵的分布及相应的熵值。 解： ??#??0xq(x)dx=S， Hc(X)???q(x)lnq(x)dx, 约束条件为0???? ?q(x)dx=1 和?xq(x)dx=s .00??????设 f(q(x),?1,?2)???q(x)lnq(x)dx??1?xq(x)dx??2?q(x)dx000????0e??1x??2q(x)lndx?q(x)???0e??1x??2q(x)(?1)dxq(x)当f(q(x),?1,?2)为最大值时,Hc(X)在约束条件下取得最大值,此时q(x)?e??1x??2 17 ????由约束条件?q(x)dx=1 和?xq(x)dx=s,得00????1x??2edx?1?{??0?0xe??1x??2dx=s 1 解得 ?1? ,?2?lnssx1?s?实现Hc(X)max的分布为q(x)?e,x?0s??xx1?s1?sHc(X)max???elnedx?lns?1#ss02.28 令概率空间X???1/21/2??，令Y是连续随机变量。已知条件概率率密度为 ???1?1??1/4p(y|x)???0试求： ?2?y?x?2其它 (a) Y的概率密度ω(y) (b) I(X；Y) (c) 若对Y作如下的硬判决： ?1?V??0??1?y?1?1?y?1 y??1 求I(X；Y)，并对结果进行解释。 解:(a) 由已知， ,?3?y?1?1p(yx??1)??4 0,其它?,?1?y?3?1p(yx?1)??4 0,其它? 18 w(y)??pxy(xy)??px(x)pyx(yx)xx?px(x??1)pyx(yx??1)?px(x?1)pyx(yx?1) ,?3?y??1?18?1?4,?1?y?1??1?8,1?y?3?其它?0,（b） ?1118314H(Y)??3?log28dy??1?log24dy??1log28dy81 ?2.5bits13141log24dy?12?4log24dy?1H(YX)??2bits12?3? ?I(X;Y)?H(Y)?H(Y/X)?0.5bit (c)由 ?1,y?1?v??0,?1?y?1 ??1,y??1?可求得V的分布为 ??101?V???111?? ?424??1,y?1?再由p(y/x)及v??0,?1?y?1可求得V的条件分布为 ??1,y??1?,(v,x)??(?1,?1),(0,?1),(0,?1),(?1,?1)??1p(v/x)??2 0,(v,x)??(?1,?1),(1,?1)??1?H(V)?2?1log4?log22?1.5bit242H(V/X)???p(x??1)p(v/x??1)log2p(v/x??1)??p(x?1)p(v/x?1)log2p(v/x?1)vv?1bitI(V;X)?H(V)?H(V/X)?0.5bit可见I(X;Y)?I(X;V),Y?V变换没有信息损失. 19 第三章 离散信源无失真编码 3.1解:长为n码字的数目为Dn ,因此长为N的D元不等长码至多有： D(DN?1) ?D? D?1k?1Ni3.2 解: (a)长为100的事件序列中含有两个和更少个a1的序列数目为012M?C100?C100?C100?1?100?4950?5051因此在二元等长编码下所需码长为 N??log25051???12.3??13 (b)误组率为长为100的事件序列中含有三个a1的序列出现的概率,因此有012Pe?1?C1000.996100?C100?0.99699?0.004?C100?0.9962?0.0042?7.755?10?33.3 解: 3.4 解: 20