（1）假设黑白消息视为前后无关，求信源熵H(X)，并画出该信源的香农线图
（2）实际上各个元素之间是有关联的，其转移概率为：P(白|白)＝0.9143，P(黑|白)＝0.0857，P(白|黑)＝0.2，P(黑|黑)＝0.8，求这个一阶马尔可夫信源的信源熵，并画出该信源的香农线图。
（3）比较两种信源熵的大小，并说明原因。 解：（1）H(X)?0.3log2P(黑|白)=P(黑)
0.70.3黑0.3白0.71010?0.7log2?0.8813bit/符号 37P(白|白)＝P(白) P(黑|黑)＝P(黑) P(白|黑)＝P(白)
（2）根据题意，此一阶马尔可夫链是平稳的（P(白)＝0.7不随时间变化，P(黑)＝0.3不随时 间变化）
H?(X)?H(X2|X1)??p(xi,yj)log2ij1p(xi,yj)?0.9143?0.7log2?0.8?0.3log210.8111?0.0857?0.7log2?0.2?0.3log2 0.91430.08570.2＝0.512bit/符号
2.17 每帧电视图像可以认为是由3?105个像素组成的，所有像素均是独立变化，且每像素又取128个不同的亮度电平，并设亮度电平是等概出现，问每帧图像含有多少信息量？若有一个广播员，在约10000个汉字中选出1000个汉字来口述此电视图像，试问广播员描述此图像所广播的信息量是多少（假设汉字字汇是等概率分布，并彼此无依赖）？若要恰当的描述此图像，广播员在口述中至少需要多少汉字？ 解：1)
H(X)?log2n?log2128?7 bit/symbolN56H(X)?NH(X)?3?10?7?2.1?10 bit/symbol
2)
H(X)?log2n?log210000?13.288 bit/symbolNH(X)?NH(X)?1000?13.288?13288 bit/symbolH(XN)2.1?1063)N? ??158037H(X)13.2882.20 给定语音信号样值X的概率密度为p(x)??e??x,???x???,求Hc(X)，并证明它小于同样方差的正态变量的连续熵。 解
1??xHc(X)???px(x)logpx(x)dx???px(x)log?edx2????1???px(x)log?dx??px(x)(??x)logedx2????11??x??log?loge??e(?x)dx22??11??log??loge??e?x??(?x)dx?log22??11??log??2loge??2xe??xdx2201??x?????log??loge?(1??x)e??0212e??log??loge?log2?E(X)?0,D(X)?2??0????????????12?01??x?e(?x)dx 2?2
1214?e2?e2e?eH(X,)?log2?e2?log2?log?log?H(X)
2?2????122.24 连续随机变量X和Y的联合概率密度为：p(x,y)????r??0x2?y2?r2其他，求H(X), H(Y), H(XYZ)和I(X;Y)。 （提示：?02log2sinxdx??log22）
2??解：
p(x)??r2?x2?r2?x2rp(xy)dy??r2?x2?12r2?x2dy? (?r?x?r)2r2?x2?r2?rHc(X)???p(x)logp(x)dx?rr2r2?x2 ???p(x)logdx2?r?rrr2 ???p(x)log2dx??p(x)logr2?x2dx?r?r?rr?r2 ?log??p(x)logr2?x2dx?r2?r21 ?log?logr?1?log2e221 ?log2?r?log2e bit/symbol2其中：?r?rp(x)logr2?x2dxr2r2?x222??logr?xdx2?r?r4r2?2?r?x2logr2?x2dx?r040令x?rcos?2??rsin?logrsin?d(rcos?)?r240??2??r2sin2?logrsin?d??r2???????444?20sin2?logrsin?d?sin?logrd???202?20??4?20sin2?logsin?d???logr?1?cos2?41?cos2?d???2logsin?d?2?02
?2??logr?2d??02???0logr?2cos2?d??0??2?20logsin?d????2?20cos2?logsin?d??logr?1?logr?2dsin2??2?(??2log22)???2?20cos2?logsin?d??logr?1???2?20cos2?logsin?d?1?logr?1?log2e2其中：???2?20cos2?logsin?d??20??1logsin?dsin2??201???sin2?logsin??????????1????2sin2?dlogsin???0???2?202sin?cos??cos?log2ed?sin??2log2e?2cos2?d?01?cos2?d?0?2??11??log2e?2d??log2e?2cos2?d?log2e?20?
??011??log2e?log2esin2?22?1??log2e2?20p(y)??r2?y2?r2?y2p(xy)dx??r2?y2?2r2?y21dx? (?r?y?r)r2?y2?r2?r2p(y)?p(x)1HC(Y)?HC(X)?log2?r?log2e bit/symbol2Hc(XY)????p(xy)logp(xy)dxdyR ????p(xy)logRR1dxdy2?r
?log?r2??p(xy)dxdy ?log2?r2 bit/symbolIc(X;Y)?Hc(X)?Hc(Y)?Hc(XY) ?2log2?r?log2e?log?r2 ?log2??log2e bit/symbol
2.25 某一无记忆信源的符号集为{0, 1}，已知P(0) = 1/4，P(1) = 3/4。 (1) 求符号的平均熵；
(2) 有100个符号构成的序列，求某一特定序列（例如有m个“0”和（100 - m）个“1”）的自信息量的表达式； (3) 计算(2)中序列的熵。 解：
?(1)H(X)???p(xi)logp(xi)??? bit/symbol ?log?log??0.811i1?4143434??1??3?p(xi)???????4??4?(2)
m100?m3100?m?10043?41.5?1.585m bit1004100?m
I(xi)??logp(xi)??log(3) H(X100)?100H(X)?100?0.811?81.1 bit/symbol 2-26
P(i)= P(ij)=
H(IJ)=
2.29 有一个一阶平稳马尔可夫链X1,X2,?,Xr,?,各Xr取值于集合A??a1,a2,a3?,已知起始概率P(Xr)为p1?1/2,p2?p3?1/4，转移概率如下图所示
j i 1 2 1/2 2/3 1/4 0 1/4 1/3 1 2 3