0.70.3黑0.3白0.7P(白|白)＝P(白)
P(黑|黑)＝P(黑) P(白|黑)＝P(白)
（2）根据题意，此一阶马尔可夫链是平稳的（P(白)＝0.7不随时间变化，P(黑)＝0.3不随时 间变化）
H?(X)?H(X2|X1)??p(xi,yj)log2ij1p(xi,yj)?0.9143?0.7log2?0.8?0.3log210.8111?0.0857?0.7log2?0.2?0.3log2
0.91430.08570.2＝0.512bit/符号
5
2.17 每帧电视图像可以认为是由3?10个像素组成的，所有像素均是独立变化，且每像素又取128个不同的亮度电平，并设亮度电平是等概出现，问每帧图像含有多少信息量？若有一个广播员，在约10000个汉字中选出1000个汉字来口述此电视图像，试问广播员描述此图像所广播的信息量是多少（假设汉字字汇是等概率分布，并彼此无依赖）？若要恰当的描述此图像，广播员在口述中至少需要多少汉字？ 解： 1)
H(X)?log2n?log2128?7 bit/symbolN56H(X)?NH(X)?3?10?7?2.1?10 bit/symbol 2)
H(X)?log2n?log210000?13.288 bit/symbolNH(X)?NH(X)?1000?13.288?13288 bit/symbol 3)
H(XN)2.1?106 N???158037H(X)13.288
2.20 给定语音信号样值X的概率密度为p(x)?小于同样方差的正态变量的连续熵。 解：
1??x?e,???x???,求Hc(X)，并证明它2
????Hc(X)???px(x)logpx(x)dx?????px(x)log1?e??xdx??2???????px(x)log1?dx??px(x)(??x)log??2edx??????log12?loge1??x???2?e(?x)dx0????log12??loge?1?e?x??(?x)dx?log??2?12?e??x(?x)dx 0????log1??2loge?1?2xe??x2dx02??log12??loge??(1??x)e??x????0??log12e2??loge?log?E(X)?0,D(X)?2?2
H(X,)?1214?e2?e2e?e2log2?e?2?2log?2?log??log??H(X)
?2.24 连续随机变量X和Y的联合概率密度为：p(x,y)??1??r2??0求H(X), H(Y), H(XYZ)和I(X;Y)。 ?（提示：?2log?02sinxdx??2log22）
解：
x2?y2?r2，
其他
p(x)??r2?x2?r2?x2rp(xy)dy??r2?x2?12r2?x2dy? (?r?x?r)2r2?x2?r2?rHc(X)???p(x)logp(x)dx?rr2r2?x2 ???p(x)logdx2?r?rrr2 ???p(x)log2dx??p(x)logr2?x2dx?r?r?rr?r2 ?log??p(x)logr2?x2dx?r2?r21 ?log?logr?1?log2e221 ?log2?r?log2e bit/symbol2其中：?r?rp(x)logr2?x2dxr2r2?x2??logr2?x2dx2?r?r4r2?2?r?x2logr2?x2dx?r040令x?rcos?2??rsin?logrsin?d(rcos?)?r240??2??r2sin2?logrsin?d??r2??4??4??20sin2?logrsin?d?sin?logrd???2?204??4??20sin2?logsin?d??1?cos2?41?cos2??logr?2d???2logsin?d?0?2?02
???20?2?logr?2d??02??0logr?2cos2?d??0??2logsin?d????2?20cos2?logsin?d??logr?1?logr?2dsin2??2?(??2log22)???2?20cos2?logsin?d??logr?1???2?20cos2?logsin?d?1?logr?1?log2e2其中：???2?20cos2?logsin?d??20??1logsin?dsin2??201???sin2?logsin??????????1????2sin2?dlogsin???0???2?202sin?cos??cos?log2ed?sin??2log2e?2cos2?d?01?cos2?d?0?2??112??log2e?d??log2e?2cos2?d?log2e?20?
??011??log2e?log2esin2?22?1??log2e2?20
p(y)??r2?y2?r2?y2p(xy)dx??r2?y2?2r2?y21dx? (?r?y?r)r2?y2?r2?r2p(y)?p(x)1HC(Y)?HC(X)?log2?r?log2e bit/symbol2Hc(XY)????p(xy)logp(xy)dxdyR ????p(xy)logRR1dxdy?r2
?log?r2??p(xy)dxdy ?log2?r2 bit/symbolIc(X;Y)?Hc(X)?Hc(Y)?Hc(XY) ?2log2?r?log2e?log?r2 ?log2??log2e bit/symbol
2.25 某一无记忆信源的符号集为{0, 1}，已知P(0) = 1/4，P(1) = 3/4。 (1) 求符号的平均熵；
(2) 有100个符号构成的序列，求某一特定序列（例如有m个“0”和（100 - m）个“1”）的自信息量的表达式； (3) 计算(2)中序列的熵。
解： (1)
133??1H(X)???p(xi)logp(xi)???log?log??0.811 bit/symbol
444??4i(2)
m100?m?1??3?p(xi)???????4??4?3100?m?10043?41.5?1.585m bit1004100?m
I(xi)??logp(xi)??log(3)
H(X100)?100H(X)?100?0.811?81.1 bit/symbol
2-26