ppp?1??p?2?22ppp?1??p?0??pp?1??p?2?
22ppp?2??p?0??p?1??pp?2?22p?0??pp?0??整理得
1p?0??p?1??p?2??
3（2）信源的熵
H???q?Si?H?ak|Si?
Si其中状态极限概率
q?Si???q?Sj?P?Si|Sj?
Sjppp?1??p?2?22ppp?1??p?0??pp?1??p?2?
22ppp?2??p?0??p?1??pp?2?22p?0??pp?0??整理得
1p?0??p?1??p?2??
3所以，此信源的熵
H???q?Si?H?ak|Si?Sip??pp??p?pp??q?0?H?p,,??q?1?H?,p,??q?2?H?,,p?2??22??2?22?
1?1p2p2??3??plog?log?log?3?p2p2p??plog12?plogpp（3）信源无记忆，符号的概率分布等于平稳分布。
1p?0??p?1??p?2??
3所以
H?X???p?ai?logi1p?ai??p?0?log?1.58?bit?111?p?1?log?p?2?log p?0?p?1?p?2?经比较，可得到H??H?X?。
（4）一阶马尔科夫信源的极值
H????1?p?log?1?p??21当且仅当p?,p?时成立。
33当p?0时，H??0；
1??1当p?1时，H????log??2?1?bit?
2??23.16 （1）状态转移矩阵
pppplog?log?log3 2222?pP???p??00ppp?0?? p??由
?p0??p??PT?1???p2???p0??p? ?1???p2??解得
p0?pp0?pp1p1?pp1?pp1 p0?p1?p2?1整理得，平稳后信源的概率分布
p0?p1?p2?（2）求信源熵H?。先求状态极限概率
1 3q?0??q?0??p?q?1??p?q?2??0q?1??q?0??0?q?1??p?q?2??p q?2??q?0??p?q?1??0?q?2??p又因为
?q?S??1
iSi可得
q?0??q?1??q?2??信源的熵
1 31?11?11H???q?Si?H?ak|Si??3??plog?plog??plog?plog
3?pp?ppSi（3）求p?0或p?1时的H?
p?0时
?11?H??lim?plog?plog??lim??p3?loge?0
p?0pp?p?0?p?1时，即p?0
?11?H??lim?plog?plog??lim??p3?loge?0
p?0pp?p?0?信源熵表示的是信源的平均不确定性。此题中p?0和p?1表明某一状态转化到另一状态的情况一定发生或一定不发生，即是确定事件，平均不确定性为零，即信源的熵为零。 3.17 可知消息的极大熵H0?log2?1?bit?，此信源的极限熵为
3311H??H1??log?log?0.811?bit?
4444由冗余度计算公式得
R?1?H?0.811?1??0.189 H013.18 （1）由一步转移概率矩阵与二步转移概率矩阵的公式P2?P?P得
?q2?pqpq?P2??q22pq?q2pq?p2??p2? pq?p2??（2）设平稳状态W??W1,W2,W3?，马尔可夫信源性质知WP?W，即
?qW1?qW2?W1?pW?qW?W?132 ?pW?pW?W33?2??W1?W2?W3?1?q2?W1?1?p?p2??pq?求解得稳态后的概率分布?W2? 21?p?p??p2?W3?1?p?p2??3.19 设状态空间S=?a,b,c?，符号空间X??a,b,c? 且
131P?X2?a|X1?b??P?X2?b|X1?b??P?X2?c|X1?b??3
1P?X2?a|X1?c??P?X2?b|X1?c??2P?X2?c|X1?c??0P?X2?a|X1?a??P?X2?b|X1?a??P?X2?c|X1?a??一步转移概率矩阵
?1?3?1P???3??1??21313121?3??1? 3??0???状态转移图
1/3a1/31/31/31/21/31/2设平稳状态W??W1,W2,W3?，由马尔可夫信源性质有
WP?W
即
11?1W?W??31322W3?W1?11?1?W1?W2?W3?W2
32?31?1?2W1?2W2?W3?可得
3?W??18?3??W2?
8?1??W3?4?马尔可夫链只与前一个符号有关，则有
s11bc
?3?H????p?Si???p?ak|Si?logp?ak|Si??i?1?k?1?3?11?3?11?1?11????3?log???3?log???2?log? 8?33?8?33?4?22??1.435?bit?3.20 消息元的联合概率是
p?白，白??p?白|白??p?白??0.7?0.9143?0.64p?黑，白??p?黑|白??p?白??0.7?0.0857?0.06p?白，黑??p?白|黑??p?黑??0.2?0.3?0.06p?黑，黑??p?黑|黑??p?黑??0.3?0.8?0.243
求得一阶马尔可夫信源的不确定性为
H?X|X???0.64log0.9143?0.06log0.0857?0.06log0.2?0.24log0.8?0.512?bit?
3.21 组成的字如下：
?,??,??,??,??,??,???,???,???,???,???,???,???,???,??? 这种简单语言平均每个码字的长度为
15938?1??2??3? 15151515则平均每个字符携带的信息量为
log15?1.542?bit? 3815所以，其冗余度为
1?1.542?0.027 log33.22 掷n次钱币就使其正面向上的概率为
?1?P?X?n?????2?n?11?1????? 2?2?n则有
1?1?H?X??????n?12?2??n?1?n?1?log????n?2?bit? ?2?n?12n