I(X;Z/Y)?I(XY;Z)??????p(xyz)logp(z/y)zyxyxp(z/xy)?0???p(xyz)logzp(z/xy)p(z)???zyxp(z/y)p(xyz)logp(z)??zyp(z/y)p(yz)logp(z)p(z/xy)p(xyz)logp(z/x)p(z/y)p(z)?I(Y;Z)I(Y;Z/X)???
???zyxzyx???p(xyz)logp(z/x)p(z)??p(yz)logzyp(z/y)p(z/x)???p(xz)logp(z)p(z)xz
?I(Y;Z)?I(X;Z)I(Y;Z/X)?I(Y;Z)?I(X;Z)?I(Y;Z)(?I(X;Z)?0)
2.18 若三个随机变量有如下关系：x＋y=z，其中x和y独立。试证明：
H(X)≤H (Z) H(Y)≤H(Z) H(XY)≥H(Z) I(X；Z)=H(Z)－H(Y) I(XY；Z)=H(Z) I(X；YZ)=H(X) I(Y；Z|X)=H(Y)
I(X；Y|Z)=H(X|Z)=H(Y|Z)
解: (a) H(X)≤H (Z)
11
I(Y;Z)?H(Z)?H(Z/Y)H(Z/Y)????p(yz)logpz/y(z/y)????p(yz)logpx(z?y)zyzy????pz/y(z/y)p(y)logpx(z?y)????px(z?y)p(y)logpx(z?y)
zyzy???px(x)logpx(x)?H(X)x?I(Y;Z)?H(Z)?H(Z/Y)?H(Z)?H(X)?0即H(X)?H(Z)(b) H(Y)≤H(Z)
同理I(X;Z)?H(Z)?H(Z/X)H(Z/X)?H(Y)?I(X,Z)?H(Z)?H(Z/X)?H(Z)?H(Y)?0
即H(Y)?H(Z)(c) H(XY)≥H(Z)
H(Z/XY)?0I(XY;Z)?H(Z)?H(Z/XY)?H(Z)I(XY;Z)?H(XY)?H(XY/Z)?H(XY)
?H(XY)?H(Z)(d) I(X；Z)=H(Z)－H(Y) I(X;Z)=H(Z)-H(Z/X)=H(Z)-H(Y) (e) I(XY；Z)=H(Z)
H(Z/XY)?0I(XY;Z)?H(Z)?H(Z/XY)?H(Z)
(f) I(X；YZ)=H(X)
H(X/YZ)?0I(X;YZ)?H(X)?H(X/XY)?H(X)
(g) I(Y；Z|X)=H(Y) H(Y/XZ)=0
I(Y;Z/X)=H(Y/X)-H(Y/XZ)=H(Y/X)=H(Y)
(h) I(X；Y|Z)=H(X|Z)=H(Y|Z)
I(X;Y/Z)=H(X/Z)-H(X/YZ)=H(Y/Z)-H(Y/XZ)
12
而 H(X/YZ)=0,H(Y/XZ)=0
所以 I（X；Y/Z）=H(X/Z）=H(Y/Z) #
2.19 证明HK(P)是概率矢量P?(p1,p2,?,pK)的上凸函数，即对?，0
设 P1?(p11,p12,...,p1k),P2?(p21,p22,...,p2k),0???1?P1?(1??)P2?(?p11?(1??)p21,?p21?(1??)p22,...,?p1k?(1??)p2k)H(?P1?(1??)P2)???(?p1i?(1??)p2i)log(?p1i?(1??)p2i)i?1k????p1ilog(?p1i?(1??)p2i)?(1??)?p2ilog(?p1i?(1??)p2i)i?1ki?1kk????p1ilogp1i?(1??)?p2ilogp2ii?1i?1k??H(P1)?(1??)H(P2)不等式中的等号当且仅当P1?P2时成立. #
2.20 用拉格朗日乘因子法求解下述泛函的极值。
Hn(pl, p2, ?, pn)， 解：
?pni?1。
Hn(p1,p2,...,pn)???pilnpii?1令f(p1,p2,...,pn,?)?Hn(p1,p2,...,pn)???pii?1n由?f??lnpi?1???0,i?1,2,...,n,得 pi?e?(1??)?pin1由?pi?1,得 pi?.ni?1?2f1又2???0?pipi111?Hn(p1,p2,...,pn)极大值为Hn(,,...,)?logn#nnn
13
2.22 令U是非负整数集合，事件k∈U的概率为p(k)，且使H(U)为最大的分布p(k)。
解：
?kp(k)?A(常数)。试求
k?0?H(P)???pklnpk,约束条件为k?0? ?pk=1 和?kpk?A k?0k?0??设 f(p0,p1,...,?1,?2)???pklnpk??1?kpk??2?pk,k?0k?0k?0???由?f??lnpk?1??1k??2?0,得 pk?e?1k??2?1?pk??k?0k?0由约束条件?pk?1 和?kpk?A,得?e?{?k?0k?0?1k??2?1?1?A?ke?1k??2?1A1 ,e?2?1=1-e?1=1+A1+A1Ak?pk?(),k?0,1,2,...1+A1+A?H(P)为P的上凸函数,此时H(P)为极大值# 解得 e?1=
2.23 设X是在[－1，1]上为均匀分布的随机变量。试求Hc(X)，Hc(X 2)和Hc(X 3)。
解: (a)
?pX(x)???112,?1?x?10,其它
HC(X)???1log1dx?1比特 22?1(b) 令y?x,2dx1? dy2y 14
?1p?,y?1Y(y)??2y
??0,其它H2??C(X)?????pY(y)logpY(y)dy1???1log1dy02y2y
?1?log2e??0.443比特(c)
令z?x3,dx1?2dz?3z3 pdxZ(z)?pX(x)dz?1?2 ??3??6z,z?1?0,其它H??C(X3)?????pZ(z)logpZ(z)dz01??1z?2223log(6z3)dz??16?1z?23log(6z3)dz06 ?log26?2log2e??0.3比特
2.24．设连续随机变量X和Y的联合概率密度为
?1x2f(xy)???a2?y2b2?1,a?0,b?0??ab?0其它试求Hc(X)，Hc(Y)，Hc(XY)及I(X；Y)。 解：
15