?103 103 103 103 103 103 103 103
+ 8 log 8 + 15 log 15 +
?5
log 5 + 12 log 12 ? 103 103
103 103
103?
103 103 103
= 2.836 bit symbol/ H YZ(
) = ?∑∑ p y z( j
k
)log p y z( j k ) j k
= ??? 20 log 20 + 23 log 23 + 32 log 32 + 28 log 28 ?? ?103 103 103 103 103 103 103 103? 1.977 = bit symbol/
H X YZ( / ) = H XYZ( ) ?H YZ(
(3)
) = 2.836?1.977 = 0.859 bit symbol/
I X YZ( ; ) = H X( ) ?H X YZ( / ) = 0.964?0.859 = 0.159 bit symbol/ 2.15 有两个二元随机变量X和Y，它们的联合概率为
Y X x1 =0 1 /8 3 /8 x 2 =1 3 /8 1 /8 y 1 =0 y =1 2 并定义另一随机变量Z = XY（一般乘积），试计算：
(1) H(X), H(Y), H(Z), H(XZ), H(YZ)和H(XYZ)；
(2) H(X/Y), H(Y/X), H(X/Z), H(Z/X), H(Y/Z), H(Z/Y), H(X/YZ), H(Y/XZ)和H(Z/XY)；
(3) I(X;Y), I(X;Z), I(Y;Z), I(X;Y/Z), I(Y;Z/X)和I(X;Z/Y)。
解： (1)
p x
p x y p x y p x
·11·
p x y
i
p x y
H X() =?∑ p x( i )log p x( i ) =1 bit symbol/
p y
p x y p x y p y
p x y
j
p x y
H Y( ) =?∑ p y( j )log p y( j ) =1 bit symbol/
Z = XY 的概率分布如下：
?Z ? ??z1 = 0 ??P Z( )??=??? 78 z2 =1?? 18 ???
?7711?
H Z( ) =?∑k2 p z( k ) =???8 log 8+ 8log8??= 0.544 bit symbol/
·12·
p x( 1) = p x z( 1 1)+ p x z( 1 2) p x z( 1 2) = 0 p x z( 1
1
) = p x( 1) = 0.5 p z( 1) =
p x z( 1 1)+ p x z( 2 1)
p z( 2) = p x z( 1 2)+
p x z( 2 2)
H XZ( ) =?
∑∑
i
k
?1133 11?
p x z( ik )log p x z( ik ) =??2log 2 + 8log8+ 8log8??=1.406 bit symbol/
?
·13·
p y( 1) = p y z( 1 1)+ p y z( 1 2) p y z( 1 2) = 0 p y z( 1 1) = p y( 1) = 0.5 p z( 1) = p y z( 1 1)+ p y z( 2 1)
p z( 2) = p y z( 1 2)+ p y z( 2 2)
H YZ(
?1133
) =?k p y z( j k )log p y z( j k ) =??2log 2 + 8log8+
?1
8log18??=1.406 bit symbol/ j ?
p x y z( 1 1 2) = 0 p x y z( 1 2 2) =
∑∑
0 p x y z( 2 1 2) = 0 p x y z( 1 1 1)+ p x y z( 1 1 2) = p x y( 1 1) p x y z( 1 1 1) = p x y( 1 1) =1/8 p x y z( 1 2 1)+ p x y z( 1 1 1) = p x z( 1 1)
p x y z( 2 1 1)+ p x y z( 2 1 2) = p x y( 2 1)
·14·
p x y z( 2 2 1) = 0
p x y z( 2 2 1)+ p x y z( 2 2 2) = p x y( 2 2)
H XYZ( ) =?∑∑∑ p x y z( i
i
j
k
jk
)log2 p x y z( i j
k
)
?11 33 33 11?=??log+ log+ log+ log?=1.811 bit symbol/ ?8 8 8 8 8 8 8 8?
(2)
H XY( ) =?
∑∑
i
j
?1133 33 11?
p x y( ij )log2 p x y( ij ) ==??8log8 + 8log8+ 8log8+ 8log8??=1.811 bit symbol/
?
) = H XY( )?H Y( ) =1.811 1? = 0.811 bit symbol/
H X Y( H Y X( H X Z( H Z X( H Y Z(
/ / / / /
) = H XY(
)?H X( ) =1.811 1? = 0.811 bit symbol/
) = H XZ( )?H Z( ) =1.406?0.544 = 0.862 bit symbol/
) = H XZ( )?H X( ) =1.406? =1 0.406 bit symbol/
) = H YZ(
)?H Z( ) =1.406?0.544 = 0.862 bit symbol/
·15·