H Z Y( H X YZ(
/ /
) = H YZ( )?H Y( ) =1.406? =1 0.406 bit symbol/
) =1.811 1.406? = 0.405 bit symbol/
) = H XYZ( )?H YZ(
H Y XZ( / ) = H XYZ( )?H XZ( ) =1.811 1.406? = 0.405 bit symbol/ H Z XY( / ) = H XYZ( )?H XY( ) =1.811 1.811? = 0 bit symbol/
(3)
I X Y( ; ) = H X( symbol/ I X Z( ;
)?H X Y( / ) = ?1 0.811= 0.189 bit
/
) = ?1 0.862 = 0.138 bit symbol/
) = H X()?H X Z(
I Y Z( ;) = H Y( )?H Y Z( / ) = ?1 0.862 = 0.138 bit symbol/
) = 0.862?0.405 =
I X Y Z( ; / ) = H X Z( / )?H X YZ( / 0.457 bit symbol/
I Y Z X( ; / ) = H Y X( / )?H Y XZ( / ) = 0.862?0.405 = 0.457 bit
symbol/ I X Z Y( ; / ) = H X Y( / )?H X YZ( / ) = 0.811?0.405 = 0.406 bit symbol/
2.16 有两个随机变量X和Y，其和为Z = X + Y（一般加法），若X和Y相互独立，求证：H(X)
≤ H(Z), H(Y) ≤ H(Z)。
证明：
?Z = X +Y
∴ p z( k / xi ) = p z( k ?xi ) = ??p y( j ) (zk ?xi )∈Y
?0 (zk ?xi )?Y
H Z X(
/ ) = ?∑∑i
k
p x z( i
?
)log p z( / x) = ?∑i p x( )k k i i ??∑k
?
p z( k / xi )log p z( k / xi )??
? ?
= ?∑ p x( i )?∑ p y( j )log2 p y( j )?= H Y( ) i ?j
?
?H Z( ) ≥ H Z X( / ) ∴H Z( ) ≥ H Y( ) 同理可
得H Z( ) ≥ H X( )。
1 ?λx
2.17 给定声音样值X的概率密度为拉普拉斯分布p x( ) = λe ,?∞ 2 并证 明它小于同样方差的正态变量的连续熵。解： ·16· Hc log= e | |x dx log= 其中： e e xdx Xp x p x dx p x e?λ p x dx p x e | |x dx | |x dx ? e ? e e xdx = e?λx log2 e?λ x 0+∞ ?∫0+∞e?λxdloge?λ ( x )= ????e ?λx0+∞ ???log2 e = log2 e 2 2e ∴Hc (X) = log +log2 e = log bit symbol/ λ λ m? e y ydy e x xdx = 0 ? E X p x xdx e y d y e x xdx e y ydy E x m E x p x x dx e x dx e x dx = ?∫0+∞x de2?λx = ????e?λx x2 0+∞ ?∫0+∞e?λxdx2 ??? = ∫0+∞e?λxdx2 = 2∫0+∞e?λx xdx = ? 2 ∫0+∞xde?λx = ?λ ?∫0+∞e?λxdx??? 2 ???e?λx x 0+∞ λ λ 2 ∴Hc (X正态) = 1 log2πσe 2 = log πe >Hc (X) = log 2e ·17· 2 λ λ ?? 12 x2 + y2 ≤r2 ，求H(X), H(Y), 其他 2.18 连续随机变量X和Y的联合概率密度为：p x y( , ) =?πr ??0 H(XYZ)和I(X;Y)。 （提示： 解： xdx ） 1 ) X p x p x dx 2 r 2 ?x 2 = ∫?r2?x2 πr 2 dy = πr 2 (?r ≤ x ≤ r) r2?x2 r2?x2 p x( ) = ∫?r2?x2 p xy dy( Hc =? ?r px r ∫ 2 r 2 ? x 2 2 r π ( )logdx ( )log dx p x r 2 x dx2 = ? πr log p x r 2 x dx2 log e ·18· log r log e bit symbol / 其中： p x r 2 x dx2 r x r 2 xdx2 πr 0 令x =4 r cos θππ r logsin ) 2 ∫ 2 r sinθ r θdr ( cosθ 4 0 =? r 2 πr 2 sin 2 π ∫ 2θlogsin r θd θ=4 π 2 0 sin 2 θlogsin r θd π∫ θ r 2 x dx2=4 π 2 sin 2 log rd 4 π2 π∫ 0 θθ + sin 2 logsin d ∫0 θ θθ =4π π log∫ r 2 1 ? cos2θ d π 0 2 θ +4 π 2 1 ? cos2 θlogsin π∫ 0 2 θdθ 19· · e 其中： = π1 =? = ?π= ?π 1 ????sin 2 logsinθ 2 0 θ π02 ? ∫π 0 2 sin 2θd logsinθ???? π ∫ 2 sinθ cosθ 2 e cosθ log e d θ sin π θ 2 2 π 0 cos 2 θθd = ?πlog 2 ∫ 2 log 2 e∫0π2 1+ cos22 θdθ 1 log e π2 dθ?1 log 2 e∫0π2 cos2θθd 2 ∫π 0 1 1 π = ? 2 log 2 e ?2πlog 2 esin 2θ 02 e p y( ) y 2 ? r ? 2 2 2 r y ? r2?y2 1 2 r 2 ?y2 = ∫p xy dx( ) = ∫?r2?y2 πr 2 dx = πr 2 (?r ≤ y ≤ r) p y( ) = p x( ) ·20·