On the equivalence between Stein identity and de Bruijn identity

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper illustrates the equivalence between two fundamental results: Stein identity, originally proposed in the statistical estimation realm, and de Bruijn identity, considered for the first time in the information theory field. Two distinctive extensions of de Bruijn identity are presented as well. For arbitrary but fixed input and noise distributions, the first-order derivative of differential entropy is expressed by means of a function of the posterior mean, while the second-order derivative of differential entropy is manifested in terms of a function of Fisher information. Several applications exemplify the utility of the proposed results.

Original languageEnglish
Title of host publication2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012
Pages145-149
Number of pages5
DOIs
Publication statusPublished - 2012
Externally publishedYes
Event2012 IEEE International Symposium on Information Theory, ISIT 2012 - Cambridge, MA, United States
Duration: 1 Jul 20126 Jul 2012

Other

Other2012 IEEE International Symposium on Information Theory, ISIT 2012
CountryUnited States
CityCambridge, MA
Period1/7/126/7/12

Fingerprint

Stein's Identity
Entropy
Equivalence
Derivatives
Posterior Mean
Statistical Estimation
Second-order Derivatives
Fisher Information
Information theory
Information Theory
First-order
Derivative
Arbitrary

ASJC Scopus subject areas

  • Applied Mathematics
  • Modelling and Simulation
  • Theoretical Computer Science
  • Information Systems

Cite this

Park, S., Serpedin, E., & Qaraqe, K. (2012). On the equivalence between Stein identity and de Bruijn identity. In 2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012 (pp. 145-149). [6283505] https://doi.org/10.1109/ISIT.2012.6283505

On the equivalence between Stein identity and de Bruijn identity. / Park, Sangwoo; Serpedin, Erchin; Qaraqe, Khalid.

2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012. 2012. p. 145-149 6283505.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Park, S, Serpedin, E & Qaraqe, K 2012, On the equivalence between Stein identity and de Bruijn identity. in 2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012., 6283505, pp. 145-149, 2012 IEEE International Symposium on Information Theory, ISIT 2012, Cambridge, MA, United States, 1/7/12. https://doi.org/10.1109/ISIT.2012.6283505
Park S, Serpedin E, Qaraqe K. On the equivalence between Stein identity and de Bruijn identity. In 2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012. 2012. p. 145-149. 6283505 https://doi.org/10.1109/ISIT.2012.6283505
Park, Sangwoo ; Serpedin, Erchin ; Qaraqe, Khalid. / On the equivalence between Stein identity and de Bruijn identity. 2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012. 2012. pp. 145-149
@inproceedings{a433380a37c84bcba00e04aa35e3afb5,
title = "On the equivalence between Stein identity and de Bruijn identity",
abstract = "This paper illustrates the equivalence between two fundamental results: Stein identity, originally proposed in the statistical estimation realm, and de Bruijn identity, considered for the first time in the information theory field. Two distinctive extensions of de Bruijn identity are presented as well. For arbitrary but fixed input and noise distributions, the first-order derivative of differential entropy is expressed by means of a function of the posterior mean, while the second-order derivative of differential entropy is manifested in terms of a function of Fisher information. Several applications exemplify the utility of the proposed results.",
author = "Sangwoo Park and Erchin Serpedin and Khalid Qaraqe",
year = "2012",
doi = "10.1109/ISIT.2012.6283505",
language = "English",
isbn = "9781467325790",
pages = "145--149",
booktitle = "2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012",

}

TY - GEN

T1 - On the equivalence between Stein identity and de Bruijn identity

AU - Park, Sangwoo

AU - Serpedin, Erchin

AU - Qaraqe, Khalid

PY - 2012

Y1 - 2012

N2 - This paper illustrates the equivalence between two fundamental results: Stein identity, originally proposed in the statistical estimation realm, and de Bruijn identity, considered for the first time in the information theory field. Two distinctive extensions of de Bruijn identity are presented as well. For arbitrary but fixed input and noise distributions, the first-order derivative of differential entropy is expressed by means of a function of the posterior mean, while the second-order derivative of differential entropy is manifested in terms of a function of Fisher information. Several applications exemplify the utility of the proposed results.

AB - This paper illustrates the equivalence between two fundamental results: Stein identity, originally proposed in the statistical estimation realm, and de Bruijn identity, considered for the first time in the information theory field. Two distinctive extensions of de Bruijn identity are presented as well. For arbitrary but fixed input and noise distributions, the first-order derivative of differential entropy is expressed by means of a function of the posterior mean, while the second-order derivative of differential entropy is manifested in terms of a function of Fisher information. Several applications exemplify the utility of the proposed results.

UR - http://www.scopus.com/inward/record.url?scp=84867521684&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867521684&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2012.6283505

DO - 10.1109/ISIT.2012.6283505

M3 - Conference contribution

SN - 9781467325790

SP - 145

EP - 149

BT - 2012 IEEE International Symposium on Information Theory Proceedings, ISIT 2012

ER -