Shannon Memorial Lecture: Shannon Source Coding, Ornstein Isomorphism and Random Process Models

Guest Speaker: Robert M. Gray, Stanford University Lucent Techs Prof in Communications and Networking Date: April 21st, 2009 Time: 4:00pm Location: Calit2 Auditorium, Atkinson Hall, UC San Diego [webcast] Host: Jack Wolf, CMRR

DESCRIPTION/ABSTRACT: This is the seventh annual Shannon Memorial Lecture organized by Jack Wolf of UCSD's Center for Magnetic Recording Research. The series is named after Claude Shannon (pictured), the Bell Labs mathematician and father of information theory. Abstract: The enormous impact of Shannon information theory on modern digital communications is widely known and appreciated. Less well known in the engineering community is the impact of Shannon's ideas of entropy and coding on ergodic theory, the theory of measure preserving (and related) transformations on probability spaces. The second most important result in ergodic theory (after the ergodic theorem or law of large numbers) is the Ornstein isomorphism theorem, which shows that Shannon entropy characterizes which random processes are really the "same" in the sense that one can be coded or filtered into the other in an invertible time-invariant way. This talk will survey a few of the many connections between information theory and ergodic theory, old and recent, with an emphasis on the particular example of Ornstein's B-processes – processes which can be modeled as time-invariant (or stationary) codings (or filterings) of independent, identically distributed processes. Also emphasized is the process version of the Monge/Kantorovich optimal transportation distance, which quantifies the goodness of approximation of two random processes that is useful in information theory, ergodic theory, and modeling discrete-time random processes. SPEAKER BIO: Robert M. Gray is the Lucent Technologies Professor of Engineering and Professor of Electrical Engineering at Stanford University, where he has been teaching for almost 40 years. His research interests are in information theory and signal processing, especially in the theory and practice of quantization, compression, and classification. Gray was a member of the Board of Governors of the IEEE Information Theory Group (1974-1980, 1985-1988) and of the IEEE Signal Processing Society (1998-2001). He was Associate Editor for Source Coding (1977-1980) and Editor-in-Chief (1980-1983) of the IEEE Transactions on Information Theory. He is currently the Editor-in-Chief of Foundations and Trends in Signal Processing. He was Co-Chair of the 1993 IEEE International Symposium on Information Theory and Program Co-Chair of the 1997 and 2004 IEEE International Conference on Image Processing. He is a Fellow of the Institute of Mathematical Statistics and the IEEE and has held fellowships from the Japan Society for the Promotion of Science at the University of Osaka (1981), the John Simon Guggenheim Foundation at the University of Paris XI (1982), and NATO/Consiglio Nazionale delle Ricerche at the University of Naples (1990). During spring 1995 he was a Vinton Hayes Visiting Scholar at the Division of Applied Sciences of Harvard University. He is a Faculty Affiliate of the Clayman Institute for Gender Studies at Stanford University, where he will be a Faculty Research Fellow during the 2008-2009 academic year. He was corecipient of the 1976 IEEE Information Theory Group Paper Award and the 1983 IEEE ASSP Senior Award. He was awarded an IEEE Centennial medal (1984) and an IEEE Third Millennium Medal (2000). He received the 1993 Society Award, the 1998 Technical Achievement Award, and the 2005 Meritorious Service Award from the IEEE Signal Processing Society. He received a Golden Jubilee Award for Technological Innovation (1998) and the 2008 Shannon Award from the IEEE Information Theory Society. He received the 2008 IEEE Jack S. Kilby Signal Processing Medal. He received a 2002 Presidential Award for Excellence in Science, Mathematics and Engineering Mentoring (PAESMEM) and the 2003 Distinguished Alumni in Academia Award from the University of Southern California. He is a member of the National Academy of Engineering (2007). Robert M. Gray was born in San Diego, Calif., on November 1, 1943. He received the B.S. and M.S. degrees from M.I.T. in 1966 and the Ph.D. degree from U.S.C. in 1969, all in Electrical Engineering.