infostructuralist.wordpress.com
April | 2015 | The Information Structuralist
https://infostructuralist.wordpress.com/2015/04
Skip to search - Accesskey = s. Counting bits with Vapnik and Chervonenkis. Posted in Information Theory. Statistical Learning and Inference. By mraginsky on April 28, 2015. Machine learning is about enabling computers to improve their performance on a given task as they get more data. Can we express this intuition quantitatively using information-theoretic techniques? In this post, I will discuss a classic paper. Models of Complex Stochastic Systems. Statistical Learning and Inference.
infostructuralist.wordpress.com
Information flow on graphs | The Information Structuralist
https://infostructuralist.wordpress.com/2014/05/03/information-flow-on-graphs
Skip to search - Accesskey = s. Information flow on graphs. Posted in Information Theory. Models of Complex Stochastic Systems. By mraginsky on May 3, 2014. In a very nice paper. Which deserves to be more widely known. The system Dawson analyzed is an instance of a Probabilistic Cellular Automaton. Is determined stochastically by its own state, as well as by the states of its neighbors, at time. Now let’s move on to precise definitions. Consider an undirected graph. Is the set of all vertices. We allow a...
infostructuralist.wordpress.com
Lossless source coding at Western Union | The Information Structuralist
https://infostructuralist.wordpress.com/2013/09/19/lossless-source-coding-at-western-union
Skip to search - Accesskey = s. Lossless source coding at Western Union. Posted in Information Theory. By mraginsky on September 19, 2013. 8230; circa 1935. Here is a quote from. An American travelogue by Ilya Ilf and Yevgeny Petrov. Typical set encoding, anyone? Laquo; ISIT 2013: two plenaries on concentration of measure. A graph-theoretic derivation of the Gilbert-Varshamov bound. Leave a Reply Cancel reply. Enter your comment here. Fill in your details below or click an icon to log in:.
infostructuralist.wordpress.com
Counting bits with Vapnik and Chervonenkis | The Information Structuralist
https://infostructuralist.wordpress.com/2015/04/28/counting-bits-with-vapnik-and-chervonenkis
Skip to search - Accesskey = s. Counting bits with Vapnik and Chervonenkis. Posted in Information Theory. Statistical Learning and Inference. By mraginsky on April 28, 2015. Machine learning is about enabling computers to improve their performance on a given task as they get more data. Can we express this intuition quantitatively using information-theoretic techniques? In this post, I will discuss a classic paper. We will stick to the setting of binary classification: we have a feature space. Is a sequen...
infostructuralist.wordpress.com
ISIT 2013: two plenaries on concentration of measure | The Information Structuralist
https://infostructuralist.wordpress.com/2013/07/29/isit-2013-two-plenaries-on-concentration-of-measure
Skip to search - Accesskey = s. ISIT 2013: two plenaries on concentration of measure. Posted in Conference Blogging. By mraginsky on July 29, 2013. Of the five plenary talks at this year’s ISIT. Two were about concentration of measure: Katalin Marton’s Shannon lecture on “Distance-divergence inequalities” and Gabor Lugosi’s talk on “Concentration inequalities and the entropy method” the next morning. Since the topic of measure concentration is dear to my heart. Then we say that. Another way to express (1).
infostructuralist.wordpress.com
Conditional mutual information and the best Markov approximation | The Information Structuralist
https://infostructuralist.wordpress.com/2013/01/04/conditional-mutual-information-and-the-best-markov-approximation
Skip to search - Accesskey = s. Conditional mutual information and the best Markov approximation. Posted in Information Theory. By mraginsky on January 4, 2013. I came across a neat and useful result about conditional mutual information while reading a paper on quantum information theory. Consider three jointly distributed random variables. If the conditional mutual information. Are conditionally independent given. Is a Markov chain). In fact,. If and only if. Now, what if. Is nonzero, but very small?
infostructuralist.wordpress.com
Two public service announcements | The Information Structuralist
https://infostructuralist.wordpress.com/2014/02/25/two-public-service-announcements
Skip to search - Accesskey = s. Two public service announcements. Posted in Information Theory. By mraginsky on February 25, 2014. 1 The 2014 IEEE North American Summer School on Information Theory. Will take place June 18-21, 2014 at the Fields Institute in Toronto, Canada. 2 For those of you who use Matlab or Octave, there is a new Information Theoretical Estimators (ITE) toolbox. By the toolbox’s creator, Zoltán Zsabó. At the Princeton Information Theory b-log. Information flow on graphs. Enter your e...
infostructuralist.wordpress.com
A graph-theoretic derivation of the Gilbert-Varshamov bound | The Information Structuralist
https://infostructuralist.wordpress.com/2013/09/23/a-graph-theoretic-derivation-of-the-gilbert-varshamov-bound
Skip to search - Accesskey = s. A graph-theoretic derivation of the Gilbert-Varshamov bound. Posted in Coding Theory. By mraginsky on September 23, 2013. Just a quick note for my reference, but it may be of interest to others. Denote the size of the largest code over a. Ary alphabet that has blocklength. The well-known Gilbert-Varshamov bound. Is the volume of a Hamming ball of radius. The usual way of arriving at the GV bound is through a greedy construction: pick an arbitrary codeword. If no two vertic...
infostructuralist.wordpress.com
It’s for a good cause! | The Information Structuralist
https://infostructuralist.wordpress.com/2013/05/02/its-for-a-good-cause
Skip to search - Accesskey = s. It’s for a good cause! Posted in Information Theory. By mraginsky on May 2, 2013. To honor Claude Elwood Shannon with a United States Postal Service stamp on the 100th anniversary of his birth. Laquo; Stochastic kernels vs. conditional probability distributions. ISIT 2013: two plenaries on concentration of measure. Leave a Reply Cancel reply. Enter your comment here. Fill in your details below or click an icon to log in:. Address never made public). Follow Blog via Email.
SOCIAL ENGAGEMENT