Month: June 2014

Rényi Entropy and Free Energy


I want to keep telling you about information geometry… but I got sidetracked into thinking about something slightly different, thanks to some fascinating discussions here at the CQT.

There are a lot of people interested in entropy here, so some of us ? Oscar Dahlsten, Mile Gu, Elisabeth Rieper, Wonmin Son and me ? decided to start meeting more or less regularly. I call it the Entropy Club. I’m learning a lot of wonderful things, and I hope to tell you about them someday. But for now, here’s a little idea I came up with, triggered by our conversations:

? John Baez, Rnyi entropy and free energy.

In 1960, Alfred Rnyi defined a generalization of the usual Shannon entropy that depends on a parameter. If $latex p$ is a probability distribution on a finite set, its Rnyi entropy of order $latex beta$ is defined to be

View original post 685 more words