Rényi Entropy and Free Energy


I want to keep telling you about information geometry… but I got sidetracked into thinking about something slightly different, thanks to some fascinating discussions here at the CQT.

There are a lot of people interested in entropy here, so some of us ? Oscar Dahlsten, Mile Gu, Elisabeth Rieper, Wonmin Son and me ? decided to start meeting more or less regularly. I call it the Entropy Club. I’m learning a lot of wonderful things, and I hope to tell you about them someday. But for now, here’s a little idea I came up with, triggered by our conversations:

? John Baez, Rnyi entropy and free energy.

In 1960, Alfred Rnyi defined a generalization of the usual Shannon entropy that depends on a parameter. If $latex p$ is a probability distribution on a finite set, its Rnyi entropy of order $latex beta$ is defined to be

View original post 685 more words

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s