Deep Belief Networks Can Represent Every Strictly Positive Distribution Exactly

Summary

A paper by Gleb Smirnov proves that every strictly positive probability distribution over the binary space {-1,1}^n can be represented exactly by a sigmoid belief network using finite parameters. The result resolves a question previously posed by Sutskever and Hinton. Earlier probability-sharing arguments established approximation, but not exact representation. The proof strengthens that result by applying Brouwer's fixed-point theorem to obtain an exact representation. The six-page theoretical paper is categorized under artificial intelligence, machine learning, and probability.