A Boltzmann machine is a type of stochastic recurrent neural network that can learn a probability distribution over its set of inputs. It is particularly used for modeling complex distributions and solving combinatorial optimization problems.<\/p>\n\n\n\n
A Boltzmann machine is defined by an energy function \\( E(\\mathbf{v}, \\mathbf{h}) \\) which assigns a scalar energy to each configuration of visible units \\( \\mathbf{v} \\) and hidden units \\( \\mathbf{h} \\). The energy function is given by:<\/p>\n\n\n\n
$$ E(\\mathbf{v}, \\mathbf{h}) = -\\sum_{i} \\sum_{j} W_{ij} v_i h_j – \\sum_{i} b_i v_i – \\sum_{j} c_j h_j $$<\/p>\n\n\n\n
where:<\/p>\n\n\n\n
The joint probability distribution over the visible and hidden units is defined by the Boltzmann distribution:<\/p>\n\n\n\n
$$ P(\\mathbf{v}, \\mathbf{h}) = \frac{1}{Z} \\exp(-E(\\mathbf{v}, \\mathbf{h})) $$<\/p>\n\n\n\n
where \\( Z \\) is the partition function, given by:<\/p>\n\n\n\n
$$ Z = \\sum_{\\mathbf{v}} \\sum_{\\mathbf{h}} \\exp(-E(\\mathbf{v}, \\mathbf{h})) $$<\/p>\n\n\n\n
The probability of a visible vector \\( \\mathbf{v} \\) is obtained by marginalizing over the hidden units:<\/p>\n\n\n\n
$$ P(\\mathbf{v}) = \frac{1}{Z} \\sum_{\\mathbf{h}} \\exp(-E(\\mathbf{v}, \\mathbf{h})) $$<\/p>\n\n\n\n
The training objective for a Boltzmann machine is to maximize the likelihood of the observed data. This can be achieved by minimizing the negative log-likelihood:<\/p>\n\n\n\n
$$ \\mathcal{L} = -\\sum_{\\mathbf{v}} P_{ ext{data}}(\\mathbf{v}) \\log P(\\mathbf{v}) $$<\/p>\n\n\n\n
The gradient of the log-likelihood with respect to a weight \\( W_{ij} \\) is given by:<\/p>\n\n\n\n
$$ \frac{\\partial \\log P(\\mathbf{v})}{\\partial W_{ij}} = \\langle v_i h_j
angle_{ ext{data}} – \\langle v_i h_j
angle_{ ext{model}} $$<\/p>\n\n\n\n
where:<\/p>\n\n\n\n
In practice, exact computation of the gradient is intractable. Contrastive Divergence (CD) is a common approximation method used to train Boltzmann machines. The CD algorithm involves the following steps:<\/p>\n\n\n\n
$$ \\Delta W_{ij} = \\epsilon (\\langle v_i h_j
angle_{ ext{data}} – \\langle v_i h_j
angle_{ ext{model}}) $$<\/p>\n\n\n\n
where \\( \\epsilon \\) is the learning rate.<\/p>\n\n\n\n
A Restricted Boltzmann Machine (RBM) is a special type of Boltzmann machine where the visible and hidden units form a bipartite graph. This restriction simplifies the training process and makes RBMs useful for practical applications.<\/p>\n\n\n\n
The energy function for an RBM is given by:<\/p>\n\n\n\n
$$ E(\\mathbf{v}, \\mathbf{h}) = -\\sum_{i} \\sum_{j} W_{ij} v_i h_j – \\sum_{i} b_i v_i – \\sum_{j} c_j h_j $$<\/p>\n\n\n\n
The conditional probabilities for the hidden and visible units are:<\/p>\n\n\n\n
$$ P(h_j = 1 | \\mathbf{v}) = \\sigma \\left( \\sum_{i} W_{ij} v_i + c_j
\\right) $$<\/p>\n\n\n\n
$$ P(v_i = 1 | \\mathbf{h}) = \\sigma \\left( \\sum_{j} W_{ij} h_j + b_i
\\right) $$<\/p>\n\n\n\n
where \\( \\sigma(x) \\) is the sigmoid function.<\/p>\n\n\n\n
Boltzmann machines and RBMs are used in various applications, including:<\/p>\n\n\n\n
Boltzmann machines provide a powerful framework for modeling complex probability distributions and solving optimization problems. Restricted Boltzmann Machines (RBMs) are particularly practical and have been successfully applied in various real-world applications.<\/p>\n","protected":false},"excerpt":{"rendered":"
A Boltzmann machine is a type of stochastic recurrent neural network that can learn a probability distribution<\/p>\n","protected":false},"author":1,"featured_media":293,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[],"class_list":["post-290","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-probability"],"_links":{"self":[{"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/posts\/290","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=290"}],"version-history":[{"count":3,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/posts\/290\/revisions"}],"predecessor-version":[{"id":486,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/posts\/290\/revisions\/486"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/media\/293"}],"wp:attachment":[{"href":"https:\/\/mp-superkler.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=290"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=290"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=290"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}