Introduction
Boltzmann machine is based on the stochastic spin-glass model with the external field, that is a Sherrington-Kirkpatrick model. Sherrington-Kirkpatrick model is a stochastic Ising Model. It is applied to Machine learning and Deep learning. A spin glass model is a magnetic system with random and competing interaction. It is used to get an insight about the physics of the disordered systems in general.
Maxwell-Boltzmann distribution equation which defines the distribution of gas at a certain temperature says that, at room temperature, gases are more likely to spread out evenly across room as this configuration maximizes the total energy of the system that they are part of. The main idea or goal of any energy based data model is to the find the best model configuration that reduces the energy of the system they represents.
What is a Boltzmann Machine?
A Boltzmann Machine is a network of symmetrically connected neuron like units that make stochastic decisions about whether or not to be ON or OFF. It is an unsupervised learning where we provide an input to a machine and let it decide the features it has and the inter-relationship between the data's. It lets the machine figure out the relations and the features that are binding the input data. Here, we don't give the output. This model understands the distribution of the input data and recreates the data based on the distribution.
figure 1: Boltzmann machine
It is an undirected model where connection goes both ways. It is used to optimize the solution; for example, Travelling salesman problem. It is usually used to provide us recommendation or a prediction.
A Boltzmann machine has two layers:
a. Visible Layer : V1, V2, V3.
b. Hidden Layer: H1, H2, H3, H4.
The visible layers are the input data and the hidden layers are usually the neurons. All the nodes are interconnected.
There were many connections even if there were few nodes, therefore researchers came up with Restricted Boltzmann Machine to resolve this problem.
Restricted Boltzmann Machine
In a Restricted Boltzmann Machine the nodes are not interconnected inside their respective layer, that is in hidden layer and visible layer.
figure 2: Restricted Boltzmann Machine.