So far, we discussed two supervised deep
learning models, which are the convolutional neural network and the
recurrent neural network. In this video, we will switch to an unsupervised deep
learning model which is the autoencoder. So what are autoencoders? Autoencoding
is a data compression algorithm where the compression and the decompression
functions are learned automatically from data. instead of being engineered by a
human. Such autoencoders are built using neural networks. Autoencoders are data
specific, which means that they will only be able to compress data similar to what
they have been trained on. Therefore, an autoencoder
trained on pictures of cars would do a rather poor job of compressing pictures
of buildings, because the features it would learn would be vehicle or car
specific. Some interesting applications of autoencoders are data denoising and
dimensionality reduction for data visualization. Here is the architecture of
an autoencoder. It takes an image, for example, as an input and uses an encoder
to find the optimal compressed representation of the input image. Then,
using a decoder the original image is restored. So an autoencoder is an
unsupervised neural network model. It uses backpropagation by setting the
target variable to be the same as the input. In other words, it tries to learn
an approximation of an identity function. Because of non-linear activation
functions in neural networks, autoencoders can learn data projections
that are more interesting than a principal component analysis PCA or
other basic techniques, which can handle only linear transformations. A very
popular type of autoencoders is the Restricted Boltzmann Machines or (RBMs) for short. RBMs have been successfully used for various applications, including
fixing imbalanced datasets. Because RBMs learn the input in order to be
able to regenerate it, then they can learn the
distribution of the minority class in an imbalance dataset ,and then generate more data points of that class, transforming the imbalance dataset into a balanced
data set. Similarly, RBMs can also be used to
estimate missing values in different features of a data set. Another popular
application of Restricted Boltzmann Machines is automatic feature extraction
of especially unstructured data. And this concludes our high-level introduction to
autoencoders and Restricted Boltzmann Machines.