Introduction
Today’s blog post is pretty short. I have been playing around with autoencoders a lot. They are super cool but also relatively simple to explain/implement.
These are the notes for day 11 of my AI/ML grind. Today’s topic: autoencoders.
You can also checkout these notes using my ai-ml-diary repository.
Autoencoders
Autoencoder: a system that teaches itself how to encode information.
An autoenconder works by taking some data as input, compressing it using a set of encoder layers and decompressing using a set of decoder layers. The diagram below is a common representation for this architecture.
Implementation
Implementing an autoencoder is pretty simple. Here’s the architecture I used to train an autoencoder for the MNIST dataset:
self.input = nn.Linear(784, 256)
self.encoder = nn.Linear(256, 64)
self.code = nn.Linear(64, 256)
self.decoder = nn.Linear(256, 784)
With just 32 epochs of training, I was able to obtain really good results!
I also tested the same model on noisy and occluded versions of the test data. The results are pretty good given that the model wasn’t trained for this specifically.
