Denoising Autoencoders — Tensorflow

Image Noise Reduction with Autoencoders

What are Autoencoders?

Autoencoders are an unsupervised learning technique in which we leverage neural networks for the task of representation learning. Specifically, we’ll design a neural network architecture such that we impose a bottleneck in the network which forces a compressed knowledge representation of the original input.

The ideal autoencoder model balances the following:

• Sensitive to the inputs enough to accurately build a reconstruction.
• Insensitive enough to the inputs that the model doesn’t simply memorize or overfit the training data.

We’d like our autoencoder to be sensitive enough to recreate the original observation but insensitive enough to the training data such that the model learns a generalizable encoding and decoding.

The idea behind Denoising Autoencoder

One approach towards developing a generalizable model is to slightly corrupt the input data but still, maintain the uncorrupted data as our target output.

With this approach, our model isn’t able to simply develop a mapping which memorizes the training data because our input and target output are no longer the same. Rather, the model learns a vector field for mapping the input data towards a lower-dimensional manifold (A manifold describes the high-density region where the input data concentrates); if this manifold accurately describes the natural data, we’ve effectively “canceled out” the added noise.

The above figure visualizes the vector field described by comparing the reconstruction of xx with the original value of xx. The yellow points represent training examples prior to the addition of noise. As you can see, the model has learned to adjust the corrupted input towards the learned manifold.

Implementing Denoising Autoencoder with Keras and TensorFlow

For this implementation, we are going to use the MNIST dataset for handwritten digits.

1. Importing Libraries

As shown below, Tensorflow allows us to easily load the MNIST data. The training and testing data loaded is stored in variables train and test respectively.

import numpy as np
#importing dataset
from tensorflow.keras.datasets import mnist
#for model architecture
from tensorflow.keras.layers import Dense, Input
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.callbacks import EarlyStopping, LambdaCallback
from tensorflow.keras.utils import to_categorical
#for visualisation
from matplotlib import pyplot as plt
%matplotlib inline

2. Normalize and reshape the data

After importing the dataset and required libraries, we’ll load train and test data from the dataset. After that, we’ll normalize and reshape it to feed as an input to the network. There are 70000 images which we reshaped in the matrices of size 28 x 28.

(x_train, y_train), (x_test,y_test) = mnist.load_data()
x_train = x_train.astype('float')/255.0
x_test = x_test.astype('float')/255.0
x_train = np.reshape(x_train, (60000,784))
x_test = np.reshape(x_test, (10000,784))

3. Adding Noise to the data

Now we’ll add noise to both train and test images which we then feed into the network. The noise factor used here is 0.9 which is a hyperparameter that can be tuned

x_train_noisy = x_train+np.random.rand(60000,784)*0.9
x_test_noisy = x_test+np.random.rand(10000,784)*0.9
x_train_noisy = np.clip(x_train_noisy, 0., 1.)
x_test_noisy = np.clip(x_test_noisy, 0., 1.)

let’s illustrate the noisy images and compare them with the clean ones.

def plot(x,p,labels = False):
    plt.figure(figsize = (20,2))
    for i in range(10):
        plt.subplot(1,10,i+1)
        plt.imshow(x[i].reshape(28,28),cmap = 'binary')
        plt.xticks([])
        plt.yticks([])
        if labels:
            plt.xlabel(np.argmax(p[i]))
    plt.show()

plot(x_train,None)

original images

plot(x_train,None)plot(x_train_noisy,None)

noisy images

4. Model without Autoencoder

Now to see the amazing difference an autoencoder can make we first will see how a model without it will perform on our noisy images. The model we are using here is very basic with just 1 hidden layer.

classifier = Sequential([
    Dense(256, activation = 'relu', input_shape = (784,)),
    Dense(256, activation = 'relu'),
    Dense(10,activation = 'softmax') ])
classifier.compile(
 optimizer = 'adam',
 loss = 'sparse_categorical_crossentropy',
 metrics = ['accuracy'])
classifier.fit(x_train, y_train, epochs = 3, batch_size = 512)
loss,acc = classifier.evaluate(x_test,y_test)
10000/10000 [==============================] - 0s 49us/sample - loss: 0.1042 - acc: 0.9691
#ACCURACY ON CLEAN TEST SET: 96.91%
loss,acc = classifier.evaluate(x_test_noisy,y_test)
10000/10000 [==============================] - 0s 39us/sample - loss: 9.1700 - acc: 0.1344
#ACCURACY ON NOISY TEST SET: 13.44%

As it can be clearly seen our model performs well with clean data but rather poorly on noisy data.

5. Building an Autoencoder

For Autoencoder, we will have 2 layers namely encoder and decoder.

• The encoder transforms the 28 x 28 x 1 image which has been flattened to 784*1 vector to 64*1 vector. This 64*1 dimensional space is called the bottleneck.
• The decoder does the exact opposite of an encoder; it transforms this 64*1 dimensional vector back to a 784*1 vector. We call this output a “reconstruction” of the original image.

input_image = Input(shape = (784,))
encoder = Dense(64, activation = 'relu')(input_image)
decoder = Dense(784, activation = 'sigmoid')(encoded)
autoencoder = Model(input_image, decoder)
autoencoder.compile(loss = 'binary_crossentropy', optimizer ='adam')
autoencoder.fit( x_train_noisy, x_train, epochs = 100,
batch_size = 512, validation_split = 0.2, verbose = False,
callbacks = [ EarlyStopping(monitor = 'val_loss', patience = 5)])
predictions = autoencoder.predict(x_test_noisy)

We have used early stopping with a patience=5 which means if the val_loss will not vary for 5 epochs the model will stop training further.

plot(x_test_noisy, None)

plot(predictions, None)

Now we’ll feed our predictions to the classifier model we use earlier and compare the accuracy of predictions before and after using the autoencoder.

loss,acc = classifier.evaluate(predictions, y_test)
10000/10000 [==============================] - 0s 38us/sample - loss: 0.1729 - acc: 0.9459

Classifier Accuracy on the noisy data.

• Without autoencoder: 13.44%
• With autoencoder: 94.59%

6. Combining the models

Now we’ll combine our previous classifier model with the autoencoder model.

input_image = Input(shape = (784,))
x = autoencoder(input_image)
y = classifier(x)
denoise_and_classify = Model(input_image, y)
predictions = denoise_and_classify.predict(x_test_noisy)

7. Results

plot(x_test_noisy, predictions, True)

THANK YOU FOR READING.

References:

https://www.jeremyjordan.me/autoencoders/