Parallelization of a Neural Network Algorithm for Handwriting Recognition: Can we Increase the Speed, Keeping the Same Accuracy

Abstract

This paper examines the problem of parallelizing neural network training. For our solution we use the backpropagation neural network, as a breakthrough example in the field of deep learning. The challenge of our solution is to twist the algorithm in such a way so it can be executed in parallel, rather than sequentially. In this paper we would like to test validity of a research hypothesis if we can increase the speed by parallelizing the back-propagation algorithm and keep the same accuracy. For this purpose we will develop a use-case of a handwriting recognition algorithm and run several experiments to test the performance, both in execution speed and accuracy. At the end we are going to examine just how much it benefits us to try and write a parallel program for a neural network, with regards to the time it takes to train the neural network and the accuracy of the predictions. Our handwriting problem is that of classification, and in order to implement any sort of solution, we must have data. The MNIST dataset of handwritten digits will provide our necessary data to solve the problem.