Deep Learning Inference with Scilab IPCV – Lenet5 with MNIST Visualization

July 20, 2018

This tutorial is the continuation from the previous tutorial, however, as this article explore more into details on how the CNN works by looking into the internal layers of the network, those who are just interested to deploy application using the model could skipped this tutorial.

On the other hand, this tutorial would be helpful for teaching and understanding how’s CNN works, so if you’re lecturers or academicians, you might find this useful to illustrate to students.

Let’s start by loading the model and perform out forward pass of the model with an input image.

// Initialize
dnn_unloadallmodels
dnn_path = fullpath(getIPCVpath() + '/images/dnn/');
net = dnn_readmodel(dnn_path + 'lenet5.pb','','tensorflow');
 
 
// Read Image
S = imread(dnn_path + '3.jpg');
 
// We use ~ as the background is black and the object is white
out = dnn_forward(net,~S,[28,28]);
disp(out')
[maxV,maxI]=max(out);
scf();imshow(S);
xnumb(10,10,maxI-1);
e = gce();
e.font_size = 10;
e.font_color = 5;

Figure 1 : Output Prediction with Original Image

The above lines of codes will load the pre-trained tensorflow model with the following architecture:

[Conv–>ReLU–>MaxPool] –> Convolution Layer – 6 filters. (5×5)
[Conv–>ReLU–>MaxPool] –> Convolution Layer – 16 filters. (5×5)
[Dense–>ReLU] –> Fully Connected Layer – 120
[Dense–>ReLU] –> Fully Connected Layer – 84
[Softmax] –> Fully Connected Layer – 10

The first convolution layer consist of 6 filters, the following lines will shows the 6 feature maps (output of the convolution filters) and also filters parameters which produce such maps.

out1 = dnn_forward(net,~S,[28,28],"conv2d/Conv2D");
scf();dnn_showfeature(out1);
 
// Get the filter parameters which produce the feature maps
para1 = dnn_getparam(net,"conv2d/Conv2D");
scf();dnn_showparam(para1);
scf();dnn_showparamf2d(para1);
scf();dnn_showparamf3d(para1);

Figure 2: Feature Maps of First Convolution Layer

Figure 3: Filter Parameters which Produce the Feature Maps (Spatial Domain)

Figure 4: Filter Parameters which Produce the Feature Maps (Frequency Domain – 2D)

Figure 5: Filter Parameters which Produce the Feature Maps (Frequency Domain – 3D)

Finally, let’s see how do we manually reproduce the feature maps from the filter parameters.

Sgray = rgb2gray(S);
S2 = double(~Sgray);
S3 = imresize(S2,[28,28]);
 
for i = 1:6
    Sout(:,:,i) = imfilter(S3',para1(:,:,i));
end
 
Sout(Sout&lt;0) = 0;
dnn_showfeature(Sout);

As we could see, doing it manually required some extra steps, including converting image to grayscale, convert to double, and resize the image to the same size with the model input. After that, we applied each of the filter on the same image to obtain the feature maps which is similar to the one we are getting from the forward pass of the model. Here’s the result:

Figure 6: Manually Reproducing the Feature Map with Filtering Technique (Convolution)

The results are similar to the one in figure 2. The slight differences are due to the number format which we used causing some rounding errors, however, this is just to illustrate how CNN works and how the training of the model produce the filters to extract the features from the image automatically.

Leave a Reply Cancel reply