Skip to content

Scilab IPCV

Image Processing and Computer Vision Module for Scilab 6.0!

  • Installation
    • Installation IPCV v1.1
    • Installation IPCV v1.2
  • Getting Started
  • Subscribe for FREE
  • Support Us
  • Courses
    • Artificial Intelligence with Scilab
    • Image Processing with Scilab

Deep Learning Inference with Scilab IPCV – Lenet5 with MNIST Visualization

July 20, 2018

This tutorial is the continuation from the previous tutorial, however, as this article explore more into details on how the CNN works by looking into the internal layers of the network, those who are just interested to deploy application using the model could skipped this tutorial.

On the other hand, this tutorial would be helpful for teaching and understanding how’s CNN works, so if you’re lecturers or academicians, you might find this useful to illustrate to students.

Let’s start by loading the model and perform out forward pass of the model with an input image.

// Initialize
dnn_unloadallmodels
dnn_path = fullpath(getIPCVpath() + '/images/dnn/');
net = dnn_readmodel(dnn_path + 'lenet5.pb','','tensorflow');
 
 
// Read Image
S = imread(dnn_path + '3.jpg');
 
// We use ~ as the background is black and the object is white
out = dnn_forward(net,~S,[28,28]);
disp(out')
[maxV,maxI]=max(out);
scf();imshow(S);
xnumb(10,10,maxI-1);
e = gce();
e.font_size = 10;
e.font_color = 5;

// Initialize dnn_unloadallmodels dnn_path = fullpath(getIPCVpath() + '/images/dnn/'); net = dnn_readmodel(dnn_path + 'lenet5.pb','','tensorflow'); // Read Image S = imread(dnn_path + '3.jpg'); // We use ~ as the background is black and the object is white out = dnn_forward(net,~S,[28,28]); disp(out') [maxV,maxI]=max(out); scf();imshow(S); xnumb(10,10,maxI-1); e = gce(); e.font_size = 10; e.font_color = 5;

Figure 1 : Output Prediction with Original Image

The above lines of codes will load the pre-trained tensorflow model with the following architecture:

[Conv–>ReLU–>MaxPool] –> Convolution Layer – 6 filters. (5×5)
[Conv–>ReLU–>MaxPool] –> Convolution Layer – 16 filters. (5×5)
[Dense–>ReLU] –> Fully Connected Layer – 120
[Dense–>ReLU] –> Fully Connected Layer – 84
[Softmax] –> Fully Connected Layer – 10

The first convolution layer consist of 6 filters, the following lines will shows the 6 feature maps (output of the convolution filters) and also filters parameters which produce such maps.

 

out1 = dnn_forward(net,~S,[28,28],"conv2d/Conv2D");
scf();dnn_showfeature(out1);
 
// Get the filter parameters which produce the feature maps
para1 = dnn_getparam(net,"conv2d/Conv2D");
scf();dnn_showparam(para1);
scf();dnn_showparamf2d(para1);
scf();dnn_showparamf3d(para1);

out1 = dnn_forward(net,~S,[28,28],"conv2d/Conv2D"); scf();dnn_showfeature(out1); // Get the filter parameters which produce the feature maps para1 = dnn_getparam(net,"conv2d/Conv2D"); scf();dnn_showparam(para1); scf();dnn_showparamf2d(para1); scf();dnn_showparamf3d(para1);

Figure 2: Feature Maps of First Convolution Layer

Figure 3: Filter Parameters which Produce the Feature Maps (Spatial Domain)

Figure 4: Filter Parameters which Produce the Feature Maps (Frequency Domain – 2D)

Figure 5: Filter Parameters which Produce the Feature Maps (Frequency Domain – 3D)

Finally, let’s see how do we manually reproduce the feature maps from the filter parameters.

Sgray = rgb2gray(S);
S2 = double(~Sgray);
S3 = imresize(S2,[28,28]);
 
for i = 1:6
    Sout(:,:,i) = imfilter(S3',para1(:,:,i));
end
 
Sout(Sout<0) = 0;
dnn_showfeature(Sout);

Sgray = rgb2gray(S); S2 = double(~Sgray); S3 = imresize(S2,[28,28]); for i = 1:6 Sout(:,:,i) = imfilter(S3',para1(:,:,i)); end Sout(Sout<0) = 0; dnn_showfeature(Sout);

As we could see, doing it manually required some extra steps, including converting image to grayscale, convert to double, and resize the image to the same size with the model input. After that, we applied each of the filter on the same image to obtain the feature maps which is similar to the one we are getting from the forward pass of the model. Here’s the result:

Figure 6: Manually Reproducing the Feature Map with Filtering Technique (Convolution)

The results are similar to the one in figure 2. The slight differences are due to the number format which we used causing some rounding errors, however, this is just to illustrate how CNN works and how the training of the model produce the filters to extract the features from the image automatically.

Share on Facebook Share
Share on TwitterTweet
Share on Google Plus Share
Share on LinkedIn Share
Send email Mail

Post navigation

Previous Post:

Deep Learning Inference with Scilab IPCV – Pre-Trained Lenet5 with MNIST

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Support Us

By Buying Us Coffee :)

Topics

  • Computer Vision & Hardware Interfaces
  • Feature Detection, Description and Matching
  • Image Fusion
  • Image Linear Filtering
  • Image Reading, Display and Exploration
  • Image Stitching
  • Image Types and Color Space Conversions
  • Installation
  • Introduction
  • Machine Learning & Deep Learning
  • Morphological Operations
  • Object Detection
  • Object Recognition
  • Spatial Transformations
  • Super Resolution
  • Uncategorized
  • Utilities

Recent Posts

  • Deep Learning Inference with Scilab IPCV – Lenet5 with MNIST Visualization
  • Deep Learning Inference with Scilab IPCV – Pre-Trained Lenet5 with MNIST
  • What’s New in IPCV 2.0
  • Computer Vision – Live Video from Webcam
  • Drawing Shapes by Overwriting Pixel Value




Tags

Affine affine transform Clarifai cnn computer vision convert to binary image convolution convolutional neural network correlation covert rgb to gray deep learning dnn feature detection feature extraction Google Vision API graphic user interface gui image 3d plot image filtering image matching image measurement image processing image reading image resize image rotation image scaling image shearing image stitching image transform installation introduction linear regression machine learning map mapping morphology object detection object removal panoramic perspective perspective transform scilab spatial transform sudoku translation
© 2021 Scilab IPCV