Weight Visualization Of 3d Convolutional Kernel

I am using 3d kernel of size 3x3x3 for the convoution layer and would like to plot the weights of the layer. Since plotting in 3d is not possible i tried to split the kernels into

Solution 1:

The code you have used is good for displaying the Conv2D Convolution kernel in gray scale OR Conv3D Convolution kernel in Color(because of the depth of kernel causing merge of three gray scale images).

You need to modify the code as below to make it reusable for printing the Conv3d Convolution kernel in Gray scale. You have to modify the Depth, outgoing_channels and incoming_channels in the code as per the Conv3D layers kernel you are printing.

Modified Code -

# normalize filter values to 0-1 so we can visualize them
f_min, f_max = filters.min(), filters.max()
filters = (filters - f_min) / (f_max - f_min)
# plot first few filters# n_filters = outgoing channels
outgoing_channels = 2
n_filters, ix = outgoing_channels, 1for i inrange(n_filters):
    # get the filter
    f = filters[:, :, :, :, i]
    # plot each channel separately# Range of incoming channels
    incoming_channels = 4for j inrange(incoming_channels):
        # Range of Depth of the kernel .i.e. 3
        Depth = 3for k inrange(Depth):
            # specify subplot and turn of axis
            ax = pyplot.subplot((outgoing_channels*3), incoming_channels, ix)
            ax.set_xticks([])
            ax.set_yticks([])
            # plot filter channel in grayscale
            pyplot.imshow(f[:, :, k,j], cmap='gray')
            ix += 1# show the figure
pyplot.show()

---

End-to-End Example: This example explains

1. Builds a Model having Conv3D Layer.
2. Calculation of Params of a layer and breaking them to understand the layer weights and bias weights.
3. print the convolution kernel of the layer.

---

# (1) Importing dependency
%tensorflow_version 1.x
import tensorflow as tf
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense, Activation, Dropout, Flatten, Conv2D, MaxPooling2D, Conv3D
from keras.layers.normalization import BatchNormalization
import numpy as np

np.random.seed(1000)

# (2) Get Dataimport tflearn.datasets.oxflower17 as oxflower17
x, y = oxflower17.load_data(one_hot=True)
x = np.expand_dims(x,-1)

# (3) Create a sequential model
model = Sequential()

# 1st Convolutional Layer
model.add(Conv3D(filters=2, input_shape=(224,224,3,1), kernel_size=(3,3,3), strides=(4,4,4), padding='Same'))
model.add(Activation('relu'))

# 2nd Convolutional Layer
model.add(Conv3D(filters=4, kernel_size=(3,3,3), strides=(1,1,1), padding='Same'))
model.add(Activation('relu'))

# 3rd Convolutional Layer
model.add(Conv3D(filters=2, kernel_size=(3,3,3), strides=(1,1,1), padding='Same'))
model.add(Activation('relu'))

# Passing it to a dense layer
model.add(Flatten())
# 1st Dense Layer
model.add(Dense(100))
model.add(Activation('relu'))

# Output Layer
model.add(Dense(17))
model.add(Activation('softmax'))

model.summary()

# (4) Compile 
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])


model.fit(x, y, batch_size=64, epochs= 4, verbose=1, validation_split=0.2, shuffle=True)

Output of the code -

Model: "sequential_6"
_________________________________________________________________Layer (type)                 Output Shape              Param #   
=================================================================
conv3d_10 (Conv3D)           (None, 56, 56, 1, 2)      56        
_________________________________________________________________
activation_26 (Activation)   (None, 56, 56, 1, 2)      0         
_________________________________________________________________
conv3d_11 (Conv3D)           (None, 56, 56, 1, 4)      220       
_________________________________________________________________
activation_27 (Activation)   (None, 56, 56, 1, 4)      0         
_________________________________________________________________
conv3d_12 (Conv3D)           (None, 56, 56, 1, 2)      218       
_________________________________________________________________
activation_28 (Activation)   (None, 56, 56, 1, 2)      0         
_________________________________________________________________
flatten_6 (Flatten)          (None, 6272)              0         
_________________________________________________________________
dense_11 (Dense)             (None, 100)               627300    
_________________________________________________________________
activation_29 (Activation)   (None, 100)               0         
_________________________________________________________________
dense_12 (Dense)             (None, 17)                1717      
_________________________________________________________________activation_30 (Activation)   (None, 17)                0         
=================================================================
Total params: 629,511
Trainable params: 629,511
Non-trainable params: 0
_________________________________________________________________
Train on 1088 samples, validate on 272 samples
Epoch 1/4
1088/1088 [==============================] - 2s 1ms/step - loss: 2.8274 - acc: 0.0597 - val_loss: 2.8071 - val_acc: 0.0551
Epoch 2/4
1088/1088 [==============================] - 1s 522us/step - loss: 2.7204 - acc: 0.1728 - val_loss: 2.6962 - val_acc: 0.1360
Epoch 3/4
1088/1088 [==============================] - 1s 517us/step - loss: 2.4334 - acc: 0.3612 - val_loss: 2.4341 - val_acc: 0.2316
Epoch 4/4
1088/1088 [==============================] - 1s 518us/step - loss: 1.9551 - acc: 0.4577 - val_loss: 2.1545 - val_acc: 0.3309
<keras.callbacks.Historyat0x7ff4bba5e5f8>

Lets print the convolution kernels for below layer (3rd Conv3D layer in the model) -

conv3d_12 (Conv3D)            (None, 56, 56, 1, 2)      218       
_________________________________________________________________

Param calculated as follows -

Params of a Conv3D layer - “(nml*k*3) + k ”

• The filter size is “n*m”.
• “l” incoming_channels as the input. In our case it is 4.
• “k” outgoing_channels as output. In our case it is 2.

Thus params = (3*3*4*2*3)+2 = 218

Params are layer weights + Bias Weights. So removing biases .i.e. 2(biases are equal to features map or channel as output). Thus total layer weights = 218 - 2 = 216. So we should be getting 216/9 = 24 gray scale images of 3*3.

I have made the code generic so that it is reusable. You just need to modify Depth, outgoing_channels and incoming_channels which will fit the all the gray scale images of that layer. In our case outgoing_channels=2 and incoming_channels=4.

Run the Visualization Code -

from matplotlib import pyplot

# retrieve weights from the 3rd Conv3D layer
filters, biases = model.layers[4].get_weights()

# normalize filter values to 0-1 so we can visualize them
f_min, f_max = filters.min(), filters.max()
filters = (filters - f_min) / (f_max - f_min)
# plot first few filters# n_filters = outgoing channels
outgoing_channels = 2
n_filters, ix = outgoing_channels, 1for i inrange(n_filters):
    # get the filter
    f = filters[:, :, :, :, i]
    # plot each channel separately# Range of incoming channels
    incoming_channels = 4for j inrange(incoming_channels):
        # Range of Depth of the kernel .i.e. 3
        Depth = 3for k inrange(Depth):
            # specify subplot and turn of axis
            ax = pyplot.subplot((outgoing_channels*3), incoming_channels, ix)
            ax.set_xticks([])
            ax.set_yticks([])
            # plot filter channel in grayscale
            pyplot.imshow(f[:, :, k,j], cmap='gray')
            ix += 1# show the figure
pyplot.show()

Output of the Visualization code -

Hope this answers your question. Happy Learning.