Training On The Merged Layer In Keras

I am implementing following this paper by Mohammad Havaei. It uses following architecture: I have modified some code from here to do so. print 'Compiling two-path model...' #local

Solution 1:

from keras.layers import *from keras.models import Model

print 'Compiling two-path model...'

# Input of the model
input_model = Input(shape=(4,33,33))
# Local pathway
#Addfirst convolution
model_l = Convolution2D(64,7,7,
                            border_mode='valid', 
                            activation='relu', 
                            W_regularizer=l1l2(l1=0.01, l2=0.01))(input_model)
model_l = BatchNormalization(mode=0,axis=1)(model_l)
model_l = MaxPooling2D(pool_size=(2,2),strides=(1,1))(model_l)
model_l = Dropout(0.5)(model_l)
#Addsecond convolution
model_l = Convolution2D(64,3,3,
                        border_mode='valid',
                        W_regularizer=l1l2(l1=0.01, l2=0.01),
                        input_shape=(4,33,33))(model_l)
model_l = BatchNormalization(mode=0,axis=1)(model_l)
model_l = MaxPooling2D(pool_size=(4,4),strides=(1,1))(model_l)
model_l = Dropout(0.5)(model_l)

#global pathway
model_g = Convolution2D(160,12,12,
                        border_mode='valid', 
                        activation='relu',
                        W_regularizer=l1l2(l1=0.01, l2=0.01))(input_model)
model_g = BatchNormalization(mode=0,axis=1)(model_g)
model_g = MaxPooling2D(pool_size=(2,2), strides=(1,1))(model_g)
model_g = Dropout(0.5)(model_g)

# mergelocalandglobal pathways

merge=Merge(mode='concat', concat_axis=1)([model_l,model_g])
merge= Convolution2D(5,21,21,
                      border_mode='valid',
                      W_regularizer=l1l2(l1=0.01, l2=0.01))(merge)
merge= Flatten()(merge)
predictions = Dense(5, activation='softmax')(merge)

model_merged = Model(input=input_model,output=predictions)
sgd = SGD(lr=0.001, decay=0.01, momentum=0.9)
model_merged.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

print('Done')
return model_merged

this is the equivalent of the network you posted but defined with the Functional API

As you can see, there is only 1 Input layer, used twice. You can then train it like you said :

model_merged.fit(X_train, Y_train, batch_size=self.batch_size, nb_epoch=self.n_epoch, validation_split=0.1, verbose=1)

does that help ?