Add Low Layers In A Tensorflow Model

Trying to develop some transfert learning algorithm, I use some trained neural networks and add layers. I am using Tensorflow and python. It seems quite common to use existing grap

Solution 1:

You can't insert layers between existing layers of a graph, but you can import a graph with some rewiring along the way. As Pietro Tortella pointed out, the approach in Tensorflow: How to replace a node in a calculation graph? should work. Here is an example:

import tensorflow as tf

with tf.Graph().as_default() as g1:
    input1 = tf.placeholder(dtype=tf.float32, name="input_1")
    l1 = tf.multiply(input1, tf.constant(2.0), name="mult_1")
    l2 = tf.multiply(l1, tf.constant(3.0), name="mult_2")

g1_def = g1.as_graph_def()

with tf.Graph().as_default() as new_g:
    new_input = tf.placeholder(dtype=tf.float32, name="new_input")
    op_to_insert = tf.add(new_input, tf.constant(4.0), name="inserted_op")
    mult_2, = tf.import_graph_def(g1_def, input_map={"input_1": op_to_insert},
                                  return_elements=["mult_2"])

The original graph looks like this and the imported graph looks like this.

If you want to use tf.train.import_meta_graph, you can still pass in the

input_map={"input_1": op_to_insert}

kwarg. It will get passed down to import_graph_def.