Tensorflow And Threading

Below is the simple mnist tutorial (i.e. single layer softmax) from the Tensorflow website, which I tried to extend with a multi-threaded training step: from tensorflow.examples.tu

Solution 1:

Note that unfortunately, threadingwith python doesn't create real parallelism because of the GIL. So what happens here is that you will have multiple threads which are all running on the same CPU where in reality they are running sequentially. Therefore, I would suggest using Coordinator in Tensorflow. More information about Coordinator can be found here:

https://www.tensorflow.org/programmers_guide/threading_and_queueshttps://www.tensorflow.org/programmers_guide/reading_data

Finally, I would suggest you say:

with tf.device('/cpu:0'):
    your code should go here... 'for the first thread'

Then use another cpu for the other thread and so on... Hope this answer finds you well!!