How To Write A Generator For Keras Model For Predict_generator

I have a trained keras model, and I am trying to run predictions with CPU only. I want this to be as quick as possible, so I thought I would use predict_generator with multiple wor

Solution 1:

When you just call .predict, Keras already tries to use all available cores / predict in parallel the data points you give it. The predict generator with multiple workers might not add any benefit in this instance because each worker will need to wait for its turn to execute or share the available cores. Either way you end up getting the same performance.

Use of generators are more common if your data:

• does not fit in memory. You can take batches at a time and predict rather than creating a large data array and call predict.
• requires on the fly processing that might change / be random per batch.
• cannot be stored easily in a NumPy array and has a different way of batching beyond slicing data points.