Running A Python Script Concurrently Over The Network. Is This Ok?

We have a python script + associated modules on a network drive. We are running this script simultaneously from several computers at once. Sometimes one of them randomly fails wit

Solution 1:

Don't do that, where "that" is have multiple computers trying to use the same .pyc(s).

The likely reason the pyc gets modified is that you likely have slightly different versions of Python on different computers. One writes its version of the .pyc, and another comes along and sees that it's not what it expected, and writes its own version.

It would surprise me not at all to learn that strange bugs are possible in the situation where different computers are simultaneously trying to deal with the same .pyc files. This is an invitation for race conditions, as there is no locking mechanism in use.

Either have each system copy the script to a local place before running it, or disable .pyc generation (-B command line flag or set the PYTHONDONTWRITEBYTECODE environment variable).

Solution 2:

It's an odd error (I've never seen one in the wild).

My hypothesis is that it has to do with one node regenerating the pyc file while another node tries to look at it. Trying deleting the pyc files and specifying -B on the Python command line:

-B     : don't write .py[co] files on import; also PYTHONDONTWRITEBYTECODE=x