Passing Python Object To Another Python Process

Let say we have a server application written in Python. Let also say that this main server process forked two more processes at the startup. Server awaits its clients, and when one

Solution 1:

Would pickling the socket object and pushing it through IPC work?

No. Inside that object is a file descriptor or handle to the kernel socket. It's just a number that the process uses to identify the socket when making system calls.

If you pickle that Python socket object and send it to another process, that process will be using a handle for a socket it didn't open. Or worse, that handle may refer to a different open file.

The most efficient way to handle this (on Linux) is like this:

• Master process opens listening socket (e.g. TCP port 80)
• Master process forks N children who all inherit that open socket
• They all call accept() and block, waiting for a new connection
• When a new client connects, the kernel will select one of the processes with a handle to that socket to accept the connection; the others will continue to wait

This way, you let the kernel handle the load balancing.

If you don't want this behavior, there is a way (in UNIX) to pass an open socket to another process. Again, this is more than just the handle; the kernel effectively copies the open socket to your processs's open file list. This mechanism is known as SCM_RIGHTS, and you can see an example (in C) here: http://man7.org/tlpi/code/online/dist/sockets/scm_rights_send.c.html

Otherwise, your master process will need to effectively proxy the connection to the child processes, reducing thr efficiency of the system.