开发者

Python Multiprocess diff between Windows and Linux

开发者 https://www.devze.com 2023-03-17 12:02 出处:网络
I have a script called jobrunner.py that calls class methods in main.py. See below... # jobrunner.py from multiprocessing import Process

I have a script called jobrunner.py that calls class methods in main.py. See below...

# jobrunner.py
from multiprocessing import Process
import main
from main import BBOX

def _a(arg):
    f = main.a()
    print f.run()

def _b(arg):
    p = main.b()
    print p.run()

if __name__ == '__main__':
    world = '-180,180,-90,90'
    BBOX.append(world.split(','))

    p1 = Process(target=_a, args=("1",))
    p2 = Process(target=_b, args=("1",))

    p1.start()
    p2.start()

    p1.join()
    p2.join()

Processes _a and _b are invoked without a开发者_如何学Cny problems on OSX and Ubuntu but when I try to run the same thing on Windows (same version of python and all), it fails saying that index is out of range. This leads me to believe that the "global" variable BBOX is not being set or passed between modules on the Windows platform. Has anyone else seen something like this and know how to fix it?

Adam

UPDATE: I figured it out even though it might be a total hack...See below!

# jobrunner.py
from multiprocessing import Process
import main
from main import BBOX

def _a(arg):
    BBOX.append(arg) #This is the key
    f = main.a()
    print f.run()

def _b(arg):
    BBOX.append(arg) #This is the key
    p = main.b()
    print p.run()

if __name__ == '__main__':
    world = '-180,180,-90,90'
    BBOX.append(world.split(','))

    p1 = Process(target=_a, args=(BBOX[0],))
    p2 = Process(target=_b, args=(BBOX[0],))

    p1.start()
    p2.start()

    p1.join()
    p2.join()


You shouldn't expect the values of global variables that you set in the parent process to be automatically propagated to the child processes.

Your code happens to work on Unix-like platforms because on those platforms multiprocessing uses fork(). This means that every child processes gets a copy of the parent process's address space, including all global variables.

This isn't the case on Windows; every variable from the parent process that needs to be accessed by the child has to be explicitly passed down or placed in shared memory.

Once you do this, your code will work on both Unix and Windows.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号