At work, I wrote a Python script which uses the multiprocessing module to process many servers in parallel. The code looks something like:
def processServer(server):
# Do work...
numParallelTasks = ...
servers = [...]
pool = multiprocessing.Pool(processes=numParallelTasks)
results = pool.map(processServer, servers)
I wanted to pass some extra state to processServer without using a global variable. My first attempt was to use a closure, so I wrote the following:
def processServer(extraState):
def processServerWorker(server):
# Do work, using extraState as needed
return processServerWorker
numParallelTasks = ...
servers = [...]
extraState = ...
pool = multiprocessing.Pool(processes=numParallelTasks)
results = pool.map(processServer(extraState), servers)
This failed with the following error:
Read more...