Python

Python Multiprocessing Module and Closures

Python Multiprocessing Module and Closures

At work, I wrote a Python script which uses the multiprocessing module to process many servers in parallel. The code looks something like: def processServer(server): # Do work... numParallelTasks = ... servers = [...] pool = multiprocessing.Pool(processes=numParallelTasks) results = pool.map(processServer, servers) I wanted to pass some extra state to processServer without using a global variable. My first attempt was to use a closure, so I wrote the following: def processServer(extraState): def processServerWorker(server): # Do work, using extraState as needed return processServerWorker numParallelTasks = .