- 1 year ago
Global Variable in Multiprocessing Pool
In Python's multiprocessing module, the workers in a multiprocessing pool operate in separate processes, and each process has its own memory space. As a result, changes made to global variables in one worker process won't be reflected in other worker processes. However, you can use shared memory or other inter-process communication techniques to achieve the desired behavior. Here's an example using a Value
object from the multiprocessing
module to share a global variable among processes:
pythonfrom multiprocessing import Pool, Value
# Define a global variable
global_var = Value('i', 0)
def worker_function(arg):
# Access the shared global variable
with global_var.get_lock():
global_var.value += 1
print(f"Worker {arg}: Global variable value: {global_var.value}")
if __name__ == '__main__':
# Create a multiprocessing pool
pool = Pool()
# Submit jobs to the pool
pool.map(worker_function, range(5))
# Close the pool and wait for the processes to finish
pool.close()
pool.join()
# Print the final value of the global variable
print(f"Final global variable value: {global_var.value}")
In this example, a Value
object is created using Value('i', 0)
, which creates a shared integer variable initialized with a value of 0. Within each worker function, the shared global variable is accessed using a lock (with global_var.get_lock():
) to ensure thread-safe access. Any modifications to the variable within the lock will be reflected in other worker processes.
By utilizing shared memory objects like Value
, you can share data between worker processes in a multiprocessing pool and achieve the desired global variable behavior.