- 1 year ago
How to use multiprocessing package in Python?
The multiprocessing
package in Python provides support for parallel processing and concurrency. It allows you to execute multiple processes simultaneously, taking advantage of multiple CPU cores. Here's a basic overview of how to use the multiprocessing
package:
Import the
multiprocessing
module:pythonimport multiprocessing
Define a target function:
Create a function that will be executed in parallel by each process. This function should encapsulate the task you want to perform concurrently. It can take arguments and return values.pythondef process_task(argument):
# Perform the task
result = argument + 10
return result
Create a
Pool
object:
ThePool
class from themultiprocessing
module allows you to create a pool of worker processes. The number of processes you specify determines the parallelism.pythonpool = multiprocessing.Pool(processes=4) # Create a pool with 4 worker processes
Submit tasks to the pool:
Use theapply_async()
method of thePool
object to submit tasks to the pool for execution. This method returns anAsyncResult
object that you can use to retrieve the result of each task.pythonresult1 = pool.apply_async(process_task, (5,))
result2 = pool.apply_async(process_task, (10,))
Retrieve the results:
Use theget()
method of theAsyncResult
object to retrieve the result of each task. This method will block until the result is available.pythonresult1_value = result1.get()
result2_value = result2.get()
Close the pool and join the processes:
After all tasks have been submitted and results retrieved, it's important to close the pool to prevent any additional tasks from being submitted. Then, use thejoin()
method to wait for all processes to finish before moving on.pythonpool.close()
pool.join()
By following these steps, you can use the multiprocessing
package in Python to parallelize tasks and leverage multiple CPU cores for increased performance.