Deepak Soundappan - Multi Processing

Multi Processing Learning multi processing makes a lot of sense.

Multiprocessing refers to using multiple CPUs or processors to perform tasks simultaneously—true parallelism. Unlike multithreading (which often just switches between tasks on a single CPU), multiprocessing means multiple things are actually happening at the same time, on different processors.

Multiple Chefs in a Kitchen (vs. One Chef): Each chef (CPU) works independently on a different dish. All dishes cook truly simultaneously, not just taking turns.
Construction Site with Different Teams: One team lays the foundation, another builds the walls, and another installs wiring—all at the same time, in different areas.
Group of Developers Working on Different Modules of the Same Project: Each developer writes different parts of a system independently and in parallel, boosting development speed.

Sample code

The process in python to use multiprocessing :

We primarily starts with using the multiprocessing library which allows use to create parallel python processes
Declare the function that has to be executed in different process. This could be high data intensive process like machine learning model, image processing etc.
Then we will use the join to wait till all the processes to completed. Like a building engineer who waits till all the departments in the building manufacturing have completed their peice of work.

What This Does:

Launches worker processes for each of the given name.
Each process computes the square of a number.
Tasks are distributed among processes, and executed in parallel.
Example : Total time ≈ 2 seconds (instead of 5 seconds serially).

Notes:

Always put multiprocessing code inside the if __name__ == '__main__': block on Windows/macOS to avoid issues.
Best for CPU-intensive tasks (e.g., math, simulations, data crunching).

In case one wants to limit the number of the processes that is needed, but still dynamically change the parameters.

One can do that usign the starmap function in multiprocessing along with multiprocessing pool

Sample code

Page updated

Google Sites

Report abuse