What will I Learn?
This tutorial covers the topics on Multiprocessing in Python.
Requirements
Note: This tutorial is performed in Pycharm IDE in laptop with Ubuntu 17.1, 64 bit OS.
Difficulty
Intermediate. I strongly recommend to learn all my previous tutorials before starting this. The link of previous tutorials are at the buttom of this tutorial.
Tutorial Content
In my previous tutorials of this series we learnt about multithreading in Python. Please visit here for Multithreading in Python. Today we are going to discuss about multiprocessing in Python.
What is multiprocessing?
If we run our programs in two or more core of CPU than it is known as multiprocessing. In multiprocessing two or more processes use different spaces unlike multithreading which uses
same space. As we need different spaces in multiprocessing, it relies on hardware rather that software.
Python provides multiprocessing package to deal with multiple processes that runs at same time. It is used to start new processes and handle it.
We will understand multiprocessing in Python by the help of a example.
As this tutorial is to show differences and compare multithreading and multiprocessing we will be using yestarday's multithreading example in multiprocessing.
At first we will start by importing the time and multiprocessing module.
import time
import multiprocessing
time module in python gives different methods to use time related function.For more details on time module, please visit here.
Now we will create two different functions to calculate area and perimeter of given lengths in list of a square.
To learn more about functions visit my previous tutorial Python tutorials for beginners : Part - III.
def area_sqr(length):
for l in length:
time.sleep(5)
print('Area:', l * l)
def peri_sqr(length):
for l in length:
time.sleep(5)
print('Perimeter:', 4 * l)
In above code, area_sqr and peri_sqr function is created to calculate area and perimeter of square which takes length as arguments. For every length we provide through a list it calculates the area and
perimeter of the square. time.sleep(5) delays the each iteration by 5 seconds. Execution process is stopped for 5 seconds before printing area of square.
if __name__ == "__main__":
Here the python interpreter is executing our source file as main program so the value of _ name _ is set to '_ main _' .
When our source file which is this for our case run as the main program, value of _ name _ is equal to _ main _ else the value of _ name _ is set to contain the name of the source file.
So if __name__ == "__main__": is satisfied then area_sqr and peri_sqr is called.
We define list of length as:
length = [ 4, 5, 6 ]
time.time() method of time module gives the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC) Thursday, 1 January 1970.
We will record it to measure the time taken to finish this whole multiprocess after execution of the whole program.
t = time.time()
process1 = multiprocessing.Process(target=area_sqr, args=(length,))
process2 = multiprocessing.Process(target=peri_sqr, args=(length,))
Now, to create a process we will call multiprocessing.Process method of multiprocessing module which takes target and args. target here passed is function name which we can think of as
task to be completed. It is called by the run() method of multiprocessing.Process object and shows what activity is to be done in that particular process. length is passed as args here, which is invoked
by target while creating process. As we can see the methods of multiprocessing and multithreading are quite similar and most of the time multiprocessing module replicates the API of the threading module.
process1.start()
process2.start()
We called start() to start process. We should call start() equall to number of our process.We call it once in that particular process object.
process1.join()
process2.join()
join() makes ease to execute process by waiting them till the processes finish execution to execute below print().
Now we also print the time that took to finish up this whole process. We do that by subtracting the time we recorded before the process was started from the time recorded after
the process is finished.
print("Finished in : ", time.time() - t)
Our final code looks like:
import time
import multiprocessing
def area_sqr(length):
for l in length:
time.sleep(5)
print('Area:', l * l)
def peri_sqr(length):
for l in length:
time.sleep(5)
print('Perimeter:', 4 * l)
if __name__ == "__main__":
length = [ 4, 5, 6 ]
t = time.time()
process1 = multiprocessing.Process(target=area_sqr, args=(length,))
process2 = multiprocessing.Process(target=peri_sqr, args=(length,))
process1.start()
process2.start()
process1.join()
process2.join()
print("Finished in : ", time.time() - t)
Output:
Area: 16
Perimeter: 16
Perimeter: 20
Area: 25
Perimeter: 24
Area: 36
Finished in : 15.016069889068604
As we have time.sleep(5) which makes CPU idle for 5 second we will have enough time to see what processes are running on in our device CPU before execution finishes.
If you are on windows open your Task Manager, but I'm on linux so via command line I will enter ps -ef to see current running procesess in CPU then we can see:
Note: Check the Process id 5366,5367 and 5368 in above image.
In above images we can see that our programm is occuping three spaces in CPU for multiprocessing. The three spaces are used for our main program process and for the two
processes we created in our program.
Now when we execute multithreading example file from previous tutorial which can be found here setting the time.sleep(5) we can see by same process:
In above image, we can see that our program is using just one space in the CPU.
Note: Check the Process id 5391 in above image.
So, from this we knew that multiprocessing module in Python includes methods similiar to threading and saw different spaces reseved in multiprocessing and shared space in multithreading.
all above codes including previous tutorials codes are available in my Github repo. Click here to download
For more details please visit Python Docs.
Curriculum
Python tutorials for beginners : Part - I
Python tutorials for beginners : Part - II
Python tutorials for beginners : Part - III
Reading and writing to files in python
Matching and Searching Strings with regex in Python
Send mail with attachments in Python Program