For the people reading on hive : this post is way nicer on my blog :)
check it out here : https://brokencode.io/how-to-easily-image-search-with-python/
This is the second time that I'm writing on how to do image search with python. The first blog post. That I wrote about the subject got a lot of interest and even today I regularly get people commenting on it or coming to the github repo asking for help. So I figured it was time for a refresher.
I have put a bit of work to put the library as a package. In order to allow you to just pip the library. This is a much better solution than me saying nonsense like "copy the file in your project". Now it is as easy as doing :
pip3 install python-imageseach-drov0
The above will probably fail or you won't be able to use the library as you need extra packages depending on your os :
sudo pip3 install python3-xlib sudo apt-get install -y scrot -y sudo apt-get install -y python3-tk sudo apt-get install -y python3-dev sudo apt-get install -y python3-opencv
brew install opencv
pip3 install -U pyobjc-core
pip3 install -U pyobjc
No extra installation steps needed :)
The simplest example to do image search with python is this:
from python_imagesearch.imagesearch import imagesearch
pos = imagesearch("./github.png")
if pos[0] != -1:
print("position : ", pos[0], pos[1])
else:
print("image not found")
Simply search for one occurrence of the image "github.png" on the screen and print its x/y position
Performs an image search on a specific rectangle of the screen, it's very useful to speed up searches as there will be less screen space to search.
It's also useful to focus the search only on a specific part of the screen to reduce the chances of having a false positive.
pos = imagesearcharea("./github.png", 0, 0, 800, 600)
if pos[0] != -1:
print("position : ", pos[0], pos[1])
else:
print("image not found")
Input:
image : path to the image file (see opencv imread for supported types)
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.8
im : a PIL image, usefull if you intend to search the same unchanging region for several elements
Output:
the top left corner coordinates of the element if found as an array [x,y] or [-1,-1] if not
Very useful to optimize imagesearcharea or imagesearch calls, by getting an already processed image you can perform multiple searches on it with great speed gains. Here's an example
# non -optimized way :
time1 = time.clock()
for i in range(10):
imagesearcharea("./github.png", 0, 0, 800, 600)
imagesearcharea("./panda.png", 0, 0, 800, 600)
print(str(time.clock() - time1) + " seconds (non optimized)")
# optimized way :
time1 = time.clock()
im = region_grabber((0, 0, 800, 600))
for i in range(10):
imagesearcharea("./github.png", 0, 0, 800, 600, 0.8, im)
imagesearcharea("./panda.png", 0, 0, 800, 600, 0.8, im)
print(str(time.clock() - time1) + " seconds (optimized)")
# sample output :
# 1.6233619831305721 seconds (non optimized)
# 0.4075934110084374 seconds (optimized)
Input: a tuple containing the 4 coordinates of the region to capture tuple should contain coordinates of : topx, topy, bottomx, bottomy
Output: a PIL image of the area selected.
Searches for an image on screen continuously until it's found, useful to make a waiting script until x image appears. For instance waiting for the end of a loading screen.
from python_imagesearch.imagesearch import imagesearch_looppos = imagesearch_loop("./github.png", 1)
print("position : ", pos[0], pos[1])
Input:
image : path to the image file (see opencv imread for supported types)
time : Waiting time after failing to find the image (seconds)
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.8
Output:
the top left corner coordinates of the element if found as an array [x,y]
Searches for an image on screen continuously until it's found or max number of samples reached.
from python_imagesearch.imagesearch import imagesearch_numLoop
pos = imagesearch_numLoop("./github.png", 1, 50)
if pos[0] != -1:
print("position : ", pos[0], pos[1])
else:
print("image not found")
Input:
image : path to the image file (see opencv imread for supported types)
time : Waiting time after failing to find the image
maxSamples: maximum number of samples before function times out.
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.8
Output: the top left corner coordinates of the element if found as an array [x,y]
Very similar to imagesearch_loop except it works with regions
from python_imagesearch.imagesearch import imagesearch_region_loop
pos = imagesearch_region_loop("./github.png", 1, 0, 0, 800, 600)
print("position : ", pos[0], pos[1])
Input:
image : path to the image file (see opencv imread for supported types)
time : Waiting time after failing to find the image
x1 : top left x value
y1 : top left y value
x2 : bottom right x value
y2 : bottom right y value
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.8
Output:
the top left corner coordinates of the element as an array [x,y]
Counts how many occurrences there are of the image there are on the screen.
from python_imagesearch.imagesearch import imagesearch_count
count = imagesearch_count("./github.png")
print(count)
Input:
image : path to the target image file (see opencv imread for supported types)
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.9
Output:
the number of times a given image appears on the screen.
optionally an output image with all the occurances boxed with a red outline.
Performs an imagesearch on all the images in a folder. This function was done by kadusalles
from python_imagesearch.imagesearch import imagesearch_countresults = str(imagesearch_from_folder('./', 0.8))
print(results)
Input:
path: to the folder containing the images (supported image types are jpg, gif, png and jpeg)
precision : the higher, the lesser tolerant and fewer false positives are found default is 0.9
Output:
A dictionnary with all the images where the key is the image path and the value is it's position
And that's about it ! Now you should be able to easily perform Image search with python. If you are interested in the actual code or want to contribute feel free to head on over to the github repository : https://github.com/drov0/python-imagesearch and if you liked my article, come to see more at https://brokencode.io