Are You Better Than The Algorithm?

We have reached a new milestone — with 99.78% accuracy in the LFW benchmark test. This test is one of the most famous and well-respected among the developers of facial recognition algorithms and solutions.

LFW is the abbreviation of Labeled Faces in the Wild. This is a description from the official website of the LFW. “This is a database of face photographs designed for studying the problem of unconstrained face recognition. The data set contains more than 13,000 images of faces collected from the web. Each face has been labelled with the name of the person pictured. 1680 of the people pictured have two or more distinct photos in the data set. The only constraint on these faces is that they were detected by the Viola-Jones face detector. More details can be found in the technical report below.

There are now four different sets of LFW images including the original and three different types of “aligned” images. The aligned images include “funnelled images” (ICCV 2007), LFW-a, which uses an unpublished method of alignment, and “deep funnelledd” images (NIPS 2012). Among these, LFW-a and the deep funnelled images produce superior results for most face verification algorithms over the original images and over the funnelledd images (ICCV 2007).”

Different companies, independent teams and individual data-scientists are using this data-set to check the quality of their algorithms. Some of them may also use it to provide proof that their products are worth the purchase. But in some cases, developers just use a stack of several NN to achieve better results. In reality, this combination of NN can’t be used in the product, because it works very slowly and consumes a lot of computing power. So, the team can show incredible results in LFW test, but the accuracy of the algorithm which is being used in the product that is actually available to the customers can be much lower.

So, I would say that there is a very good reason to be proud of getting great results if they have been reached by running a neural network, which is the basement of the final product available to customers. In our case, we used Faceter’s NN, which is used in our product. It means that our customers can rely on this result and expect that in real life it will be the same or very close to this numbers. Of course, the number can vary. It depends on many factors: angle of the camera, the visibility of the face, the illuminance, the distance to the person, etc. In our PoC projects, we saw several different results — in some cases, the accuracy was lower, in some — higher. However, what this LFW result shows is that our algorithm is one of the best in the world. We are in the top-6 according to this benchmark. Actually, we share the 5th line with another team with the same result. And we continue working hard to improve our algorithm’s accuracy, despite that, in many cases, 99% accuracy is more than enough.

Yet, there are some examples where algorithms were not entirely accurate. How about you, can you recognize the same person in two different pics? You will find the correct answers below.