OSINT: Automate Face Comparison With Python and Face++

Need to compare photos from social media that might show the same person? Learn how to automate the task with Python and Face++.

Investigating people online often requires looking at two different photos and trying to decide if they show the same person or two people that resemble each other.

For example, sometimes my person-of-interest is in a photo with a second person that appears to be a spouse, business partner, etc., but I don't know who the second person's identity. Then I find a 2nd photo of maybe that same person, they look similar but I'm not sure. In the 2nd photo the mystery person (or someone that looks like them) is tagged so I know their name. Did I discover their identity or just someone that looks like them?

This post will explain how to use a tool to do this comparison, reference a case study reminding you why this is useful, and try out a few matching and non-matching photos. I will provide you two Python scripts for running the comparison tool and, if you are new to Python, I will give you a walkthrough at the end that starts from scratch (and is very specific and easy).

Use a More Objective Comparison (that is easier too)

It is possible to automate the process of comparing two photos of similar-looking people to determine if they are photos of the same person. The tool Face++ evaluates two photos and provides an objective score of the likelihood that the photos show the same person.

Face++ is normally a fee-for-service product but it allows users to send several requests per month for free if they use the API. We can use Python to send requests to the API and therefore use this tool for free.

SIDENOTE:  If you don't know what an API is, it is basically a means for us to communicate with a computer or specifically the Face++ tool, but we need to communicate by sending a computer code instead of typing and clicking as we would with a standard website. For more information, click here.

This post explains how to use Python scripts provided below to objectively and automatically compare photos with the Face++ tool.

For people that are completely new to Python and APIs, there is a walk through at the end.

Comparing Faces

First, let's start with the more interesting part.

Researchers often run into situations where they need to compare faces in different photos, such as in the case below.

The case of the fake company Surefire Intelligence (click here to read a bellingcat.com article on it) shows that there are several situations where a researcher might need this tool. In that case, image searchers of profile photos additional media of the real perpetrator and found true identities of fake employees.

See an example below for two different photos of the same person with different supposed names attached. These were posted on social media, the second name is the real person. The point is that confirming that these are photos of the same person proved that Surefire's "employee" was fake.

How to Use Face++

Th Face++ tool avoids the messy and subjective process of personally examining faces in two photos by providing a "confidence" score of 1 to 100. The first step is to signup for a free account at faceplusplus.com, The tool lets you use photos that are posted online or photos saved to your computer.

If you are comparing photos posted on the internet, you can use the script below that is also posted here on Github. This a script that uses two examples of matching photos.

#Face_Comparison_URLs 
#this script is for using the Face++ api to compare two faces in images from two URLs

import requests
import json

#You obtain an 'api key' and a 'api secret' from faceplusplus.com when you sign up for a free account

params = {'api_key': 'PASTE YOUR API KEY HERE', 
    'api_secret' : 'PASTE YOUR API SECRET HERE', 
    'image_url1' : 'https://pyxis.nymag.com/v1/imgs/c71/fb1/c5e5566dc3a6fe3db549e6042becb92415-04-charlize-theron.rsquare.w330.jpg', 
    'image_url2' : 'https://4.bp.blogspot.com/-CbRqTz_mINY/T8SkVwME65I/AAAAAAAATqU/A1o6qgabPF4/s1600/Charlize+Theron.jpg'}

#right-click on any image on the internet, click 'copy image 
#address', or right click on the image and click 'open in new tab' 
#and then right-click on the image and click 'copy image address' and 
#paste it between parens next next to image_url1 or image_url2. Above 
#there are two example urls for photos of Charlize Theron.

api_url = 'https://api-us.faceplusplus.com/facepp/v3/compare'

#the api url was identified in the documentation from faceplusplus
# the terms of all of the params were identified in the documentation 
#from faceplusplus.com

r = requests.post(api_url, params)
response = json.loads(r.text)

#json.loads outputs the response from the api into a more readable 
#format of key value pairs, the main key of interest is named #'confidence' because it scores the probability that the
#faces are a match.

x = response['confidence']
print(x)

This is the same script as above, but with photos that do not match

#Nonmatching_Face_Comparison_URLs 
import requests
import json
params = {'api_key': 'YOUR UNIQUE API_KEY', 
'api_secret' : 'YOUR UNIQUE API_SECRET', 
'image_url1' : 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTE3SA1z6KAuQPLHNiQoF9dlp2HwHL9dmV_Lw&usqp=CAU', 
'image_url2' : 'https://widgetwhats.com/app/uploads/2019/11/free-profile-photo-whatsapp-4.png'}
api_url = 'https://api-us.faceplusplus.com/facepp/v3/compare'
#the api url was identified in the documentation from faceplusplus
#the terms of all of the params were identified in the documentation from faceplusplus
r = requests.post(api_url, params)
response = json.loads(r.text)
#json.loads outputs the response from the api into a more readable format of key value pairs, one key
#is confidence
x = response['confidence'] , response['thresholds']
print(x)

This is a different script, it is for comparing photos from two files on your computer rather than comparing photos from urls. The scrip is posted here on Github.

#Face_Comparison_Files
#This script is for uploading two jpg files to faceplusplus to compare the faces in them. The purpose in using files is so that you can download copies of photos from the internet and crop them to exclude other faces in the photos, or possibly improve your results.
import requests
import json
params = {'api_key': '', 
'api_secret' : '', }
api_url = 'https://api-us.faceplusplus.com/facepp/v3/compare'
#the api url was identified in the documentation from faceplusplus
#the terms of all of the params were identified in the documentation 
#from faceplusplus.
#make sure the files are saved to the same folder as the Python script.
r = requests.post(api_url, params)
files = {'image_file1': ('croppedimage1.jpg', open('croppedimage1.jpg', 'rb')), 'image_file2': ('croppedimage1.jpg', open('croppedimage1.jpg', 'rb'))}
r = requests.post(api_url, params, files=files)
response = json.loads(r.text)
x = response['confidence'] , response['thresholds']
print(x)

Example For Photos That Do Match

Now lets try the very first script at the top of this post, #Face_Comparison_URLs. The urls in that script are for the following photos:

Using the Face++ script on these photos gives us a confidence score of 87%. This tells us that with an 87% confidence we can expect that the photos are for the same person.

Example For Photos That Do Not Match

Lets try out the scripts and see how well Face++ works. First lets check out to photos of different people. We find two photos online at urls (https://images.pexels.com/photos/220453/pexels-photo-220453.jpeg?auto=compress&cs=tinysrgb&h=750&w=1260) and (https://widgetwhats.com/app/uploads/2019/11/free-profile-photo-whatsapp-4.png).

The photos are of two obviously different people but we must appreciate the similarities. The photos are both profile photos and they are of two young white men.

We use the script above that has the title, #Nonmatching_Face_Comparison_URLs. Running the script gives me a confidence score of 56%. So we know that two completely different people with some similarities gives us this level of confidence.

I found two more random photos of two different but vaguely similar looking guys in profile and I get the same confidence score.

Effect of Cropping the Photos

Interesting, I tried downloading the photos as files so that I could crop the images to just show the faces with nothing else in the photos that might affect the confidence score. I used the second script in this post  (the one that was written for uploading and comparing photos) for comparing the cropped photos.

Interestingly, cropping the two photos of different people increased the confidence score from 56% to 59%. However, when I cropped the two photos of the same person, the confidence score remained the same, 87%.

How does this work? Python and APIs

To start with, if you are completely new you can download Python from https://www.python.org/downloads/. Then you can download Sublime Text, a tool for accessing Python scripts, at  https://www.sublimetext.com/3.

Now, access your Command Line (if you are using Microsoft) or Terminal (if you are using Mac).

Next we want to get a new python library called "requests" and we will do so by using pip. Pip is included in Python but you can see guidance for installation and updating at https://pip.pypa.io/en/stable/installing/.

To obtain "requests" (if you are using Windows) from the command line, type "python -m pip install requests".

If you are using Mac, from Terminal type "pip install requests".

The next step is to open Sublime Text, then open a new file. Copy and paste the first script above (Face_Comparison_URLs) into the file.

Now you need to have a Face++ account. This takes only a little time. Go to Face Comparing at https://www.faceplusplus.com/face-comparing/. Scroll down to where it says Comparing Api and click to the write where it says Start Free. You will want a free account for the face comparing API.

Once you are signed into your account, click on Apps and then API Key to see your api keys that will be identified as api_key and api_secret.

Now paste those keys into your script and then paste in the urls for the photos you want to compare. remember that you need to right click on any photo and copy image address or open it in a new tab and then copy that url that appears for the new tab.

When your script has the api keys and then image urls you save it by clicking on "file" and then "save". Finally, you run the script by clicking "tools" and then clicking "build" (it is not intuitive). A  message should show up with a number saying the confidence level and the time necessary to run the script.

Success.

About The Art Used In This Article

The awesome artwork used in this article is the work of Zaki Abdelmounim. He is a 25-year-old Moroccan 3D generalist living in Qatar and working in the TV industry.  This project is called Hardcoding:Redshift Study, Zaki started the project after watching the movie Chappie, he really liked the concept of the command chair that was done by George Hull, and wanted to recreate something similar in 3D for the sake of practice and fun, resulting in what we think is the worlds best hacker desk design. Learn more about this project here.