Finding OSINT Eggs In Mobile Apps

For a while, knowing how to view the source of a page or using developer tools was enough to access “hidden” treasures. But what solutions are available for smartphone apps for which these functions are not available? Lets find out!

A lot of tips and tricks for collecting open source information involve some sort of scrolling though the source code of a page. Think Facebook User ID or the URL for an image you can’t click. Being able to code in HTML is not necessary but knowing how to “view the source” and search for strings is a valuable skill all OSINTers should possess.

In fact, this is quite easy: on Windows machines, in every major browsers, simply press Ctrl+Shift+U to open a new tab that will show the raw HTML of the page you’re on and in MacOS, press Cmd+Option+U or Cmd+U for Firefox). Right click also works.

When viewing the source is not enough, browsers have tools designed for debugging web pages that we can use. To circumvent Instagram protection against unlogged browsing, for example. Enter the developer tools. You can access them by pressing F12 or Ctrl+Shift+I in your favorite browser.

I can’t really describe every tool available, not because I am lazy but because I do not know how to use them all. There are some good tutorials that will teach you how to use the tools in different browsers and even if you will probably not need every aspect, it is always interesting to see how the magic works behind a well designed page. You can start with OSINT Curious and their 10 Minute Tip: Browser Developer Tools. You can then go further with this Google Chrome Developer Tools Crash Course and watch some applied knowledge with Justin Seitz in his Advanced Website Analysis for OSINT video.

Interesting fact, you can also use the developer tools in Chrome to debug “remote” Android devices browser. All you need is an USB cable, Android Debug Bridge (adb) drivers for your main machine, any specific driver for the phone and Chrome on both device. You can find a nice tutorial on Google web developers page. I’m still not sure how I would use this feature in my OSINT work and research though…

Developer tools also give you the possibility to simulate a mobile device and rendering a website as such. This function is called “Toggle device toolbar”.

“Time to go mobile”

People are moving away from their laptop and desktop and spend more and more time on mobile devices. And more and more time on apps (see for example “US Time Spent with Mobile 2019”). We are probably not living the death of the browser yet and even if a lot of mobile applications have a web interface we can play with as investigators, that’s something we have to prepare for when we are doing OSINT investigations or research.

And let’s also note that some developers decide to NOT include a web interface. This is especially the case for social media apps and messaging apps like Snapchat or TikTok. Apps that are extremely popular and therefore extremely interesting for OSINT investigators.

So, what can you do when you have to investigate something that’s only available in a mobile application (or desktop application actually)?

First, Google it! The OSINT/Infosec community is usually sharing a lot of the knowledge and if someone ever tried to “hack” a way outside of the app, you will probably find it. Also, be sure to check the developers announcements for the app. Check if they made some features available through browsers, check for APIs or betas. You may also have a look at users subreddits or forums for tricks.

Still nothing? Let the fun begin!

From what we can read online, most mobile apps use HTTP(S) protocol to communicate with their servers. TikTok uses HTTP, Snapchat uses HTTPS, Tinder uses a mix of HTTPS and HTTP. You get the idea.

The nice thing with HTTP is it’s a protocol you can easily proxy. A proxy server is […] “is a server that sits in front of a group of client machines. When those computers make requests to sites and services on the Internet, the proxy server intercepts those requests and then communicates with web servers on behalf of those clients, like a middleman” (source: https://www.cloudflare.com/learning/cdn/glossary/reverse-proxy/)

There are many proxy servers you can use on your machine: WinGate or FreeProxy on Windows, SquidMan on MacOS. But there’s another solution that will combine the power of a proxy server and developer tools: Fiddler, “the free web debugging proxy for any browser, system or platform”, created in 2003 by Eric Lawrence.

Fiddler

First things first: install Fiddler on the computer you want to use as a proxy/analysis computer. Then, grab the smartphone or start the virtual machine you will use for playing with the apps. Both systems have to be on the same network.

My setup is simple: I use an Android phone with 4G connection and I connect my “Fiddler laptop” and the Android testing phone via WiFi to it.

Next step is to find the IP address of the machine that will serve as a proxy. On a Windows command line, type ipconfig In a MacOS / Linux terminal, ifconfig .

Write the IP address down. Or type it in a notepad. Or take a picture of it. Or anything.

Then run Fiddler, go to “Tools”, “Options…” and click on the “Connections” tab. Check the “Allow remote computers to connect” box. Notice that Fiddler will be listening on port 8888. You have a proxy server!

Next you have to configure your phone/virtual machine to use the proxy. On my old Samsung, I open the settings, tap “Wi-Fi”, tap and hold the connection I am using, select “Modify network config”, tick “Show advanced options”, scroll down and choose “Manual” under “Proxy”. It’s where I enter the IP address I found on my computer. I also enter the port, 8888.

Sorry but there’s a last step. You will need to do this one to decrypt any HTTPS traffic. Switch back to Fiddler and on the lower left corner, you will find a black box. It is the “QuickExec” box. Type the following command in this box and press “Enter”.about:config

A list of variables will appear on the right pane. Add the following variable:

fiddler.certmaker.ValidDays

and give it a value of 820. This will change the expiry date of the certificate you will install next. We are doing that to avoid some errors on Android.

Then let’s install the certificate on the phone. Launch a browser and point it to http://ipv4.fiddler:8888.

Download and install the FiddlerRoot certificate. You are done, now every HTTP(S) traffic from your phone should pass through Fiddler. By the way, all your HTTP(S) traffic for the machine will also pass through Fiddler!

Check the setup by browsing some websites on the phone and watching the result on Fiddler.

Now that the proxy is working, let’s tweak Fiddler to filter the traffic we don’t want to see. Go to the “Filters” tab and on the “- No Host Filter -” drop down list, choose “Hide the following Hosts”. Enter the proxy IP address, tick the “Use Filters” checkbox and on “Actions”, choose “Run Filterset now”.

Again, I am not a web developer or an application debugger so the idea now is to keep it simple, like some low level reverse engineering. Clear the sessions Fiddler have collected so far by clicking the “X” on the menu bar.

Launch the app you want to investigate. Let’s start with TikTok as an example. Launching the app shows multiple connections to api2–16-h2.musical.ly and other subdomains on the musical.ly domain. Choose the “Inspectors” tab and go to the “TextView”. When you select a session, Fiddler will decode and show it as text.

My advice if for you to start with a profile you already know: one of your own account where you can easily find the information you already possess. Because it’s more fun, for this article let’s search for “Loren Gray”, the “most successful account outside of China” (yes, I had to look that up). Searching for this user will populate Fiddler with calls to different URLs. Let’s keep going until we are on the actual profile page.

Depending on what your phone is doing at the time you scroll to the profile, you may have to search for the right session amongst dozens of sessions. One easy way of doing that is switching to “ImageView” and go up and down the sessions until you find some of the pictures you see on the phone. The actual profile session is usually nearby!

For me, it was 6 sessions up. I have the same domain, api2–16-h2.musical.ly, an URL with some user and profile in it. Sounds good. The response is even better: it’s JSON formatted (you can switch to the “JSON” tab for a more comfortable view) and contains interesting fields like:

user
    uid=934514
    unique_id=lorengray    avatar_larger
      url_list
        http://p16.muscdn.com/img/musically-maliva-obj/1660343861335046~c5_1080x1080.jpeg    video_icon
      url_list
        http://p16.muscdn.com/img/musically-maliva-obj/1660344574058502~tpl-1_q65.webp    youtube_channel_id=UCRYe4G08okte96rmocFNU1g
    youtube_channel_title=Loren Gray

We already have some interesting information here, including a link to a YouTube channel, a large avatar picture URL, a video icon URL. Some of these elements were not available on the profile page of the app or the web interface (yes, TikTok has a web interface!). And this JSON gives us access to the URL of the video icon that was not even visible on the web profile:

The next session in the list is another JSON response but way larger. It contains a aweme_list field with descriptive elements for every TikTok videos that appeared on the profile screen.

Now you know that the TikTok app calls api2–16-h2.musical.ly to get responses in JSON. You know that the username is called unique_id and that the images are stored on http://p16.muscdn.com/img in jpeg od webp format. You can use this information and go on with your research, viewing videos, scrolling through likes, followers and following to see how the application handles the request.

Maybe we can try to see if api2–16-h2.musical.ly is answering as an API?

The Party Is Over

Now, let’s head over Snapchat to see how this other app works.

First, I will quit TikTok, wait for the calls to TikTok servers to stop and clean the sessions by pressing the “X” and choosing “Remove all”. When everything is ready, I will launch Snapchat.

Snapchat makes a lot of requests to *.snapchat.com HTTP(S) servers. I will use the same routine and go to the search bar. This time, I will try to find the “King of Snapchat”, DJ Khaled. But I can’t. The app tries to connect to http://aws.api.snapchat.com:443 but there’s no response. Something is preventing it from accessing the servers: Fiddler.

Snapchat and a lot of other applications are using a system to protect the user from “man-in-the-middle” interception and decryption called certificate pinning.

In summary, the app only trusts specific certificates. Inserting a non-trusted certificate in the system like the one provided with Fiddler doesn’t work, the app will not establish the connection.

Can you try to bypass it? Yes, tools like Frida will help but this goes beyond the scope of this article.

Bad news is, a lot of apps have implemented certificate pinning or other measure to prevent a proxy from working on HTTPS traffic.

Good news is, certificate pinning prevents white hat hackers and researchers from finding bugs in applications. Therefore, app developers have introduced a “Whitehat Settings” to allow HTTPS traffic interception and decryption.

Facebook rolled these settings for the Facebook app, Messenger and Instagram.

Conclusion

Apps are already part of the OSINT landscape and proxying traffic from a smartphone to a server may be a nice and easy way to find some treasures. As OSINT investigators/researchers, we will have to find new techniques for gathering information about users on apps that more than often contain a massive amount of intelligence.