OSINT In Penetration Testing

During penetration testing, pentesters have to work with large amounts of information. Finding this information can be done using manual command-line methods. Doing it manually can take up lots of time as you’d also have to sort this data by yourself because it might not be in a preferable format. The second option is relying on open-source intelligence, or OSINT, which is the go-to method for most pentesters nowadays.

Tools that simplify OSINT gathering are very beneficial for any pentester as they speed up and simplify workflow. Before acquiring any of these tools, it’s good to have a solid understanding of what OSINT is and how it is used in penetration testing.

What is Open Source Intelligence?

Open-source intelligence is information that is available to the general public. Whenever you ask any search engine a question or explore Google Maps, you’re using open-source intelligence. Similarly, security e experts, business analysts, and others can utilize open sources to gather valuable information for their company.

With this came the emergence of OSINT tools dedicated to data collection in its various forms; texts, files, images, satellite imagery, etc.

According to the CSR Report for Congress, OSINT has been produced from the public information that is correctly disseminated, collected, and exploited effectively. This information is available to users to address specific intelligence requirements.

Why Pentesters Require Such Tools

If a pentester needs to find data for a website or infrastructure, searching and sorting data manually can be tedious; taking up lots of time and resources. New OSINT services can efficiently find the most important data they might need (IPv4 hosts, subdomains, DNS records, WHOIS data and more) from all layers of the internet, and after enriching it, provide it in a sorted and accessible format.

Many bug bounty programs do not allow the use of certain types of scanners, which makes the process of finding possible vulnerabilities more complicated. In this case, services like Spyse are the go-to method as they can scan and collect all the necessary information in seconds.

For pentesters, it's essential to have a tool like Spyse in their kit. Companies often let private information leak into the public eye, and attackers can use this information. Employees with proper security training can make public some information that could potentially harm the company. Using OSINT based tools, white hat pentesters can detect that information and similar vulnerabilities before attackers exploit them. This greatly improves the security level of any company.

Other Uses of OSINT Tools

A common pool of information that pentesters and attacks go to are domains and subdomains of organizations. These organizations can have different Top Level Domains or TLDs, as well as auxiliary businesses. A company can have a .com TLD, or others like .net., .co, or .xxx.

This is where Sublist3r comes into play. With the help of this service, you can easily map the attack perimeter, find connections between companies, find all subdomains of a domain, and more. Sublist3r enumerates subdomains using many search engines such as well as Netcraft, Virustotal, ThreatCrowd, DNSdumpster, and ReverseDNS. Spyse, in turn, works using self-developed OSINT techniques, which are unique in their simplicity and resourcefulness. They are currently developing more features and collecting user feedback — be sure to try their service for free and drop them a note.