An Introduction To Open Source Intelligence (OSINT) Gathering

The revolution of the Internet has turned the world into a small village. Unleashing the Internet network to billions of people worldwide to communicate and exchange digital data has shifted the entire world into what is now an information age. Open Source Intelligence (OSINT) refers to all the publicly available information.

There is no specific date on when the term OSINT was first proposed; however, a relative term has probably been used for hundreds of years to describe the act of gathering intelligence through exploiting publicly available resources.

In recent history, OSINT was introduced during World War II as an intelligence tool by many nations security agencies, however, with the explosive growth of the internet communications and the huge volume of digital data produced by the public worldwide, OSINT gathering becomes a necessity for different organizations, for instance, government departments, nongovernmental organization, (NGO) organizations, and business corporations are starting to rely to a large extent on OSINT rather than private and classified information.

OSINT sources are distinguished from other forms of intelligence because they must be legally accessible by the public without breaching any copyright or privacy laws. This distinction makes the ability to gather OSINT sources applicable to more than just security services. For example, businesses can benefit from exploiting these resources to gain intelligence about their competitors.

OSINT types

OSINT includes all publicly accessible sources of information. This information can be found either online or offline:

1. The Internet, which includes the following and more: forums, blogs, social networking sites, video-sharing sites like YouTube.com, wikis, Whois records of registered domain names, metadata and digital files, dark web resources, geolocation data, IP addresses, people search engines, and anything that can be found online.
2. Traditional mass media (e.g., television, radio, newspapers, books, magazines).
3. Specialized journals, academic publications, dissertations, conference proceedings, company profiles, annual reports, company news, employee profiles, and résumés.
4. Photos and videos including metadata .
5. Geospatial information (e.g., maps and commercial imagery products)

OSINT Organizations

Some specialized organizations provide OSINT services. Some of them are government based, and others are private companies that offer their services to different parties such as government agencies and business corporations on a subscription basis. The following are the most well-known OSINT organizations:

Government Organizations

1. Open Source Center: controlled by U.S. government
2. BBC Monitoring (https://monitoring.bbc.co.uk/login) is a department within the British Broadcasting Corporation (BBC) that monitors foreign media worldwide. BBC Monitoring is directed by the BBC and offers its services on a subscription basis to interested parties such as commercial organizations and UK official bodies.

Private Sector

1. Jane’s Information Group (http://www.janes.com) is a British company founded in 1898. Jane’s is a leading provider that specializes in military, terrorism, state stability, serious and organized crime, proliferation and procurement intelligence, aerospace, and transportation subjects
2. The Economist Intelligence Unit (https://www.eiu.com/home.aspx) is the business intelligence, research, and analysis division of the British Economist Group.
3. Oxford Analytica (http://www.oxan.com) is a relatively small OSINT firm compared with the previous two. Oxford Analytica specializes in geopolitics and macroeconomics subjects.

Parties Interested in OSINT Information

OSINT can be beneficial for different actors. We will briefly list them and mention what motivate each one to search for OSINT resources.

1. Government: Government bodies, especially military departments, are considered the largest consumer of OSINT sources. Governments need OSINT sources for different purposes such as national security, counterterrorism, cybertracking of terrorists, understanding domestic and foreign public views on different subjects, supplying policy makers with required information to influence their internal and external policy, and exploiting foreign media like TV to get instant translations of different events happening outside.
2. International Organizations: International organizations like the UN use OSINT sources to support peacekeeping operations around the globe. Humanitarian organizations, like the International Red Cross, use OSINT sources to aid them in their relief efforts in a time of crisis or disaster. They use OSINT intelligence to protect their supply chain from terrorist groups by analyzing social media sites and Internet messaging applications to predict future terrorist actions.
3. Law Enforcement Agencies: Police use OSINT sources to protect citizens from abuse, sexual violence, identity theft, and other crimes. This can be done by monitoring social media channels for interesting keywords and pictures to help prevent crimes before they escalate.
4. Business Corporations: Information is power, and corporations use OSINT sources to investigate new markets, monitor competitors’ activities, plan marketing activities, and predict anything that can affect their current operations and future growth.
   Businesses also use OSINT intelligence for other nonfinancial purposes such as the following:
   A. To fight against data leakage, knowing that the business exposure of confidential information and the security vulnerabilities of their networks is a cause of future cyber-threats.
   B. To create their threat intelligence strategies through analyzing OSINT sources from both outside and inside the organization and then combining this information with other information to accomplish an effective cyber-risk management policy that helps them to protect their financial interests, reputation, and customer base.
5. Penetration Testers and Black Hat Hackers/Criminal Organizations: OSINT is used extensively by hackers and penetration testers to gather intelligence about a specific target online. It is also considered a valuable tool to assist in conducting social engineering attacks. The first phase of any penetration testing methodology begins with reconnaissance (in other words, with OSINT).
6. Privacy-Conscious People: These are ordinary people who might want to check how outsiders can break into their computing devices and what their ISP knows about them. They also need to know their online exposure level to close any security gap and delete any private data that may have been published inadvertently. OSINT is a great tool to see how your digital identity appears to the outside world, allowing you to maintain your privacy. Individuals can also use OSINT to fight against identity theft, for example, in case someone is impersonating you.
7. Terrorist Organizations: Terrorists use OSINT sources to plan attacks, collect information about targets before attacking them (like when using satellite images such as Google Maps to investigate the target location), procure more fighters by analyzing social media sites, acquire military information revealed accidentally by governments (like how to construct bombs), and spread their propaganda across the world using different media channels.

Information Gathering Types

OSINT sources can be collected using three main methods: passive, semi-passive, and active. The usage of one in favor of another is dependent on the scenario in which the gathering process operates in addition to the type of data that you are interested in.

Passive Collection

This is the most used type when collecting OSINT intelligence. Indeed, all OSINT intelligence methods should use passive collection because the main aim of OSINT gathering is to collect information about the target via publicly available resources only.

Semi-passive

From a technical view, this type of gathering sends limited traffic to target servers to acquire general information about them. This traffic tries to resemble typical Internet traffic to avoid drawing any attention to your reconnaissance activities. In this way, you are not implementing in-depth investigation of the target’s online resources, but only investigating lightly without launching any alarm on the target’s side.

Active Collection

In this type, you interact directly with the system to gather intelligence about it. The target can become aware of the reconnaissance process since the person/entity collecting information will use advanced techniques to harvest technical data about the target IT infrastructure such as accessing open ports, scanning vulnerabilities (unpatched Windows systems), scanning web server applications, and more. This traffic will look like suspicious or malicious behavior and will leave traces on the target’s intrusion detection system (IDS) or intrusion prevention system (IPS). Conducting social engineering attacks on the target is also considered a type of active information gathering.

Benefits of OSINT

The benefits of OSINT span many areas in today’s world. The following are the main ones:

1. Less risky: Using publicly available information to collect intelligence has no risk compared with other forms of intelligence such as using spying satellites or using human resources on the ground to collect information, especially in hostile countries.
2. Cost effective: Collecting OSINT is generally less expensive compared with other intelligence sources. For instance, using human resources or spying satellites to collect data is costly.
3. Ease of accessibility: OSINT sources are always available, no matter where you are, and are always up-to-date.
4. Legal issues: Most OSINT resources can be shared between different parties without worrying about breaching any copyright license as these resources are already published publicly.
5. Aiding financial investigators: OSINT allows specialized government agencies to detect tax evaders, for instance. Many famous celebrities and some giant companies are involved in tax evasion, and monitoring their social media accounts, vacations, and lifestyles has a great value for a government inspector who may be chasing them for undeclared income.
6. Fighting against online counterfeiting: OSINT techniques can be used to find false products/services and direct law enforcement to close such sites or to send warnings to users to stop dealing with them.
7. Maintaining national security and political stability: This might be the most important role of OSINT; it helps governments to understand their people’s attitudes and to act promptly to avoid any future clashes.

Challenges of Open Source Intelligence

All intelligence gathering methodologies have some limitations, and OSINT is not exempt from this rule. The following are the main challenges that face OSINT gathering:

1. Sheer volume of data: Collecting OSINT will produce a huge amount of data that must be analyzed to be considered of value. Of course, many automated tools exist for this purpose, and many governments and giant companies have developed their own set of artificial intelligence tools and techniques to filter acquired data. However, the tremendous volume of data will remain a challenge for the OSINT gatherer.
2. Reliability of sources: Bear in mind that OSINT sources, especially when used in the intelligence context, need to be verified thoroughly by classified sources before they can be trusted.
3. Human efforts: As we already mentioned, the sheer volume of data is considered the greatest challenge for OSINT collection. Humans need to view the output of automated tools to know whether the collected data is reliable and trustworthy; they also need to compare it with some classified data (this is applicable for some military and commercial information) to assure its reliability and relevance. This will effectively consume time and precious human resources.

Summary

In this article we tried to shed the light on the essence of OSINT, its types and users, and how it can be used in different contexts by different parties to gain intelligence. In the next series of articles, we delve deeply and demonstrate different techniques and tools that can be used to locate information online.

Final Note: To read more about OSINT gathering methods and tools, you can read author book titled: Open Source Intelligence Methods and Tools: A Practical Guide to Online Intelligence Published by Apress | https://www.apress.com/gp/book/9781484232125 A companion website is also available for this book and can be found at http://www.OSINT.link

Main Image Credit : The awesome piece of artwork used to head this article is called 'Research' and it was created by graphic designer Miroslav Kostic.