Online Research Tools

    العربية |  বাংলা | Español | Français

Online Research Tools and Investigative Techniques by the BBC’s ace online sleuth Paul Myers has long been a starting point for online research by GIJN readers. His website, Research Clinic, is rich in research links and “study materials.” Here’s a tipsheet about finding people online that Myers presented at a 2019 GIJN webinar. And a GIJN article about his presentation at GIJC19, 4 Questions for Online Super-Sleuth Paul Myers.

Finding People Online: A Tipsheet From Paul Myers

See also Myers’ other guides on gijn.org:

Using Twitter to Find People at the Scene of a Breaking Story

Customise your Browser: Using Add-ons for your Web Research

Browser Add-ons (Part 2): Traveling Back in Time

GIJN’s Investigative Toolbox, a column by GIJN’s Alastair Otter, explores selected topics:

Tracking Names and Websites, Verifying Video, a Clustering Search Engine

Digging for People, Trawling the Web and Keeping Yourself Safe

Backgrounding People and Companies

Mine Twitter and Monitor Website Updates

Beyond Spreadsheets and Deep Searching the Web

Jake Creps keeps up a steady stream of useful tips on his blog.  He also hosts a podcast, osintpodcast.com. Creps is an open source intelligence analyst with experience in the public and private sector.

OSINTcurio.us features weekly podcasts, webcasts and “10 minute tips” on video covering many aspects of doing open source investigations. It’s a community project begun in late 2018 by about 10 contributing experts.

Online Methods to Investigate the Who, Where, and When of a Person. Another great list by Internet search expert Henk Van Ess. Also see Who Posted What, “a Facebook keyword search generator with which you can search for posts on Facebook who have they keyword in it, on a specific date or between two dates,” a Van Ess idea developed by Daniel Endresz.

The Open Source Intelligence Framework is a very detailed and ever-growing list of digital investigative tools.

Searching the Deep Web, by Giannina Segnini. Beginning with advanced tips on sophisticated Google searches, this presentation at GIJC17 by the director of Columbia University Journalism School’s Data Journalism Program moves into using Google as a bridge to the Deep Web using a drug trafficking example. Discusses tracking the container, the ship, and customs. Plus, Facebook research and more.

How to Become a Deep Web Super Sleuth. This GIJN article describes advice given by Albrecht Ude, a German journalist, researcher, and trainer at the 11th Global Investigative Journalism Conference. “Search engines are completely useless for finding any content on the deep web,” said Ude, who’s first bit of advice is to think abstractly, followed by more tips. Also see his related tipsheet: Finding and using databases. GIJN wrote about his presentation here.

OSINT Essentials by Eoghan Sweeney provides links to mostly free tools and services that are useful for online verification, digital journalism, and open-source intelligence work.

Tools, Useful Links & Resources, by Raymond Joseph, a journalist and trainer with South Africa’s Southern Tip Media. Six packed pages of information on Twitter, social media, verification, domain and IP information, worldwide phonebooks, and more. In a related GICJ17 presentation, Joseph described “How to be Digital Detective.” 

Knowing Your Way Around The Command Line, by Max Harlow, Senior Newsroom Developer at The Financial Times, was presented at GIJC19. What’s possible? He summarizes: Find who owns a website, Analyse data at lightning speed, Match up names written differently, Link up your data to online databases, Bulk search over hundreds of documents.

AML RightSource, a private US firm “solely focused on Anti-Money Laundering (AML)/Bank Secrecy Act (BSA) and financial crimes compliance solutions,” put together a packed sheet of resources.

Smart searching with googleDorking, a guide from Exposing the Invisible.

Investigate with Document Cloud, by Doug Haddix, Executive Director, Investigative Reporters and Editors. A guide to using 1.6 million public documents shared by journalists, analyzing and highlighting your own documents, collaborating with others, managing document workflows and sharing your work online.

Malachy Browne’s Toolkit. More than 80 links to open source investigative tools by one of the best open-source sleuths in the business. When this New York Times senior story producer flashed this slide at the end of his packed GIJC17 session, nearly everyone requested access.

An Investigative Guide to LinkedIn created by Nathan Patin for Bellingcat in 2019. “This guide aims to provide helpful tools and techniques for identifying LinkedIn profiles and for extracting information that will then allow you to pivot to other social media profiles belonging to the target.”

A Guide to Open Source Intelligence (OSINT) by Michael Edison Hayden for the Tow Center for Digital Journalism at the Columbia Graduate School Journalism School (2019) covers a variety of topics:

  • The Difference between Open and Closed Networks
  • Searching the Open Web
  • Verifying the Authenticity of Social Media Accounts
  • Verifying Images and Videos
  • Exploring Fringe Websites
  • Using Archives, Saving Your Work
  • Learning New Platforms and Interacting with Hostile Communities

Social Media Sleuthing by Michael Salzwedel. “Not Hacking, Not Illegal,” begins this presentation from GIJC17 by a founding partner and trainer at Social Weaver.

Finding Former Employees, by James Mintz. “10 Tips on Investigative Reporting’s Most Powerful Move: Contacting Formers,” according to veteran private investigator Mintz, founder and president of The Mintz Group.

Tools and tips for digging into Facebook from two investigative journalists A 2019 interview with Brooke Williams, an award-winning investigative reporter and Associate Professor of the Practice of Computational Journalism at Boston University and Henk van Ess, lead investigator for Bellingcat.com.

Investigative Research Links from Margot Williams. The former research editor at The Intercept offers an array of suggestions, from “Effective Google Searching” to a list of “Research Guru” sites. Also see her GIJC19 tipsheet: Research Plan for Investigative Project.

Bellingcat’s Online Investigative Toolkit  includes a wide variety of resources: on maps, geo-based searches, images, social media, transport, data visualization, experts and more.

Bureau Local collaborative investigative journalism tools is a crowd-sourced spreadsheet of more than 80 listings, some of them research tools, sponsored by the UK-based Bureau of of Investigative Journalism.

Tools for Reporters by Samantha Sunne, provides regular e-mails on new tricks, such as Finding emails when they don’t want to be found.

Fundamental search for journalists This series of online classes from Datajournalism.com, taught by Vincent Ryan, covers advanced searching, verification, visualization and geo-mapping.

List of Tools for Social Media Research, a tipsheet from piqd.de’s Frederik Fischer at GIJC15.

SPJ Journalist’s Toolbox from the Society of Professional Journalists in the US, curated by Mike Reilley. Includes an extensive list of, well, tools.

IntelTechniques is a site by Michael Bazzell, a former US government computer crime investigator and now an author and trainer. It hosts his podcast, The Privacy, Security, & OSINT Show.

How to find an academic research paper, by David Trilling, a staff writer for Journalist’s Resource, based at Harvard’s Shorenstein Center on Media, Politics and Public Policy.

World.192.com has a list of international telephone directories.

Using Phone Contact Book Apps For Digital Research, an instruction 2019 posting by Aric Toler of Bellingcat.

Use Chrome Developer Tools to View Masked Phone Numbers for Free on a Popular People Search Site, by Sean Lawson, a University of Utah professor, was published in 2019.

Using deep web search engines for academic and scholarly research, an article by Chris Stobing in VPN & Privacy, a publication of Comparitech.com, a UK company that aims to help consumers make more savvy decisions when they subscribe to tech services such as VPNs.

Step by step guide to safely accessing the darknet and deep web, an article by Paul Bischoff in VPN & Privacy, a publication of Comparitech.com, a UK company that aims to help consumers make more savvy decisions when they subscribe to tech services such as VPNs.

Research Beyond Google: 56 Authoritative, Invisible, and Comprehensive Resources, a resource from Open Education Database, a US firm that provides a comprehensive online education directory for both free and for-credit learning options.

The Engine Room, a US-based international NGO, created an Introduction to Web Resources, that includes a section on making copies of information to protect it from being lost or changed.

Awesome Public Datasets, a very large community-built compilation organized by topic.

Resources from Startme.com A multitude of resources from a company that created an integrated bookmarks manager. The free library contains pages with hundreds of links:

Addons
Databases
Search Engines
Tools
Verification Toolkit
Tutorials

Databases, for example, is a page with links to a wide range of material, about stolen property, weather, hacking, media, traffic, analytics, domain registration, buildings, animals, drones, finance and more.

Bates InfoTips, by Mary Ellen Bates, is a good source of up-to-date ideas, such as Limiting Google News searches by time and Finding people through Facebook Graph Search. Bates is ” founder and principal of Bates Information Services Inc. and a long-time info geek.”

Research Buzz wrote Going Old School to Solve A Google Search Problem to describe a method to exclude certain sites from search results.

MakeUseOf regularly features good tips, such as 4 Anonymous Web Browsers That Are Completely Private.

6 Unique & Free Keyword Research Tools You Didn’t Know You Needed, from Search Engine Journal.

This Tool Shows Exposed Cameras Around Your Neighbourhood, an article in Motherboard explaining how Kamerka can take an address, landmark, or coordinates and display exposed internet connected cameras on a map.

Open-Source Intelligence (OSINT) Reconnaissance, a long article by “researcher-writer-hacker-tinkerer” Ian Barwise in Medium, covers techniques such as “Google dorking.”

After the GDPR: researching domain name registrations  A look at the impact of the European General Data Protection Regulation, which makes researching domain name registrations harder, necessitating different tools and techniques, according to OSINTCurious.

Using the New Russian Facial Recognition Site SearchFace.ru, a 2019 description from Bellingcat.

Mapping Social Media with Facial Recognition by Jacob Wilkin of Spiderlabs Blog in 2019.

How to Write Facebook Graph Search Queries a 2019 WikiHow guide on using Facebook’s graph search syntax to find posts, comments, people, and other entities on Facebook.

The Most Comprehensive TweetDeck Research Guide in Existence (Probably) by Charlotte Godart, an investigator and trainer for Bellingcat (2019).

Google Search Operators: The Complete List (42 Advanced Operators), from Joshua Hardwick (2018).

Finding Hidden Business Resources A presentation at the Special Libraries Association 2019 conference by Mary Ellen Bates on finding materials on the “deep web.”

Our Search for the Best OCRTool, and What We Found, by Ted Han and Amanda Hickman in Source (2019).

DuckDuckGo vs. Google: An In-Depth Search Engine Comparison published by Search Engine Journal in 2019.

How To Find Old Websites That Google Won’t Show, Lifehacker (2019).

Instagram Location Search See Instagram description of other search options.

How to Identify an Unknown Person Using a Photograph by Jan Tegze (2019), author of the Sourcing.GamesRecruitment.CampSourcingTest.Online and blogger.