Security and Trust in Distributed Systems Master Degree in New Technologies in Computer Science 2022/23 Open Source Intelligence (OSINT) Antonio Ruiz Martínez, Pantaleone Nespoli, Félix Gómez Mármol Outline Part I • What’s OSINT? • OSINT Techniques • OSINT Tools • OSINT Workflows Part II • Maltego – Getting started – Developing a new transform 3What’s OSINT? Cyber Threat Intelligence • Its key mission is to research and analyze trends and technical developments in three areas Cybercrime Hacktivism Cyberespionage Cyber Threat Intelligence • Collection of intelligence using different means • Open Source Intelligence (OSINT) – Data collected from publicly available sources • Social media intelligence – Intelligence gathered from social media sites, using both intrusive or non-intrusive means, from open and closed social networks • Sentiment analysis • Human intelligence – Information collected and provided by human sources • Interrogations and conversations with persons having access to information • Social engineering • Technical intelligence – Collection, evaluation, analysis, and interpretation of scientific and technical information • SIEM (Security Information and Event Management) OSINT Use Cases OSINT Sources • Media – Newspapers, magazines, radio, television, etc. • Internet – Online publications, blogs, discussion groups, social media websites (i.e. – Facebook, Twitter, Instagram, etc.) • Public Government Data – Public government reports, budgets, hearings, telephone directories, press conferences, websites, and speeches • Professional and Academic Publications – Information acquired from journals, conferences, symposia, academic papers, dissertations and theses • Commercial Data – Commercial, financial and industrial assessments and databases • Grey literature – Technical reports, preprints, patents, working papers, business documents, unpublished works, newsletters, etc. OSINT Sources: Media • Income by street name • https://elpais.com/economia/2019/09/11/actualidad/1568217626_928704.html OSINT Sources: Media • Mortality by municipality name • https://elpais.com/elpais/2020/02/05/ciencia/1580906716_232241.html OSINT Sources: Media • Political inclination by municipality name • https://elpais.com/politica/2019/11/10/actualidad/1573410266_570919.html OSINT & Detection Maturity Level (DML) http://ryanstillions.blogspot.com/2014/04/the-dml-model_21.html 12 OSINT Techniques 13 OSINT Techniques • Search Engines • Social Networks • Email Address • Username • Real Name • Location • IP Address • Domain 14 OSINT Techniques: Search Engines 15 OSINT Techniques: Search Engines ❑ “” – Force an exact‐match search – E.g. “Félix Gómez Mármol” ❑ OR – Search for X or Y – The | operator is equivalent – E.g. murcia|heidelberg ❑ AND – Search for X and Y – E.g. murcia AND heidelberg ❑ - – Exclude a term or phrase – E.g. murcia –heidelberg ❑ * – Acts as wildcard – E.g. Félix * Mármol ❑ () – Group terms or search operators – E.g. Félix AND (Gómez|Mármol) ❑ .. – Search for a range of numbers – E.g. 2010..2019 16 OSINT Techniques: Search Engines ❑ filetype: – Restrict results to those of a certain file type (e.g., pdf, ppt, docx, txt, etc.) – ext is equivalent – E.g. filetype:pdf ❑ site: – Limit results to those from a specific website – E.g. site:um.es ❑ intitle: – Find pages with a certain word (or words) in the title – E.g. intitle:heidelberg ❑ inurl: – Find pages with a certain word (or words) in the URL – E.g. inurl:heidelberg ❑ intext: – Find pages with a certain word (or words) somewhere in the content – E.g. intext:heidelberg 17 OSINT Techniques: Search Engines • Search for index directories within the domain um.es – intitle:“index of /” site:um.es • Search for Excel sheets within the domain um.es with the term “salario” – salario site:um.es filetype:xlsx • Search for ftp sites within the domain um.es – site:um.es inurl:ftp -inurl:(https|http) • Search for usernames and passwords – filetype:pwd inurl:(service | authors | administrators | users) “# -FrontPage-” – intitle:“index of” “Index of /” password.txt – filetype:sql “# dumping data for table” “`PASSWORD` varchar” 18 OSINT Techniques: Search Engines • “Term” • Term1 OR Term2 • Term1 AND Term2 • Term1 * Term2 • -Term • +Term • ~Term • #Term • $price • cache:URL • filetype:EXT • site:URL • related:URL • intitle:Term • allintitle:Term • inurl:Term • allinurl:Term • intext:Term • allintext:Term • AROUND(number) • weather:Location • stocks:$TAG • map:Location • movie:Term • Amount in Unit • source:SRC • _ Term • 000..000 • inanchor:Term • allinanchor:Term • blogurl:URL • location:Location • inpostauthor:Term • allinpostauthor:Term • inposttitle:Term • link:URL • info:URL (also id:URL) • daterange:0000-0000 • phonebook:Term Full list of search commands https://ahrefs.com/blog/google-advanced-search-operators 19 OSINT Techniques: Social Networks 20 OSINT Techniques: Social Networks 21 OSINT Techniques: Social Networks • Step 1 → Get your Facebook ID – Hover your cursor over your profile picture As per 2020, this field might be indicated by the referrer_profile_id param 22 OSINT Techniques: Social Networks • Step 2.a → Get the Facebook ID of someone else – Hover your cursor over his/her profile picture ;-) 23 OSINT Techniques: Social Networks • Step 2.b → Get the Facebook ID of someone else – Visit https://whopostedwhat.com/ • Alternative websites – https://findmyfbid.com/ – https://lookup-id.com/ 24 OSINT Techniques: Social Networks Visit: https://graph.tips/beta/ • E.g., photos by José Luján Alcaraz https://www.facebook.com/search/posts/?q=*&epa=FILTERS&filters=ey JycF9hdXRob3IiOiJ7XCJuYW1lXCI6XCJhdXRob3JcIixcImFyZ3NcIjpcIjE5OTI xNDg0MzEwNzQyNzNcIn0ifQ%3D eyJycF9hdXRob3IiOiJ7XCJuYW1lXCI 6XCJhdXRob3JcIixcImFyZ3NcIjpcIj E5OTIxNDg0MzEwNzQyNzNcIn0ifQ BASE 64 {"rp_author": {"name":"author", "args":"1992148431074273"} } 25 OSINT Techniques: Email Address • Is the email address valid? – https://hunter.io • Has the email address been hacked? – https://haveibeenpwned.com 26 OSINT Techniques: Email Address • Check search engines • Check Pipl – https://pipl.com 27 OSINT Techniques: Username • Check availability in social networks • Check availability in domains – https://knowem.com 28 OSINT Techniques: Real Name • Check search engines • Check Pipl – https://pipl.com 29 OSINT Techniques: Real Name • Check social networks • Check genealogy sites 30 OSINT Techniques: Location • Get GPS coordinates from location name – https://www.gps-coordinates.net • Find out location from GPS coordinates • E.g., GPS coordinates for the Faculty of Computer Science at UMU are – 38.023796; -1.17404590000001 31 OSINT Techniques: IP Address • Get location from IP Address – https://www.iplocation.net – https://viewdns.info • Whois, Reverse IP Lookup, Traceroute, etc, 32 OSINT Techniques: IP Address • Whois 155.54.1.1 33 OSINT Techniques: IP Address • Shodan search engine – https://www.shodan.io 34 OSINT Techniques: IP Address • Shodan search keywords examples – webserver, webcam, ssh, telnet, default password, apache, cisco, linksys,… • Shodan search filters ❑ city: – Find devices in a particular city ❑ country: – Find devices in a particular country ❑ geo: – You can pass it coordinates ❑ hostname: – Find values that match the hostname ❑ net: – Filter results by a specific IP range or subnet ❑ os: – Search based on operating system ❑ port: – find particular ports that are open ❑ before/after: – find results within a timeframe 35 OSINT Techniques: Domain • Visualize domain connections – https://www.threatcrowd.org – http://www.visualsitemapper.com 36 OSINT Techniques: Domain • Check DNS and mailservers – http://www.domaincrawler.com – https://who.is/dns 37 OSINT Techniques: Domain • Check DNS and mailservers – https://mxtoolbox.com/NetworkTools.aspx 38 OSINT Techniques: Domain • Check traffic statistics – https://www.alexa.com – https://www.similarweb.com 39 OSINT Techniques: Domain • Check subdomains – https://findsubdomains.com 40 OSINT Techniques: Domain • Check for archives – http://web.archive.org 41 OSINT Tools 42 OSINT Tools Tool License Input Data Platform Maltego MIT Domain, username, url, email image, DNS, IP, location… Metagoofil GNU 2.0 URL and type of file (extension)… The Foca GPL 3.0 Type of file, domain, search engine… Shodan MIT IP, country, protocol, keywords, url, dns… The Harvester GPL 2.0 Domain, search engine,,, Recon-NG GNU 2.0 Domain, special modules for gathering… Spiderfoot GPL 2.0 Domain, username, files, url, email… Intel Techniques N/A (almost) All the above 43 OSINT Tools 44 OSINT Tools: IntelTechniques • https://inteltechniques.com 45 OSINT Tools: OSINT Framework • https://osintframework.com/ 46 OSINT Tools: Aware Online • https://www.aware-online.com 47 OSINT Tools: Maltego • https://www.maltego.com/products 48 OSINT Tools: Maltego • Maltego clients 49 OSINT Tools: Maltego • Maltego concepts – An Entity is represented as a node on a graph and can be anything such as a DNS Name, Person, Phone number, etc. • The Maltego client comes with about 20 entities targeted for use in online investigations, but you can also make your own custom ones – A Transform is a piece of code that takes one entity to another • It does this by querying a data source and returning the results as new entities on your graph • The data sources are places like DNS servers, search engines, social networks, WHOIS information, etc. – Machines chain multiple transforms together to automate common/tedious tasks 50 OSINT Tools: Maltego • Maltego graph 51 OSINT Tools: Maltego • Maltego entities 52 OSINT Tools: Maltego • Maltego transforms 53 OSINT Tools: Maltego • Maltego machines 54 OSINT Workflows 55 OSINT Workflows 56 OSINT Workflows: Email Address • Which new information can I obtain from an Email Address? • Which paths can I follow to reach such new information? 57 OSINT Workflows: User Name • Which new information can I obtain from a User Name? • Which paths can I follow to reach such new information? 58 OSINT Workflows: Real Name • Which new information can I obtain from a Real Name? • Which paths can I follow to reach such new information? 59 OSINT Workflows: Telephone Number • Which new information can I obtain from a Telephone Number? • Which paths can I follow to reach such new information? 60 OSINT Workflows: Domain Name • Which new information can I obtain from a Domain Name? • Which paths can I follow to reach such new information? 61 OSINT Workflows: Location • Which new information can I obtain from a Location? • Which paths can I follow to reach such new information? 62 Bibliography [1] J. Pastor Galindo, P. Nespoli, F. Gómez Mármol, and G. Martínez Pérez, “The not yet exploited goldmine of OSINT: Opportunities, open challenges and future trends,” IEEE Access, vol. 8, no. 1, pp. 10282–10304, 2020 [2] Bazzell, M., “Open Source Intelligence Techniques: Resources for Searching and Analyzing Online Information”, 6th Edition, ISBN 978-1984201577, 2018 [3] J. Pastor-Galindo, F. Gómez Mármol, G. Martínez Pérez, “Nothing to Hide? On the Security and Privacy Threats Beyond Open Data”, IEEE Internet Comput. 25(4): 58-66 (2021) [4] Quick, D., Kim-Kwang, R. C., “Digital forensic intelligence: Data subsets and Open Source Intelligence (DFINT+ OSINT): A timely and cohesive mix”, Future Generation Computer Systems, vol. 78, pp. 558-567, 2018 [5] Mediná Martin, J. H., et al., “Open source intelligence (OSINT) in a Colombian context and sentiment analysis”, Revista Vínculos: Ciencia, tecnología y sociedad, vol. 15, no. 2, pp. 195-214, 2018