How big is the Deep Web? It is estimated that the Deep Web comprises 7,500 terabytes – although an exact size is not known, and figures vary widely on this question. The magnitude, complexity and siloed nature of the Deep Web means researchers cannot turn to one specific guide or one search engine to access the vast range of information, data, files and communications that comprise it. The ubiquitous search engines (Google, Bing, Yahoo) manage, index and deliver results from the Surface web. These search results include links, data, information, reports, news, subject matter content and a large volume of advertising that is optimized to increase traffic to specific sites and support marketing and revenue focused objectives. On the other hand, the Deep Web – which is often misconstrued as a repository of dark and disreputable information [Note – it is not the Dark Web], has grown tremendously beyond that characterization to include significant content on a wide range of subject matter covering a broad swath of files and formats, databases, pay-walled content as well as communications and web traffic that is not otherwise accessible through the surface Web.
In the last several years many more applications, tools and techniques have become available that provide researchers with targeted access to specific portions of the Deep Web using discovery techniques that leverage algorithms to access data and information in many formats. Documents used by organizations and entities to communicate within their domains, including databases and files of information that may be password protected and encrypted communications, may also be accessible. Searching for this information requires that users apply search engines, techniques and search tools that function as a means of accessing these repositories of corporate, data, news and research that is not transparent using the Surface web activities. Searching the Deep Web requires the use of multiple techniques and resources – it is challenging, complicated and far less straight forward than just using Google, and much more time consuming as well. This guide providers researchers with an extensive range of sites, services, resources, tools and techniques from which to choose so as to target their efforts to locate and then utilize information and data from within the Deep Web. In addition, this guide assists researchers to identify and implement security and privacy applications while using the Deep Web, as well as offering a large number of training and learning resources to conduct more effective and efficient searches.
This Deep Web Research and Discovery Resources 2019 report and guide is divided into the following topics:
Articles, Papers, Forums, Audios and Videos
Cross Database Search Services
Cross Database Search Tools
Peer to Peer, File Sharing, Grid/Matrix Search Engines
Presentations
Resources – Deep Web Research
Resources – Semantic Web Research
Bot and Intelligent Agent Research Resources and Sites
Articles, Papers, Forums, Audios and Videos
8 Best Deep Web People Search Engines
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/16323-8-best-deep-web-people-search-engines-updated
10 Things You Should Know About Dark Web Websites by Josh Fruhlinger – November 23, 2018
https://www.csoonline.com/article/3322134/cyber-attacks-espionage/10-things-you-should-know-about-dark-web-websites.html
99 Resources to Research & Mine the Invisible Web by Jessica Hupp
https://www.collegedegree.com/library/college-life/99-resources-to/
A Basic Guide to Diving In To the Dark Web
https://www.zdnet.com/pictures/a-basic-guide-to-the-deep-dark-web/
A Beginner’s Guide to the Dark Web by Ben Dickson
https://www.dailydot.com/layer8/what-is-dark-web/
Academic and Scholar Search Engines and Sources 2019
https://www.ScholarSearchEngines.com/
Accessing the Deep Web and Dark Web with Tor by Jack Jones
https://mobile.audible.com/search.htm?narrator=Dan+Gralick
Access the Deep Web and Protect Your Privacy Online with Anonabox by Marco Chiappetta
https://www.forbes.com/sites/marcochiappetta/2016/04/29/access-the-deep-web-and-protect-your-privacy-online-with-the-anonabox/#72cd0dc337c2
All of OCLC’s WorldCat Heading Toward the Open Web by Barbara Quint
https://newsbreaks.infotoday.com/nbreader.asp?ArticleID=16353
An Executive Overview of the Deep and Dark Web
https://www.cio.com/article/3199632/cyber-attacks-espionage/an-executive-overview-of-the-deep-dark-web.html
An Insider’s Look Into the Dark Web by Aaron Tan
https://www.computerweekly.com/feature/An-insiders-look-into-the-dark-web
An Interactive Clustering-based Approach to Integrating Source Query interfaces on the Deep Web by W. Wu, C. Yu, A. Doan, W. Meng
https://www.cs.binghamton.edu/~meng/pub.d/sigmod04-final.pdf
Annotation for the Deep Web
https://dl.acm.org/citation.cfm?id=1137372
An Up-To-Date Layman’s Guide To Accessing The Deep Web
https://www.fastcompany.com/3026989/an-up-to-date-laymans-guide-to-accessing-the-deep-web
A Primer on Staying Secure and Anonymous on the Dark Web by Mark Turner
https://www.techspot.com/guides/1292-web-security-anonymizer-primer/
Articles on Deep and Dark Web Published by Associations of Internet Research Specialists (AIRS)
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/itemlist/category/20-deep-web
Automatic Extraction of Web Search Interfaces for Interface Schema Integration by H. He, W. Meng, C. Yu, Z. Wu
https://www.cs.binghamton.edu/~meng/pub.d/WWWposterhe.pdf
Automatic Information Extraction From Semi-Structured Web Pages By Pattern Discovery
https://dl.acm.org/citation.cfm?id=640423&dl=ACM&coll=portal
Automatic Meaning Discovery Using Google by Rudi Cilibrasi and Paul M. B. Vitanyi
https://arxiv.org/abs/cs.CL/0412098
Battle of the Secure Smartphones
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17276-battle-of-the-secure-smartphones
Bibliomining for Automated Collection Development in a Digital Library Setting: Using Data Mining to Discover Web-Based Scholarly Research Works by Dr. Scott Nicholson
https://scottnicholson.com/pubs/asisdiss.pdf
Big Data and AI: 30 Amazing (And Free) Public Data Sources for 2018 by Bernard Marr
https://www.forbes.com/sites/bernardmarr/2018/02/26/big-data-and-ai-30-amazing-and-free-public-data-sources-for-2018/
Bot Research 2019
https://www.BotResearch.info/
BrightPlanet Launches Deep Web Data Feeds: Global News Data Feed Is First Available Data Feed
https://www.brightplanet.com/2015/01/brightplanet-launches-deep-web-data-feeds-global-news-data-feed-first-available-data-feed/
Client-Side Deep Web Data Extraction
https://www.computer.org/csdl/proceedings/cec-east/2004/2206/00/22060158-abs.html
Clustering E-Commerce Search Engines by Q. Peng, W. Meng, H. He, C. Yu
https://www.cs.binghamton.edu/~meng/pub.d/WWWposterPeng.pdf
Common Deep Web and Big Data Questions Answered (Part 1)
https://www.brightplanet.com/2014/11/common-deep-web-big-data-questions-answered-part-1/
Common Deep Web and Big Data Questions Answered (Part 2)
https://www.brightplanet.com/2014/12/common-deep-web-big-data-questions-answered-part-2/
Creating Intelligence from Big Data
https://bigdata.brightplanet.com/creating-new-intelligence-from-big-data
Current Awareness Discovery Tools on the Internet 2019
https://www.zillman.us/white-papers/current-awareness-discovery-tools-on-the-internet/
Current Awareness Tools 2019
https://www.currentawarenesstools.com/
DarkCyber Beyond Search by Stephen E. Arnold
https://arnoldit.com/wordpress/category/dark-web/
Darknet Resembles the Open Internet – Just With Anonymity by Roderick S. Graham and Brian Pitman
https://www.newsweek.com/darknet-resembles-internet-itself-anonymity-550324?rx=us
DarkReading
https://www.darkreading.com/
Dark Web (DarkNet) by Margaret Rouse
https://whatis.techtarget.com/definition/dark-web
Dark Web Data Theft May Have Finally Met Its Match by Vaughn Highfield
https://www.alphr.com/security/1009340/dark-web-data-theft-fujitsu-intsights
Darkweb for Newbees: Hacking and Marketplaces by Gaurav Sangwani
https://www.businessinsider.in/darkweb-for-newbees-hacking-and-marketplaces/articleshow/55941181.cms
Dark Web News
https://www.DarkWebNews.com/
Dark Web Security Tips by Enjeck Mbeh Cleopatra
https://latesthackingnews.com/2018/07/17/dark-web-security-tips/
Dark Web Version of Facebook Shows a New Way to Secure the Web by Tom Simonite
https://www.technologyreview.com/s/532256/dark-web-version-of-facebook-shows-a-new-way-to-secure-the-web/
Dark Web – CRS Report by Kristin Finklea
https://fas.org/sgp/crs/misc/R44101.pdf
Deep Web by Margaret Rouse
https://whatis.techtarget.com/definition/deep-Web
Deep Web – Exploring the Secrets of the Hidden Internet by Marcus P. Zillman, M.S., A.M.H.A., – 23 minutes – Internet/Technology Channel
https://www.planetearthradio.com/technology.htm
Deep Web: Legal Due Diligence by Lisa Brownlee
https://federatedsearchblog.com/2015/04/29/deep-web-legal-due-diligence/
Deep Web Links – Massive Deep Websites Links and URL by Martin Grossner
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/19104-deep-web-links-massive-deep-websites-links-and-url
Deep Web May Be 500 Times Bigger Than Normal Web
https://mobile.wnd.com/2018/09/deep-web-may-be-500-times-bigger-than-normal-web/
Deep Web Research and Discovery Resources 2019
https://www.DeepWeb.us/
Deep Web Search Engines to Explore the Hidden Internet by Rakesh Krishnan
https://cdn.ampproject.org/c/s/amp.thehackernews.com/thn/2016/02/deep-web-search-engine.html
Deep Web Technologies Launches Explorit! EBM, the leading Evidence-Based Medicine Search Engine for Medical Libraries
https://www.benzinga.com/pressreleases/18/06/r11847737/deep-web-technologies-launches-explorit-ebm-the-leading-evidence-based
Deep Web: The Mysterious Side of the Internet and How To Access It by Michael Lowry
https://factschronicle.com/deep-web-the-mysterious-side-of-the-internet-and-how-to-access-it-14770.html
Deep Web: What Is It and How To Access it? – The Ultimate Guide by Samantha Carter
https://darkwebnews.com/deep-web/
Digging Deeper into Deep Web Databases by Breaking Through the Top-k Barrier
https://arxiv.org/abs/1208.3876
Digital Shadows Launches Shadow Search
https://www.darkreading.com/threat-intelligence/digital-shadows-launches-shadow-search/d/d-id/1332921
Digital Shadows Launches Shadow Search – Enabling Organizations to Self-Serve and Find Cyber Threats Against Them from Across The Open, Deep and Dark Web
https://www.businesswire.com/news/home/20180926005135/en/Digital-Shadows-Launches-Shadow-Search%E2%84%A2-%E2%80%93-Enabling
Diving Into the Darknet by Michal Salat
https://blog.avast.com/diving-into-the-darknet
Do’s and Don’ts of the Dark Web by Martin Beltov
https://securityboulevard.com/2018/07/dos-and-donts-of-the-dark-web/
Effective and Scalable Metasearch Project
https://www.cs.binghamton.edu/~meng/metasearch.html
Efficient Deep Web Crawling Using Reinforcement Learning
https://link.springer.com/chapter/10.1007%2F978-3-642-13657-3_46
Everything You Need To Know About the Deep Web In One Simple Infographic
https://www.businessinsider.com/everything-you-need-to-know-about-the-deep-web-in-one-simple-infographic-2015-2
Experiences In Crawling Deep Web In The Context Of Local Search
https://dl.acm.org/citation.cfm?id=1460016
First Comprehensive Map of the Dark Web Reveals a Remarkable Antisocial Corner of the Internet by Matthew Hutson
https://www.sciencemag.org/news/2017/04/first-comprehensive-map-dark-web-reveals-remarkably-antisocial-corner-internet
Grey Literature
https://en.wikipedia.org/wiki/Gray_literature
Grey Literature Network Service (GreyNet)
https://www.greynet.org/
Hackers Aggregating Data Leaks on the Dark Web
https://it.toolbox.com/article/hackers-aggregating-data-leaks-on-the-dark-web
Here Are the 10 Best Deep Web Search Engines by Kristen Hubby
https://www.dailydot.com/layer8/best-deep-web-search-engines/
How Does the Tor Browser Functions? By Ben Prince
https://www.technologynewsextra.com/how-does-the-tor-browser-functions/96041.html
How Scary Is the Dark Web by Aria Inthavong
https://www.buzzfeed.com/watch/video/68371
How To Access the Dark Web: What Is Tor and How Do I Access Dark Websites? By Victoria Woollaston
https://www.alphr.com/technology/1002667/how-to-access-the-dark-web-what-is-tor-and-how-do-i-use-it
How To Access the Dark Web by Brandt Ranj
https://www.independent.co.uk/life-style/gadgets-and-tech/how-to-access-the-dark-web-a7047041.html
How To Access the Dark Web by Tyler Lacoma
https://www.digitaltrends.com/computing/how-to-access-the-dark-web/
How to Access the Dark Web (Deep Web)
https://www.thedarkweblinks.com/how-to-access-the-dark-web/
How To Access the Dark Web Using Tor by Cozmo
https://mixance.com/how-to-access-the-dark-web-using-tor/
How To Access the Deep Web Anonymously by Ritika Tiwari
https://www.cloudwards.net/the-deep-web/
How To Access the Hidden Wiki?
https://talkdailynews.com/how-to-access-the-hidden-wiki/
How To Browse the Deep Web Using Tor on iPhone and iPad by Sumeet Sharma
https://www.iappnalysis.com/2015/04/browse-deep-web-on-iphone-ipad-using-tor-browser.html
How To Do a Deep Web Search and Why You Should by Kelli Uhrich of Komando.com
https://www.komando.com/tips/396770/how-to-do-a-deep-web-search-and-why-you-should
How to Mine the Invisible Wev: The Ultimate Guide by Jerri Collins
https://www.lifewire.com/mine-invisible-web-3482495
How To Safely Access The Dark Web by David Nield
https://www.lifehacker.com.au/2018/03/how-to-search-the-deep-web-safely/
How To Tread Lightly Into The Dark Web by Logan Hochstetler
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/18649-how-to-tread-lightly-into-the-dark-web
How To Use a VPN with Tor
https://www.expressvpn.com/how-to-use-vpn/tor-vpn
If Your Data Is Found On the Dark Web, Firefox Monitor Will Let You Know by Chuong Nyguyen
https://www.digitaltrends.com/computing/firefox-monitor-lets-your-know-if-youve-been-pwned/
Illuminating the Dark Web by Robert Gehl
https://www.scientificamerican.com/article/illuminating-the-dark-web
Inequality and the Intellectual Dark Web by Christian Alejandro Gonzalez
https://www.nationalreview.com/2018/05/intellectual-dark-web-members-united-views-inequality/
Information Retrieval and the Semantic Web by Tim Finin, James Mayfield, Clay Fink, Anupam Joshi, and R. Scott Cost
https://ebiquity.umbc.edu/paper/html/id/185/
In Search of the Deep Web
https://www.salon.com/2004/03/09/deep_web/
Inside the Dark Web: The Truth Is There Is a Lot of Evil Out There by Conor Pope
https://www.irishtimes.com/culture/tv-radio-web/inside-the-dark-web-the-truth-is-there-is-a-lot-of-evil-out-there-1.3653092
Intellectual Dark Web: New Movement or Just a Rebarnding of Old Ideas? By Kevin Dickinson
https://bigthink.com/kevin-dickinson/what-is-the-so-called-intellectual-dark-web.amp
IR and IE on the Web – PhD and MSc Dissertations
https://groups.yahoo.com/neo/groups/webir/info
https://www.webir.org/
Is the Tor Browser Safe? Why Should You Use It? By Mohammed Mujtab
https://technocodex.com/is-tor-browser-safe-why-should-you-use-it/
Journey Into the Dark by Jason Smith
https://medium.com/@jasisrad/journey-into-the-dark-8c7922a48265
Journey Into the Hidden Web: A Guide for New Researchers by Ryan Dube
https://www.makeuseof.com/tag/journey-into-the-hidden-web-a-guide-for-new-researchers/
Just the Tip of the Iceberg: Why You Should Be Monitoring the Deep Web
https://www.information-age.com/technology/security/123461668/just-tip-iceberg-why-you-should-be-monitoring-deep-web
Lessons from the Deep Web That Could Lead To a More Secure IoT by Revathl Subramanian
https://blogs.ca.com/2015/04/02/lessons-from-the-deep-web-that-could-lead-to-a-more-secure-iot/?mrm=425878&cid=GLOB-SMM-ABUS-AAR-000002-00000571
Mining Newsgroups Using Networks Arising From Social Behavior
https://dl.acm.org/citation.cfm?id=775227
Mining the Deep Web: Search Strategies That Work by Lee Ratzan
https://www.computerworld.com/s/article/9005757/Mining_the_Deep_Web_Search_strategies_that_work?pageNumber=1
Mining Topic-Specific Concepts and Definitions on the Web
https://www.cs.uic.edu/~liub/publications/WWW-2003.pdf
NASA Is Indexing the Deep Web to Show Mankind What Google Won’t by Danielle Bronner
https://fusion.net/story/145885/nasa-is-indexing-the-deep-web-to-show-mankind-what-google-wont/
New AI System DARKMENTION Will Detect Upcoming Cyberattackes From Dark Web by Waqas
https://www.hackread.com/ai-system-darkmention-detects-cyberattacks-from-dark-web
Next Generation TOR is Arriving and It Will Include a Distributed Random Number Generator
https://securityaffairs.co/wordpress/47683/deep-web/distributed-tor-random-number-generator.html
NYU-Poly Researcher Awarded DARPA Contract To Explore the Deep Web by Rhea Kelly
https://campustechnology.com/articles/2015/02/25/nyu-poly-researchers-awarded-darpa-contract-to-explore-the-deep-web.aspx
Onion Browser – An Open-Source Privacy Enhancing Web Browser for iOS
https://mike.tig.as/onionbrowser/
OnionLand Demystifying the Mystery of the Deep Web
https://m.digitaljournal.com/pr/3927243
OntoMiner: Bootstrapping and Populating Ontologies From Domain Specific Web Sites
https://www.public.asu.edu/~hdavulcu/VLDB-WS03.pdf
OpenIndex – Creating a Public Internet Index
https://www.openindex.org
Out-googling Google: Federated Searching and the Single Search Box
https://library.marist.edu/ACRL/Foxhunt_demo.html
Privacy, the Dark Web and Hacker Devices by Don Hawkins
https://www.libconf.com/2018/04/18/privacy-the-dark-web-and-hacker-devices/
Publications about Web Analysis, Web Search, Citation Indexing, Digital Libraries, Machine Learning, Neural Networks
https://research.google.com/
QProber: Classifying and Searching “Hidden-Web” Text Databases
https://www.ipeirotis.com/?project=qprober-classifying-and-searching-hidden-web-text-databases
Really Private Browsing: An Unofficial User’sGuide to Tor by Andre Infante
https://www.makeuseof.com/tag/really-private-browsing-an-unofficial-users-guide-to-tor/
Research Beyond Google: 56 Authoritative, Invisible, and Comprehensive Resources
https://oedb.org/ilibrarian/research-beyond-google/
Researchers Call for a Shared Dark Web Taxonomy by Phil Muncaster
https://www.infosecurity-magazine.com/news/researchers-call-shared-dark-web
Scan the Dark Web For Threat Intelligence by Michelle Drolet
https://www.csoonline.com/article/3245587/cyber-attacks-espionage/scan-the-dark-web-for-threat-intelligence.html
Scientific American: Featured Article: The Semantic Web In Action
https://www.scientificamerican.com/article/semantic-web-in-actio/
Search Engine Meeting Archives
https://www.SearchEngineMeeting.net/
Search Engine Technology and Digital Libraries
https://www.dlib.org/dlib/june04/lossau/06lossau.html
Searching the Dark Web for Business Intelligence by Robert Vamosi
https://www.forbes.com/sites/robertvamosi/2018/04/28/searching-the-dark-web-for-business-intelligence/
Searching the Deep Web by Alex Wright
https://mags.acm.org/communications/200810/?pg=16
Searching the Deep Web
https://www.dlib.org/dlib/january01/warnick/01warnick.html
Searching the Internet 2018 – The Primer – 2019 Guide to Searching the Internet
https://www.SearchingTheInternet.info/
Search Interfaces on the Web: Querying and Characterizing by Denis Shestakov
https://www.scribd.com/document/231936534/Search-Interfaces-on-the-Web-Querying-and-Characterizing
Seeing through the ‘invisible’ Web
https://usatoday30.usatoday.com/tech/2001/10/15/invisible-web-search.htm
Semantic Web Content Accessibility Guidelines for Current Research Information Systems (CRIS) by A. Lopatenko
https://derpi.tuwien.ac.at/~andrei/AURIS_DE.htm
Shining a Light on the So-Called Dark Web by Kate Knibbs
https://www.theringer.com/tech/2018/5/10/17339900/dark-web-lexicon
Startup CYRCON’s Artificial Intelligence Predicts Enterprise Cyberattacks by PRWEB, November 5, 2018
https://www.prweb.com/releases/startup_cyr3cons_artificial_intelligence_predicts_enterprise_cyberattacks/prweb15887247.htm
Staying Safe in the Shadow of the Dark Web
https://www.benzinga.com/pressreleases/18/05/p11804571/staying-safe-in-the-shadow-of-the-dark-web
Stop Talking About the Dark Web: Tor Project CoFounder Roger Dingledine
https://technical.ly/philly/2017/05/15/dark-web-roger-dingledine/
Stop Worrying About the Dark Web by Rich Matta
https://www.forbes.com/sites/forbestechcouncil/2018/10/15/stop-worrying-about-the-dark-web/#31e060935fcc
Structured Databases on the Web: Observations and Implications
https://dl.acm.org/citation.cfm?id=1031584
Testbed for Information Extraction from Deep Web
https://research.microsoft.com/users/nickcr/pubs/yamada_www2004poster.pdf
The 10 Best Deep Web Search Engines to Explore the Hidden Web by Michelle Fuchs
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17217-top-10-best-deep-web-search-engines-to-explore-hidden-web
The 10 Best Deep Web Search Engines to Explore Hidden Web by Issac Avila
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/18749-top-10-best-deep-web-search-engines-to-explore-hidden-web
The Black Market Report
https://www.armor.com/vantage-points/black-market-report/
The Darknet: A Quick Introduction for Business Leaders by Trey Degrassi
https://hbr.org/2016/12/the-darknet-a-quick-introduction-for-business-leaders
The Dark Web and How To Access It by MacKenzie Sigalos
https://www.cnbc.com/2018/04/13/the-dark-web-and-how-to-access-it.html
The Dark Web and Remote Communication – November 10, 2018 – Tech Talk Radio
http://federalnewsnetwork.com/stratford-universitys-tech-talk/2018/11/the-dark-web-and-remote-communication-november-10-2018/
The Dark Web: What Every Parent Should Know by Cybermum Australia
https://securingtomorrow.mcafee.com/consumer/family-safety/dark-web-what-every-parent-should-know/
The Dark Web What It Is and Staying Safe by Annie Quireshi
https://sustainablebusinessforum.com/dark-web-what-it-is-and-staying-safe
The Dark Web? Why Small Businesses Should Concern Themselves With the Threat by Jim Anderson
https://www.entrepreneur.com/article/320347
The Deep Web May Be 500 Times Bigger Than the Normal Web
https://www.cnbc.com/2018/09/06/beyond-the-valley-understanding-the-mysteries-of-the-dark-web.html
The Deep Web: Surfacing Hidden Value by Michael K. Bergman
https://quod.lib.umich.edu/j/jep/3336451.0007.104?view=text;rgn=main
The Deep Web: What’s Lurking in the Underbelly of the Internet? By Michelle Alvarez
https://securityintelligence.com/the-deep-web-whats-lurking-in-the-underbelly-of-the-internet/#.VUIUNmfD9D8
The Future Of News: The Digital Information Librarian
http://www.masternewmedia.org/2004/03/24/the_future_of_news_the.htm
The Growing Force That Will Soon Reshape The Entire Internet by Frank Minite
https://www.forbes.com/sites/frankminiter/2017/12/30/the-growing-force-that-will-soon-reshape-the-entire-internet/
The Hidden Potential of the Web
https://www.theguardian.com/society/2004/apr/21/epublic.technology18
The Intellectual Dark Web Conservatives Fear by Taylor Link
https://www.salon.com/2018/09/02/the-intellectual-dark-web-conservatives-fear
The Invisible Web by Chris Sherman
https://web.freepint.com/go/newsletter/64#feature
The Invisible Web: What it is, Why it exists, How to find it, and Its Inherent Ambiguity
https://www.newworldencyclopedia.org/entry/Deep_Web
The Invisible Web: Where Search Engines Fear To Go
https://www.powerhomebiz.com/vol25/invisible.htm
The Most Amazing Deep Web Search Engines Finally Introduced To the World Called: OnlindLandSearchEngines.com .
https://m.digitaljournal.com/pr/3928224
The New Search Engines Shining a Light On the Deep Web by Carola Frediani
https://kernelmag.dailydot.com/issue-sections/features-issue-sections/10376/how-to-search-deep-web-tor/
The Other Side of Internet by Robert Johnny, Jr.
https://it.toolbox.com/blogs/robertjohnnyjr/the-other-side-of-internet-090618
The Problems With Searching the Deep Web by Nancy K. Herther
https://newsbreaks.infotoday.com/NewsBreaks/The-Problems-With-Searching-the-Deep-Web-121884.asp
The Super-Private Tor Browser Get a Huge Update, But Should You Switch From Chrome?
https://gizmodo.com/the-super-private-tor-browser-gets-a-huge-update-but-s-1828879886
The Ultimate Guide to the Invisible Web
https://oedb.org/ilibrarian/invisible-web/
The Virtual Private Library™ and The Deep Web Video by Melissa Barker
https://zillman.blogspot.com/2009/07/virtual-private-library-and-deep-web.html
Things You Can Do On the Dark Web That Aren’t Illegal by Jacob Kleinman
https://lifehacker.com/things-you-can-do-on-the-dark-web-that-arent-illegal-1819790298
This Tool Lets You Scan the Dark Web for Your (stolen) Personal Data by Anna K. Sasaki
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17251-this-tool-lets-you-scan-the-dark-web-for-your-stolen-personal-data
Timeline of Events Related to the Deep Web
https://papergirls.wordpress.com/2008/10/07/timeline-deep-web/
Top 5 Legal Reasons To Use the Darknet by JP Buntinx
https://themerkle.com/top-5-legal-reasons-to-use-the-darknet/
Topological Measures and Maps Of the Web
https://informatics.indiana.edu/fil/Web/
TOR Browser Arrives On Android To Protect Your Secret Searches
https://www.theinquirer.net/inquirer/news/3062424/tor-browser-arrives-on-android-to-protect-your-secret-searches
Tor Coders Harden the Onion Against Surveillance
https://nakedsecurity.sophos.com/2016/06/23/tor-coders-harden-the-onion-against-surveillance/
TOR For Newbies – When Should You Use It?
https://www.makeuseof.com/tag/tor-for-newbies/
TorVPN by Max Eddy
https://uk.pcmag.com/torvpn/19165/review/lockspin-torvpn
Toward the Semantic Deep Web by James Geller, Soon Ae Chun, and Yoo Jung An
https://www.mendeley.com/catalog/toward-semantic-deep-web/
Towards Automatic Incorporation of Search Engines Into A Large-Scale Metasearch Engine
https://www.cs.binghamton.edu/~meng/pub.d/wi2003.pdf
Traffic-Based Feedback on the Web by Jonathan Aizen, Daniel Huttenlocher, Jon Kleinberg, and Antal Novak
https://www.pnas.org/content/101/suppl_1/5254.abstract
Travel Industry and Deep Web: Exclusive Interview with Marcus P. Zillman
https://plrplr.com/90014/deep-web-and-travel-industry-exclusive-interview-with-marcus-p-zillman/
UMBC – AgentNews
https://agents.umbc.edu/
Understanding Metadata: What is Metadata, and What What Is It For? A Primer
https://www.niso.org/publications/understanding-metadata-2017
Understanding the Deep Web In 10 Minutes
https://www.brightplanet.com/2013/03/whitepaper-understanding-the-deep-web-in-10-minutes/
Using the Internet As a Dynamic Resource Tool for Knowledge Discovery 2018
https://www.zillman.us/white-papers/using-the-internet-as-a-dynamic-resource-tool-for-knowledge-discovery/
Web Characterization Activity
https://www.w3.org/WCA/
Web Data Extractors 2019
https://www.WebDataExtractors.com/
Web Pages Search Engine Based on DNS by Wang Liang, Guo Yi-Ping, and Fang Ming
https://arxiv.org/pdf/cs.NI/0403035
WebScales: Towards a Highly Scalable Metasearch Engine
https://www.cs.binghamton.edu/~meng/pub.d/PIreport04.html
What Exactly Is the Mysterious Dark Web and How It Is Different From the Deep Web by Kshitij Pujari
https://tech.firstpost.com/news-analysis/what-exactly-is-the-mysterious-dark-web-and-how-it-is-different-from-deep-web-379668.html
What Happens When You Spend a Week Fishing on the Dark Web? By Anthony Cuthbertson
https://www.independent.co.uk/news/long_reads/dark-web-search-drugs-weapons-crimes-illegal-a8524341.html
What Is The Dark Web? By David Glance
https://www.iflscience.com/technology/what-dark-web/
What Is the Dark Web and How Does It Work?
https://theconversation.com/what-is-the-dark-web-and-how-does-it-work-63613
What Is the Dark Web and How To Access This Scary Part of Internet by Karandeep Singh
https://www.mobipicker.com/what-is-the-dark-web-how-to-access-the-dark-web/
What is the Dark Web – Our Guide to the Dark Web by Laurie Clarke
https://www.techworld.com/security/what-is-dark-web-3645157/
What Is the Deep Web and How Can You Access It? By Adam Rowe
https://tech.co/what-is-the-deep-web-2018-05
What Is the Deep Web? A WhatIs Podcast 15 Minute Interview with Marcus P. Zillman
http://zillman.blogspot.com/2006/10/what-is-deep-web.html
What Happens When You Spend a Week Fishing On the Dark Web?
https://www.independent.co.uk/news/long_reads/dark-web-search-drugs-weapons-crimes-illegal-a8524341.html
What Is Deep Web? All You Need To Know About It
https://newmexicocourierexpress.com/what-is-deep-web-all-you-need-to-know-about-it/50160
What is the Invisible Web? A Crawler Perspective by Natalia Arroyo, Laboratorio de Internet
http://cybermetrics.wlv.ac.uk/AoIRASIST/arroyo.html
What is the Difference Between the Dark Web and the Deep Web by Laurie Clarke
https://www.techworld.com/data/what-is-difference-between-dark-web-deep-web-3680293/
What It Looks Like On the Dark Side of the Web by Felix Simon
https://medium.com/@FelixSimon/what-it-looks-like-on-the-dark-side-of-the-web-f53ff9f4ef3c
What’s Really On the Dark Web? By Jeff Deal
https://www.wftv.com/news/local/what-s-really-on-the-dark-web-/860574573
Where Your Stolen Personal Details Go Online by Nicholas and Matthew Dunn
https://www.news.com.au/technology/online/security/inside-the-dark-web-where-your-stolen-personal-details-go-online/news-story/1f13c214950187d6412ce4b89bfe42fe
Why the Dark Web Could Be a Growing Threat to Your Small Business by Ben Lobel
https://smallbusiness.co.uk/why-the-dark-web-could-be-a-growing-threat-to-your-small-business-2545071/
Wikipedia – Deep Web
https://en.wikipedia.org/wiki/Deep_web
WISE-Cluster: Clustering E-Commerce Search Engines Automatically by Q. Peng, W. Meng, H. He, C. Yu
https://www.cs.binghamton.edu/~meng/pub.d/PengWIDM04.pdf
Search Tools Reports: Searching for Text Information in Databases
https://www.searchtools.com/info/database-search.html
UK Web Archiving Consortium
https://www.webarchive.org.uk
CROSS DATABASE SEARCH SERVICES
FDsys – Search Across Multiple Government Databases
https://www.GovInfo.gov/
GovInfo – Discover Government Information
https://www.GovInfo.gov/
King County Library System
https://www.kcls.org/
NLM Gateway Search
https://wwwcf.nlm.nih.gov/hsr_project/home_proj.cfm
SUMSearch 2 [Health Sciences]
https://sumsearch.org/
Bright Planet – Deep Web Intelligence
https://brightplanet.com/
Copernic
https://www.copernic.com/
Dieselpoint Java Search and Navigation Software
https://www.dieselpoint.com/
Dublin Core Metadata Initiative (DCMI)
https://www.dublincore.org/
EEVL Xtra – Cross Database Search
https://www.ariadne.ac.uk/issue44/eevl/
Gold Rush – Database Search Tool
https://goldrush.coalliance.org/
MetaLib
https://www.exlibrisgroup.com/category/MetaLibOverview
MetaSearch Initiative
https://www.niso.org/workrooms/mi
MuseGlobal
https://www.museglobal.com/
Peter’s PolySearch Engines
https://www2.hawaii.edu/~jacso/extra/poly-page.html
PBCore – The Public Broadcasting Metadata Dictionary
https://www.pbcore.org/
Search Federal Research and Development
https://www.osti.gov/
SRU – Search/Retrieve via URL
https://www.loc.gov/standards/sru
The Flamenco Search Interface Project
https://flamenco.berkeley.edu/
PEER TO PEER (P2P), FILE SHARING, GRID AND MATRIX SEARCH ENGINES
ALPINE Network – SourceForge: Project
https://sourceforge.net/projects/alpine/
Azureus – Vuze Java Bittorrent Client
https://www.vuze.com/
BadBlue [Uncensored News]
https://badblue.com/
Between Rhizomes and Trees: P2P Information Systems by Bryn Loban
https://firstmonday.org/ojs/index.php/fm/article/view/1182
BigChampagne
https://www.bigchampagne.com/
Bitmessage – P2P Communication Protocol To Send Encrypted Messages
https://bitmessage.org/wiki/Main_Page
Bit Torrent Official Site and Search Engine
https://www.BitTorrent.com/
Coral – The Coral P2P Content Distribution Network
https://www.coralcdn.org/
Capn’s PHP Gnutella Search [Only code is available for download]
https://capnbry.net/gnutella/gs.php
Distributed Search Engines
https://en.wikipedia.org/wiki/Distributed_search_engine
https://wiki.p2pfoundation.net/Distributed_Search_Engines
Distributed Search in P2P Networks
https://www.computer.org/csdl/mags/ic/2002/01/w1068-abs.html
FAROO – P2P Web Search
https://www.faroo.com/
FilesOverMiles – Browser to Browser File Sharing (P2P)
https://www.filesovermiles.com/
Filetopia – File sharing tool with public key encryption
https://www.filetopia.org/
Free Haven Project
https://www.freehaven.net
Frost Project – Freenet Messaging and File Sharing Client
https://jtcfrost.sourceforge.net/
FuzzBox: Tangent Research Artificial Intelligence and Robotics
https://tangentresearch.com/news/07252001_p2p_ai.html
GNUnet – Secure P2P Networking – Free Software Foundation (FSF)
https://gnunet.org/
Grid, Distributed and Cloud Computing Resources 2018
https://www.GridResources.info/
GNU GRUB – Multiboot Boot Loader
https://www.gnu.org/software/grub/
Ian Clarke’s Blog
https://blog.locut.us/
Internet Movie Database (IMDb)
https://www.imdb.com/
IPFS Distributed Web
https://ipfs.io/
MoleSter – A Tiny File-Sharing Application
http://ansuz.sooke.bc.ca/software/molester/
MusicBrainZ – Open Music Encyclopedia
https://www.MusicBrainZ.org/
MysterNetworks – The Evolution of Peer-to-Peer
https://www.mysternetworks.com/
OpenNap: Open Source Napster Server
https://opennap.sourceforge.net/
P2P and the Future of Private Copying by Peter K. Yu, Michigan State University College of Law
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=578568
Peer-To-Peer Wikipedia
https://en.wikipedia.org/wiki/Peer-to-peer
Peer to Peer File Sharing – P2P Networking
https://compnetworking.about.com/od/p2ppeertopeer/Peer_to_Peer_File_Sharing_P2P_Networking.htm
Port Knocking
https://www.portknocking.org/
PowerFolder – P2P Whole Folder Synchronization
https://www.powerfolder.com/
Rodi – Tiny P2P Client/Host
https://rodi.sourceforge.net/
Skype
https://www.skype.com/
Slyck – File Sharing News and Info
https://www.slyck.com/
Solid – Reshape the Web
https://solid.inrupt.com/
Stealth Mode Online Privacy Resources 2019
https://www.StealthMode.info/
Swarm – A Transparently Scalable Distributed Programming Language
https://swarmframework.org/
The Anthill Project
https://www.cs.unibo.it/projects/anthill/
The Freenet Project
https://freenetproject.org/
ToPeer
https://www.2peer.com/
Transmission – Fast, Easy and Free BitTorrent Client
https://www.transmissionbt.com
Tribler – Privacy Using Our Tor-Inspired Onion Routing
https://www.tribler.org/
TrustyFiles
https://www.trustyfiles.com/
Understanding BitTorrent: An Experimental Perspective by Arnaud Legout, Guillaume Urvoy-Keller, and Pietro Michiardi
https://hal.inria.fr/inria-00000156/en
WASTE (Secure P2P communication)
https://slackerbitch.free.fr/waste/
YaCy – Distributed P2P Based Web Indexing and Anonymous Search Engine
https://www.yacy.net/
YouServ – A P2P (peer-to-peer) Web Hosting/File Sharing System
https://www.bayardo.org/youserv/
Zebra – Structured Text Indexing and Retrieval
https://www.indexdata.com/zebra
Zilok – Peer To Peer Rental Marketplace
https://zilok.com/
Deep Web
https://whatis.techtarget.com/definition/deep-Web
Searching the Internet 2018 – The Primer – 2019 Guide to Searching the Internet
https://www.SearchingTheInternet.info/
The Virtual Private Library™ and The Deep Web Video by Melissa Barker
https://zillman.blogspot.com/2009/07/virtual-private-library-and-deep-web.html
6 Dark Web Pricing Trends – It Isn’t Good News
https://craigpeterson.com/radio-show/6-dark-web-pricing-trends-it-isnt-good-news/19603
Academic and Scholar Search Engines and Sources 2019
https://www.ScholarSearchEngines.com/
A Data-Driven Look At Dark Web Marketplaces
https://www.visualcapitalist.com/data-driven-look-dark-web-marketplaces/
AEON (Automatic Evaluation of ONtologies)
https://code.google.com/p/aeon-project/
AnkaSearch – Meta Search and Deep Web Search Desktop Tool
https://www.ankasoftware.com/ankasearch.html
Anonymous Web Browsing – Wikipedia
https://en.wikipedia.org/wiki/Anonymous_web_browsing
An Up-To-Date Layman’s Guide To Accessing The Deep Web
https://www.fastcompany.com/3026989/an-up-to-date-laymans-guide-to-accessing-the-deep-web
AskReddit – What Are Your Experiences With the Deep Web
https://www.reddit.com/r/AskReddit/comments/lm4dl/reddit_what_are_your_experiences_in_the_deep_web/
Awesome Public Datasets
https://github.com/caesar0301/awesome-public-datasets
BASE – Bielefeld Academic Search Engine
https://www.base-search.net
Biznar – Deep Federated Search
https://www.deepwebtech.com/2015/08/biznar-a-deep-web-google-alternative/
Bot and Intelligent Agent Research Resources 2019
https://www.BotResearch.info/
BrightPlanet – Deep Web Intelligence
https://www.brightplanet.com/
Catalog of U.S. Government Publications (CGP)
https://catalog.gpo.gov/
Cazoodle – Search, Integrate, and Organize — The Real World
https://www.cazoodle.com/
Censys Search Engine for Devices and Networks
https://censys.io/
CloserLook Search Engine
https://www.closerlooksearch.com/
Creative Commons RDF-Enhanced Search
https://search.creativecommons.org/
Cyber Cemetery
https://govinfo.library.unt.edu/
CyberGhost – One of the World’s Most Trusted and Secure Virtual Private Networks
https://www.cyberghostvpn.com/
Dark Reading
https://www.darkreading.com/
Dark Web Academy – Dark Web Curriculum
https://www.youtube.com/channel/UCJ31aJo8U-ZaRnZ4Y27so_Q
Dark Web Intelligence and Monitoring Service – Perfecta
https://www.prnewswire.com/news-releases/perfecta-expands-into-the-commercial-sector-and-announces-perfecta-lantern-dark-web-intelligence-and-monitoring-service-300648152.html
DarkWeb News – Deep Web
https://darkwebnews.com/category/deep-web/
DarkWeb News – The Ultimate Dark Web Resource
https://darkwebnews.com/
Data Mining Resources 2019
http://www.DataMiningResources.info/
Datasets for Data Mining and Data Science
https://www.kdnuggets.com/datasets/index.html
DeepDive – Analyze Data On a Deeper Level Than Ever Before
https://deepdive.stanford.edu/
Deep.Dot.Web
https://www.DeepDotWeb.com/
Deep Web – Discover Resources That Help You Mine the Deep or Invisible Web Instead of Just Searching the Surface
https://libguides.msubillings.edu/c.php?g=242182&p=1610131
Deep Web Links – Massive Deep Websites Links and URLs
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/19104-deep-web-links-massive-deep-websites-links-and-url
Deep Web Research and Discovery Resources 2019
https://www.DeepWebResearch.info/
https://www.DeepWeb.us/
Deep Web Search
https://deep-web.org/
Deep Web Sites
https://www.deepweb-sites.com/
Deep Web Technologies – federated search
https://www.deepwebtech.com/
Deep Web University
https://brightplanet.com/deep-web-university/
Digital Shadows
https://www.digitalshadows.com/
Directory Resources 2018
https://www.DirectoryResources.info/
eGreenBot 2018 – Green Resources Search Engine
https://www.eGreenBot.com/
eHealthcare Bot 2019 Deep Meta Search Engine
https://www.eHealthcareBot.com/
eMarketing Bot 2019 Deep Meta Search Engine
https://www.eMarketingBot.com/
ENDECA
https://www.oracle.com/us/products/applications/commerce/endeca/overview/index.html
Engineering Village
https://www.engineeringvillage.com
EU Open Data Portal
https://data.europa.eu/euodp/en/home
Federated Search Blog
https://federatedsearchblog.com/
Firefox Monitor
https://monitor.firefox.com/
Freely Accessible Databases for the Public
https://www.istl.org/01-winter/internet.html
Google Fusion Tables
https://www.google.com/drive/apps.html#fusiontables
Google Scholar
https://scholar.google.com/
Healthdata.gov
https://www.healthdata.gov/
How To Do a Deep Web Search and Why You Should
https://www.komando.com/tips/396770/how-to-do-a-deep-web-search-and-why-you-should
Indiegogo Datasets
https://webrobots.io/indiegogo-dataset/
Internet Archive
https://www.archive.org/
Invisible Library
https://invislib.blogspot.com/
IOGDS: International Open Government Dataset Search
https://logd.tw.rpi.edu/node/9903
Kapow Web Collector
https://www.automated-info-solutions.com/
Karma – Data Integration Tool
https://www.isi.edu/integration/karma/
KDnuggets: Data Mining, Web Mining, and Knowledge Discovery Guide
https://www.kdnuggets.com/
Kickstarter Datasets
https://webrobots.io/kickstarter-datasets/
Knowledge Discovery 2019
https://www.KnowledgeDiscovery.info/
Knowlesys – Use Deep Data Mining To Find the Intelligence
https://knowlesys.com/
Large-Scale Deep Web Integration: Incomplete Bibliography
https://metaquerier.cs.uiuc.edu/webibib.html
Linked Data – Connect Distributed Data Across the Web
https://linkeddata.org/
LinkingOpenData – W3C SWEO Community Project
https://www.w3.org/wiki/SweoIG/TaskForces/CommunityProjects/LinkingOpenData
List of Academic Databases and Search Engines – Wikipedia
https://en.wikipedia.org/wiki/List_of_academic_databases_and_search_engines
List of Tor Hidden Services from the Wikipedia
https://en.wikipedia.org/wiki/List_of_Tor_hidden_services
MagPortal
https://www.magportal.com/
Mappa.Mundi Magazine
https://mappa.mundi.net/
Mason OER Metafinder
https://mason.deepwebaccess.com/mason__MasonLibrariesOpenEducationResources_5f4/desktop/en/search.html
Mednar – Innovative Medical Search
https://mednar.com/
Mining the Deep Web for Economic Data
https://www.collectiveip.com/grants/NSF:0207603
New Zealand Digital Library
https://www.nzdl.org/
OAI-PMH Implementation Guidelines – Conveying rights expressions about metadata in the OAI-PMH framework
https://www.openarchives.org/OAI/2.0/guidelines-rights.htm
OAIster
https://www.oclc.org/oaister.en.html
OECD.StatExtracts – Complete Databases Available Via OECD’s iLibrary
https://data.oecd.org/
Omnity Insight Engine – Everything Connected
https://www.omnity.io/
OneLook Dictionary Search
https://www.onelook.com/
Onion Browser – An Open-Source Privacy Enhancing Web Browser for iOS
https://mike.tig.as/onionbrowser/
OnionLand Search Engine
https://onionlandsearchengine.com/
Open Archives Initiative
http://www.openarchives.org/
Open Datasets
https://www.DataPortals.org/
https://github.com/caesar0301/awesome-public-datasets
https://www.kaggle.com/datasets
https://www.data.gov/
https://www.quora.com/Where-can-I-find-large-datasets-open-to-the-public
https://aws.amazon.com/public-datasets/
https://data.world/
https://data.worldbank.org
Open Educational Resources (OER) Sources 2019 and Search Engine
https://www.OERSources.com/
https://www.OERSearchEngine.com/
OpenIndex – Creating a Public Internet Index
https://www.openindex.org/
Open Source Intelligence
https://www.oss.net/
Open Vulnerability Assessment System (OpenVAS)
https://www.darknet.org.uk/2015/01/openvas-7-released-open-source-vulnerability-scanner/
Privacy Resources 2019
http://www.PrivacyResources.info/
Project Maelstrom – The Internet We Build Next
http://blog.bittorrent.com/category/labs/
re3data.org – 2,000 Data Repositories
https://www.re3data.org/
Recommended Gateway Sites for the Deep Web
https://people.hws.edu/hunter/deepwebgate03.htm
ReportLinker: Industry Reports, Company Profiles and Market Statistics
https://www.reportlinker.com/
SAO/NASA Astrophysics Data System (ADS)
https://adswww.harvard.edu/
reSearcher
https://researcher.sfu.ca/
Science and Technology Sources on the Internet
https://www.loc.gov/rr/scitech/resources.html
Scientific and Technical Information Network (STINET)
https://www.loc.gov/flicc/Exemplars/DTIC/DTIC-STINET.PDF
Science Commons
https://creativecommons.org/science
Science.gov – FirstGov for Science – Government Science Portal
https://www.science.gov/
ScienceResearch.com – Deep Web Search Engine
https://www.scienceresearch.com/
Scientific Data Repository – Real Time Visualization and Exploration Techniques
https://www.mlvis.com/platform.php
SciTech Connect
https://www.osti.gov/scitech/
Scrapinghub Crawls the Deep Web
https://blog.scrapinghub.com/2015/02/24/memex
Scrapy Webcrawler
http://scrapy.org/
SDARTS – A Protocol and Toolkit for Metasearching
https://www.cs.columbia.edu/~gravano/Papers/2001/jcdl01b.pdf
SIMILE Widgets – Free, Open-Source Data Visualization Web Widgets and More
https://simile-widgets.org/
Social Buzz Bot 2019 (PDF download)
http://www.SocialBuzzBot.com/
STN International – Databases in Science and Technology
http://www.stn-international.de/
SurfEasy – Online Privacy
https://www.surfeasy.com/
Swoogle – Semantic Bot
https://en.wikipedia.org/wiki/Swoogle
SWRC Ontology
http://ontoware.org/swrc/
Tails – The Amnesic Incognito Live System
https://tails.boum.org/
TechDeepWeb – How-To Guide to the Deep Web for IT Professionals
http://www.TechDeepWeb.com/
Terbium Labs – Matchlight Proactive Security In an Insecure World
https://terbiumlabs.com/
The 10 Best Deep Web Search Engines to Explore the Hidden Web by Michelle Fuchs
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17217-top-10-best-deep-web-search-engines-to-explore-hidden-web
The Deep Web: Shutdowns, New Sites, New Tools by Vincenzo Ciancaglini
http://blog.trendmicro.com/trendlabs-security-intelligence/the-deep-web-shutdowns-new-sites-new-tools/
The Enterprise Onion Toolkit
https://github.com/alecmuffett/eotk
The Graph That Knows the World
https://www.datanami.com/2018/10/02/the-graph-that-knows-the-world/
The Invisible Internet Project (I2P)
https://geti2p.net/en/
The World Bank – Data
http://data.worldbank.org/
THOR: Deep Web Data Extraction
http://www.cc.gatech.edu/projects/disl/THOR/
Top 10 Best Deep Web Search Engines To Explore the Hidden Web
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/18749-top-10-best-deep-web-search-engines-to-explore-hidden-web
Tor Browser Bundle – Anonymity
https://www.torproject.org/projects/torbrowser.html.en
TOR For Newbies – When Should You Use It?
http://www.makeuseof.com/tag/tor-for-newbies/
Tor Project
https://www.torproject.org/
Tor Stack Exchange
http://tor.stackexchange.com/
TRID – The TRIS and ITRD Database (Transportation Research Board)
http://trid.trb.org/
TunnelBear – Simple, Private, Free Access to the Global Internet
https://www.tunnelbear.com/
Twitter/Search #deepweb
https://twitter.com/search?q=%23deepweb
UC Irvine Machine Learning Repository
https://archive.ics.uci.edu/ml/index.php
UNdata – Data Access System To UN Databases
http://data.un.org/
UNESCO Information Services – Databases
http://www.unesco.org/unesdi/index.php/eng/doc/tous.html
United States Census Bureau Research
https://www.census.gov/research/
Useful Tips and Tools to Research the Deep Web
http://www.online-college-blog.com/features/100-useful-tips-and-tools-to-research-the-deep-web/
US Government Web Services and XML Data Sources
http://usgovxml.com/
Virtual Private Networks Directory of Best Services
http://www.makeuseof.com/tag/best-vpn-services/
Web Data Extractors 2019
http://www.WebDataExtractors.com/
WebFountain™ – Analytical engine unstructured data
http://en.wikipedia.org/wiki/IBM_WebFountain
Web IR & IE
https://groups.yahoo.com/neo/groups/webir/info
http://www.webir.org/
WebScales: Towards a Highly Scalable Metasearch Engine
http://www.cs.binghamton.edu/~meng/pub.d/PIreport04.html
What Is a VPN – Beginner’s Guide to Virtual Private Networks
https://www.vpnsrus.com/what-is-a-vpn/
Wolfram Data Respository
https://datarepository.wolframcloud.com/
WTO Statistics Database
http://stat.wto.org/
Zanran – Search the Web for Data and Statistics
http://www.zanran.com/
Zeef – Deep Web Links
https://zeef.com/?query=deep%20web&in=all
Bot and Intelligent Agent Research Resources and Sites
api.ai – Speech Interface for Apps and Devices
http://api.ai/
Applied Soft Computing
http://www.sciencedirect.com/science/journal/15684946
Article Search API – New York Times Articles 1981 to Present
http://developer.nytimes.com/docs/article_search_api
Artificial Intelligence Resources 2019
http://www.AIResources.info/
artoo.js – The Client-Side Scraping Companion
http://medialab.github.io/artoo/
BEN Portal: BioSciEdNet
http://www.biosciednet.org/portal/
BotAnalytics – Analytics for Bots
http://botanalytics.co/
Bot and Intelligent Agent Research Resources 2019
http://www.BotResearch.info/
Botometer
https://botometer.iuni.iu.edu/
Bots, Blogs and News Aggregators 2019
http://www.BotsBlogs.com
Caffe – Deep Learning Framework
http://caffe.berkeleyvision.org/
Chatbase – Analyze and Optimize Bots More Easily
https://chatbase.com/
ChatBottle Search
https://chatbottle.co/
ChatterBot – Convert More Customers Automatically
https://chtb.io/
ChatterBots Resources and Sites 2019
http://www.ChatterBots.info/
Chatfuel – Build a Facebook Bot Without Coding
https://chatfuel.com/
Clara – Digital Employee That Schedules Meetings
https://claralabs.com/
Common Crawl – Open Repository of Web Crawl Data Composed Of Over 5 Billion Freely Available Web Pages
http://www.CommonCrawl.org/
ContusBot – Build Chatbots Super-Charged With Artificial Intelligence
https://www.contus.com/
Contentmine – Text and Data Mining Open Source Tools
http://contentmine.org/
cQuery – Content Query Engine
http://cquery.com/
CrawlTrack – Your Web Statistics Tool
http://www.crawltrack.net/
Create a Crawler – Extract Data From an Entire Website
http://import.io/
Data Mining Resources 2019
http://www.DataMiningResources.info/
Dataminr – Real-time Information Discovery
http://www.dataminr.com/
DataparkSearch Engine – Full-Featured Open Source Web-Based Search Engine
http://www.dataparksearch.org/
DataRobot – Build Better Predictive Models – Faster
http://www.datarobot.com/
Deep Web Research 2019
http://www.deepwebresearch.info/
Design of a Parallel and Distributed Web Search Engine by Salvatore Orlando, Raffaele Perego, and Fabrizio Silvestri
http://arxiv.org/abs/cs.IR/0407053
Dictionary of Algorithms and Data Structures
http://xlinux.nist.gov/dads/
Diffbot Analyze API
https://www.diffbot.com/dev/docs/analyze/
Digital Footprints – Collect Facebook Data
http://digitalfootprints.dk/
Ejenta – Intelligent Agents for Connected Care
https://www.ejenta.com/
Eliza – The Original ChatterBot
http://www-ai.ijs.si/eliza/eliza.html
Engati – Bot Innovation
http://engati.com/
Ethereum Frontier Release – A Decentralized Software Platform
https://www.ethereum.org/
EU Open Data Portal
https://data.europa.eu/euodp/en/home
EventBots by Sciensio
https://www.sciensio.com/eventbots
Facepager – Fetching Public Data From Facebook
https://github.com/strohne/Facepager
FAME (Facilitating Agents in Multiculture Exchange) Project
https://cordis.europa.eu/projects/rcn/58337_en.html
File Information Tool Set (FITS)
http://fitstool.org/
Findo – An Intelligent Search Assistant
https://findo.com/
Foundation for Intelligent Physical Agents
https://www.fipa.org/
Free Robots.txt Generator and Validator
https://www.toolsiseek.com/robots-txt-generator-validator
Friends of ChatBot Coalition
https://www.fcc.ai/
Google Guide
https://www.googleguide.com/
Gupshup – Bot Building Platform
https://www.gupshup.io/
HAIKU – Open Source Operating System That Specificially Targets Personal Computing
https://www.haiku-os.org/
Healthdata.gov
https://www.healthdata.gov/
How to Make a Web Crawler With Selenium
https://www.makeuseof.com/tag/make-web-crawler-selenium/
Huginn – Your Agents Are Standing By
https://github.com/cantino/huginn
HYPHE – Web Corpus Curation Tool Featuring A Research-Driven Web Crawler
http://hyphe.medialab.sciences-po.fr/
IBM Watson Services
https://www.ibm.com/watson/products-services/
idavatars – Intelligent Digital Avitars
http://idavatars.com/
IFR – Federation of Robotics
https://ifr.org/
Imagination Engines
http://www.imagination-engines.com/
Import.io – Turn the Web Into Data With Extractors, Crawlers and Connectors
https://import.io/
Indexing Robot Crawler Checklist
http://www.searchtools.com/robots/robot-checklist.html
Indiegogo Datasets
https://webrobots.io/indiegogo-dataset/
InfoExtractor – Extract Relevant Information from Various Sources Like Blogs, YouTube, and Wikipedia
http://www.infoextractor.org/
Institute for Human and Machine Cognition (IHMC)
http://www.ihmc.us/
Intellexer – Custom Built Search Engines, Knowledge Management Tools, Natural Language Processing
http://www.intellexer.com/
Intelligent Information Systems Research Laboratory
http://iis.ist.psu.edu/
International Journal of Agent-Oriented Software Engineering (IJAOSE)
http://www.inderscience.com/jhome.php?jcode=ijaose
jSEO – Web Crawler For Search Engine Optimization
http://codecanyon.net/item/jseo-web-crawler-for-search-engine-optimization/8770392
Keras: The Python Deep Learning Library
https://keras.io/
Kickstarter Datasets
https://webrobots.io/kickstarter-datasets/
Knowledge Discovery 2019
http://www.knowledgediscovery.info/
LAIR – Laboratory of Applied Informatics Research
http://lair.unc.edu/
List of User-Agents (Spiders, Robots, Crawler, Browser)
http://www.user-agents.org/index.shtml
Ludwig – Find Your Sentence
https://ludwig.guru/
Lurchr – I Keep Track of What’s Shared by Your Team So You Can Stay Focused On Work
https://lurchr.com/
MindMeld – Advanced Artificial Intelligence To Power the New Generation of Intelligent Conversational Interfaces
https://www.mindmeld.com/
Minimal-Intelligence Agents for Bargaining Behaviors in Market-Based Environments by Dave Cliff and Janet Bruten
http://www.hpl.hp.com/techreports/97/HPL-97-91.html
MIT Media Lab: Software Agents
http://agents.media.mit.edu/index.html
Modelling and Mining of Network Information Systems
http://www.mathstat.dal.ca/~mominis/index.html
Motion AI – ChatBots and Artificial Intelligence Made Easy
http://motion.ai/
Mozenda Web Agent Builder – Web Data Extraction
http://www.mozenda.com/
MySpiders
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.21.3013
NCapture – Capture Web Content
http://www.qsrinternational.com/products_nvivo_add-ons.aspx
Networks and agents Network (NaN)
http://cnets.indiana.edu/groups/nan/
NewsBot – Related News At a Click Of a Button
https://getnewsbot.com/
Open Educational Resources (OER) Sources Search Engine
https://www.OERSearchEngine.com/
oSkope Visual Search
https://oskope.com/
Oxyus Open Source Search Engine
http://sourceforge.net/projects/oxyus/
Postfire – AI Powered Social Media Assistant
https://postfire.co/
ProBot – Automate and Improve Your Workflow
https://probot.github.io/
Rank Dynamics – Make Your Search Dynamic
https://www.rankdynamics.com/
ResearchKit Framework – Medical Research Apps
https://github.com/ResearchKit
Robo Brain – Large Scale Computational System That Learns from Publicly Available Internet Resources
https://robobrain.me/
ReVerb – Open Information Extraction Software
https://reverb.cs.washington.edu/
SoftwareSciensio – AI Specific Events Chatbots
https://www.sciensio.com/eventbots
Scrapple – A Framework For Creating Web Scrapers and Web Crawlers
https://pypi.python.org/pypi/scrapple
Search Engine Robots
https://www.jafsoft.com/searchengines/webbots.html
Search Engine Watch News
https://www.searchenginewatch.com/
Search Tools – Information Guides and News
https://www.searchtools.com/
SeerSuite – CiteSeerX Toolkit
http://sourceforge.net/projects/citeseerx/
Semantic Web
http://www.semanticweb.org/
ShoppingBots 2019
http://www.ShoppingBots.info/
Siri – Your Virtual Personal Assistant
https://www.apple.com/ios/siri/
Sisense Bot Framework
https://www.sisense.com/product/impact/bot/
Smarter Bots 2019
http://www.SmarterBots.com/
SocialBuzzBot 2019- The Business and Social Intelligence Search Engine for Information Discovery from Social Communities
https://www.SocialBuzzBot.com/
SocSciBot – Social Sciences Link Analysis Research
http://socscibot.wlv.ac.uk/
Spidering Hacks
https://www.oreilly.com/catalog/spiderhks/
STACKS – Social Media Tracker, Analyzer, & Collector Toolkit at Syracuse
https://github.com/bitslabsyr/stack
Structure and Interpretation of Computer Programs – Video Lectures by HalAbelson and Gerald Jay Sussman
https://groups.csail.mit.edu/mac/classes/6.001/abelson-sussman-lectures/
SupportBots – Chatbots for MSP and IT Service Providers
https://www.supportbots.io/
Supybot, A Superb Python IRC Bot
https://freecode.com/projects/supybot?branch_id=31808&release_id=181322
Surf Canyon Search Engine
https://www.surfcanyon.com/
Swirlds – The Platform for Distributed Applications
https://www.swirlds.com/
Swoogle – Semantic Bot
https://en.wikipedia.org/wiki/Swoogle
TextRunner Search – Searches Hundreds of Millions of Assertions Extracted from 500 Million High-Quality Web Pages
https://openie.cs.washington.edu/
The Essential Landscape of Enterprise AI Companies
https://www.topbots.com/essential-landscape-overview-enterprise-artificial-intelligence/
The Intelligent Software Agents Lab
https://www.cs.cmu.edu/~softagents/
The Lemur Toolkit – Language Modeling and Information Retrieval Research
https://www.lemurproject.org/
The Search Engine Project (TSEP)
https://freecode.com/projects/tsep
The Simon Lavern Page
https://www.simonlaven.com/
Transcriptic – Discovery Biology On Demand
https://www.transcriptic.com/
UC Irvine Machine Learning Repository
https://archive.ics.uci.edu/ml/index.php
UMBC AgentWeb
https://agents.umbc.edu/
UMBC eBiquity
https://ebiquity.umbc.edu/
Universe – Measurement and Training for Artificial Intelligence
https://universe.openai.com/
Web Curator Tool (WCT)
https://webcurator.sourceforge.net/
Web Data Extractors 2019
https://www.WebDataExtractors.com/
Web Intelligence Consortium
https://wi-consortium.org/
Web IR & IE
https://groups.yahoo.com/neo/groups/webir/info
https://www.webir.org/
WolframAlpha Computational Knowledge Engine – Trillions of Pieces of Curated Data and Millions of Lines of Algorithms
https://www.wolframalpha.com/
Xenioo – Professional Chatbot Platform for Your Business
https://www.xenioo.com/
Zanran – Search the Web for Data and Statistics
https://www.zanran.com/