Deep Web Research and Discovery Resources 2019

How big is the Deep Web? It is estimated that the Deep Web comprises 7,500 terabytes – although an exact size is not known, and figures vary widely on this question. The magnitude, complexity and siloed nature of the Deep Web means researchers cannot turn to one specific guide or one search engine to access the vast range of information, data, files and communications that comprise it. The ubiquitous search engines (Google, Bing, Yahoo) manage, index and deliver results from the Surface web. These search results include links, data, information, reports, news, subject matter content and a large volume of advertising that is optimized to increase traffic to specific sites and support marketing and revenue focused objectives. On the other hand, the Deep Web – which is often misconstrued as a repository of dark and disreputable information [Note – it is not the Dark Web], has grown tremendously beyond that characterization to include significant content on a wide range of subject matter covering a broad swath of files and formats, databases, pay-walled content as well as communications and web traffic that is not otherwise accessible through the surface Web.

In the last several years many more applications, tools and techniques have become available that provide researchers with targeted access to specific portions of the Deep Web using discovery techniques that leverage algorithms to access data and information in many formats. Documents used by organizations and entities to communicate within their domains, including databases and files of information that may be password protected and encrypted communications, may also be accessible. Searching for this information requires that users apply search engines, techniques and search tools that function as a means of accessing these repositories of corporate, data, news and research that is not transparent using the Surface web activities. Searching the Deep Web requires the use of multiple techniques and resources – it is challenging, complicated and far less straight forward than just using Google, and much more time consuming as well. This guide providers researchers with an extensive range of sites, services, resources, tools and techniques from which to choose so as to target their efforts to locate and then utilize information and data from within the Deep Web. In addition, this guide assists researchers to identify and implement security and privacy applications while using the Deep Web, as well as offering a large number of training and learning resources to conduct more effective and efficient searches.

This Deep Web Research and Discovery Resources 2019 report and guide is divided into the following topics:

Articles, Papers, Forums, Audios and Videos
Cross Database Search Services
Cross Database Search Tools
Peer to Peer, File Sharing, Grid/Matrix Search Engines
Resources – Deep Web Research
Resources – Semantic Web Research
Bot and Intelligent Agent Research Resources and Sites

Articles, Papers, Forums, Audios and Videos

8 Best Deep Web People Search Engines

10 Things You Should Know About Dark Web Websites by Josh Fruhlinger – November 23, 2018

99 Resources to Research & Mine the Invisible Web by Jessica Hupp

A Basic Guide to Diving In To the Dark Web

A Beginner’s Guide to the Dark Web by Ben Dickson

Academic and Scholar Search Engines and Sources 2019

Accessing the Deep Web and Dark Web with Tor by Jack Jones

Access the Deep Web and Protect Your Privacy Online with Anonabox by Marco Chiappetta

All of OCLC’s WorldCat Heading Toward the Open Web by Barbara Quint

An Executive Overview of the Deep and Dark Web

An Insider’s Look Into the Dark Web by Aaron Tan

An Interactive Clustering-based Approach to Integrating Source Query interfaces on the Deep Web by W. Wu, C. Yu, A. Doan, W. Meng

Annotation for the Deep Web

An Up-To-Date Layman’s Guide To Accessing The Deep Web

A Primer on Staying Secure and Anonymous on the Dark Web by Mark Turner

Articles on Deep and Dark Web Published by Associations of Internet Research Specialists (AIRS)

Automatic Extraction of Web Search Interfaces for Interface Schema Integration by H. He, W. Meng, C. Yu, Z. Wu

Automatic Information Extraction From Semi-Structured Web Pages By Pattern Discovery

Automatic Meaning Discovery Using Google by Rudi Cilibrasi and Paul M. B. Vitanyi

Battle of the Secure Smartphones

Bibliomining for Automated Collection Development in a Digital Library Setting: Using Data Mining to Discover Web-Based Scholarly Research Works by Dr. Scott Nicholson

Big Data and AI: 30 Amazing (And Free) Public Data Sources for 2018 by Bernard Marr

Bot Research 2019

BrightPlanet Launches Deep Web Data Feeds: Global News Data Feed Is First Available Data Feed

Client-Side Deep Web Data Extraction

Clustering E-Commerce Search Engines by Q. Peng, W. Meng, H. He, C. Yu

Common Deep Web and Big Data Questions Answered (Part 1)

Common Deep Web and Big Data Questions Answered (Part 2)

Creating Intelligence from Big Data

Current Awareness Discovery Tools on the Internet 2019

Current Awareness Tools 2019

DarkCyber Beyond Search by Stephen E. Arnold

Darknet Resembles the Open Internet – Just With Anonymity by Roderick S. Graham and Brian Pitman


Dark Web (DarkNet) by Margaret Rouse

Dark Web Data Theft May Have Finally Met Its Match by Vaughn Highfield

Darkweb for Newbees: Hacking and Marketplaces by Gaurav Sangwani

Dark Web News

Dark Web Security Tips by Enjeck Mbeh Cleopatra

Dark Web Version of Facebook Shows a New Way to Secure the Web by Tom Simonite

Dark Web – CRS Report by Kristin Finklea

Deep Web by Margaret Rouse

Deep Web – Exploring the Secrets of the Hidden Internet by Marcus P. Zillman, M.S., A.M.H.A., – 23 minutes – Internet/Technology Channel

Deep Web: Legal Due Diligence by Lisa Brownlee

Deep Web Links – Massive Deep Websites Links and URL by Martin Grossner

Deep Web May Be 500 Times Bigger Than Normal Web

Deep Web Research and Discovery Resources 2019

Deep Web Search Engines to Explore the Hidden Internet by Rakesh Krishnan

Deep Web Technologies Launches Explorit! EBM, the leading Evidence-Based Medicine Search Engine for Medical Libraries

Deep Web: The Mysterious Side of the Internet and How To Access It by Michael Lowry

Deep Web: What Is It and How To Access it? – The Ultimate Guide by Samantha Carter

Digging Deeper into Deep Web Databases by Breaking Through the Top-k Barrier

Digital Shadows Launches Shadow Search

Digital Shadows Launches Shadow Search – Enabling Organizations to Self-Serve and Find Cyber Threats Against Them from Across The Open, Deep and Dark Web

Diving Into the Darknet by Michal Salat

Do’s and Don’ts of the Dark Web by Martin Beltov

Effective and Scalable Metasearch Project

Efficient Deep Web Crawling Using Reinforcement Learning

Everything You Need To Know About the Deep Web In One Simple Infographic

Experiences In Crawling Deep Web In The Context Of Local Search

First Comprehensive Map of the Dark Web Reveals a Remarkable Antisocial Corner of the Internet by Matthew Hutson

Grey Literature

Grey Literature Network Service (GreyNet)

Hackers Aggregating Data Leaks on the Dark Web

Here Are the 10 Best Deep Web Search Engines by Kristen Hubby

How Does the Tor Browser Functions? By Ben Prince

How Scary Is the Dark Web by Aria Inthavong

How To Access the Dark Web: What Is Tor and How Do I Access Dark Websites? By Victoria Woollaston

How To Access the Dark Web by Brandt Ranj

How To Access the Dark Web by Tyler Lacoma

How to Access the Dark Web (Deep Web)

How To Access the Dark Web Using Tor by Cozmo

How To Access the Deep Web Anonymously by Ritika Tiwari

How To Access the Hidden Wiki?

How To Browse the Deep Web Using Tor on iPhone and iPad by Sumeet Sharma

How To Do a Deep Web Search and Why You Should by Kelli Uhrich of

How to Mine the Invisible Wev: The Ultimate Guide by Jerri Collins

How To Safely Access The Dark Web by David Nield

How To Tread Lightly Into The Dark Web by Logan Hochstetler

How To Use a VPN with Tor

If Your Data Is Found On the Dark Web, Firefox Monitor Will Let You Know by Chuong Nyguyen

Illuminating the Dark Web by Robert Gehl

Inequality and the Intellectual Dark Web by Christian Alejandro Gonzalez

Information Retrieval and the Semantic Web by Tim Finin, James Mayfield, Clay Fink, Anupam Joshi, and R. Scott Cost

In Search of the Deep Web

Inside the Dark Web: The Truth Is There Is a Lot of Evil Out There by Conor Pope

Intellectual Dark Web: New Movement or Just a Rebarnding of Old Ideas? By Kevin Dickinson

IR and IE on the Web – PhD and MSc Dissertations

Is the Tor Browser Safe? Why Should You Use It? By Mohammed Mujtab

Journey Into the Dark by Jason Smith

Journey Into the Hidden Web: A Guide for New Researchers by Ryan Dube

Just the Tip of the Iceberg: Why You Should Be Monitoring the Deep Web

Lessons from the Deep Web That Could Lead To a More Secure IoT by Revathl Subramanian

Mining Newsgroups Using Networks Arising From Social Behavior

Mining the Deep Web: Search Strategies That Work by Lee Ratzan

Mining Topic-Specific Concepts and Definitions on the Web

NASA Is Indexing the Deep Web to Show Mankind What Google Won’t by Danielle Bronner

New AI System DARKMENTION Will Detect Upcoming Cyberattackes From Dark Web by Waqas

Next Generation TOR is Arriving and It Will Include a Distributed Random Number Generator

NYU-Poly Researcher Awarded DARPA Contract To Explore the Deep Web by Rhea Kelly

Onion Browser – An Open-Source Privacy Enhancing Web Browser for iOS

OnionLand Demystifying the Mystery of the Deep Web

OntoMiner: Bootstrapping and Populating Ontologies From Domain Specific Web Sites

OpenIndex – Creating a Public Internet Index

Out-googling Google: Federated Searching and the Single Search Box

Privacy, the Dark Web and Hacker Devices by Don Hawkins

Publications about Web Analysis, Web Search, Citation Indexing, Digital Libraries, Machine Learning, Neural Networks

QProber: Classifying and Searching “Hidden-Web” Text Databases

Really Private Browsing: An Unofficial User’sGuide to Tor by Andre Infante

Research Beyond Google: 56 Authoritative, Invisible, and Comprehensive Resources

Researchers Call for a Shared Dark Web Taxonomy by Phil Muncaster

Scan the Dark Web For Threat Intelligence by Michelle Drolet

Scientific American: Featured Article: The Semantic Web In Action

Search Engine Meeting Archives

Search Engine Technology and Digital Libraries

Searching the Dark Web for Business Intelligence by Robert Vamosi

Searching the Deep Web by Alex Wright

Searching the Deep Web

Searching the Internet 2018 – The Primer  – 2019 Guide to Searching the Internet

Search Interfaces on the Web: Querying and Characterizing by Denis Shestakov

Seeing through the ‘invisible’ Web

Semantic Web Content Accessibility Guidelines for Current Research Information Systems (CRIS) by A. Lopatenko

Shining a Light on the So-Called Dark Web by Kate Knibbs

Startup CYRCON’s Artificial Intelligence Predicts Enterprise Cyberattacks by PRWEB, November 5, 2018

Staying Safe in the Shadow of the Dark Web

Stop Talking About the Dark Web: Tor Project CoFounder Roger Dingledine

Stop Worrying About the Dark Web by Rich Matta

Structured Databases on the Web: Observations and Implications

Testbed for Information Extraction from Deep Web

The 10 Best Deep Web Search Engines to Explore the Hidden Web by Michelle Fuchs

The 10 Best Deep Web Search Engines to Explore Hidden Web by Issac Avila

The Black Market Report

The Darknet: A Quick Introduction for Business Leaders by Trey Degrassi

The Dark Web and How To Access It by MacKenzie Sigalos

The Dark Web and Remote Communication – November 10, 2018 – Tech Talk Radio

The Dark Web: What Every Parent Should Know by Cybermum Australia

The Dark Web What It Is and Staying Safe by Annie Quireshi

The Dark Web? Why Small Businesses Should Concern Themselves With the Threat by Jim Anderson

The Deep Web May Be 500 Times Bigger Than the Normal Web

The Deep Web: Surfacing Hidden Value by Michael K. Bergman;rgn=main

The Deep Web: What’s Lurking in the Underbelly of the Internet? By Michelle Alvarez 

The Future Of News: The Digital Information Librarian

The Growing Force That Will Soon Reshape The Entire Internet by Frank Minite

The Hidden Potential of the Web

The Intellectual Dark Web Conservatives Fear by Taylor Link

The Invisible Web by Chris Sherman

The Invisible Web: What it is, Why it exists, How to find it, and Its Inherent Ambiguity

The Invisible Web: Where Search Engines Fear To Go

The Most Amazing Deep Web Search Engines Finally Introduced To the World Called: .

The New Search Engines Shining a Light On the Deep Web by Carola Frediani

The Other Side of Internet by Robert Johnny, Jr.

The Problems With Searching the Deep Web by Nancy K. Herther

The Super-Private Tor Browser Get a Huge Update, But Should You Switch From Chrome?

The Ultimate Guide to the Invisible Web

The Virtual Private Library™ and The Deep Web Video by Melissa Barker

Things You Can Do On the Dark Web That Aren’t Illegal by Jacob Kleinman

This Tool Lets You Scan the Dark Web for Your (stolen) Personal Data by Anna K. Sasaki

Timeline of Events Related to the Deep Web

Top 5 Legal Reasons To Use the Darknet by JP Buntinx

Topological Measures and Maps Of the Web

TOR Browser Arrives On Android To Protect Your Secret Searches

Tor Coders Harden the Onion Against Surveillance

TOR For Newbies – When Should You Use It?

TorVPN by Max Eddy

Toward the Semantic Deep Web by James Geller, Soon Ae Chun, and Yoo Jung An

Towards Automatic Incorporation of Search Engines Into A Large-Scale Metasearch Engine

Traffic-Based Feedback on the Web by Jonathan Aizen, Daniel Huttenlocher, Jon Kleinberg, and Antal Novak

Travel Industry and Deep Web: Exclusive Interview with Marcus P. Zillman

UMBC – AgentNews

Understanding Metadata: What is Metadata, and What What Is It For? A Primer

Understanding the Deep Web In 10 Minutes

Using the Internet As a Dynamic Resource Tool for Knowledge Discovery 2018

Web Characterization Activity

Web Data Extractors 2019

Web Pages Search Engine Based on DNS by Wang Liang, Guo Yi-Ping, and Fang Ming

WebScales: Towards a Highly Scalable Metasearch Engine

What Exactly Is the Mysterious Dark Web and How It Is Different From the Deep Web by Kshitij Pujari

What Happens When You Spend a Week Fishing on the Dark Web? By Anthony Cuthbertson

What Is The Dark Web?  By David Glance

What Is the Dark Web and How Does It Work?

What Is the Dark Web and How To Access This Scary Part of Internet by Karandeep Singh

What is the Dark Web – Our Guide to the Dark Web by Laurie Clarke

What Is the Deep Web and How Can You Access It? By Adam Rowe

What Is the Deep Web? A WhatIs Podcast 15 Minute Interview with Marcus P. Zillman

What Happens When You Spend a Week Fishing On the Dark Web?

What Is Deep Web? All You Need To Know About It

What is the Invisible Web? A Crawler Perspective by Natalia Arroyo, Laboratorio de Internet

What is the Difference Between the Dark Web and the Deep Web by Laurie Clarke

What It Looks Like On the Dark Side of the Web by Felix Simon

What’s Really On the Dark Web? By Jeff Deal

Where Your Stolen Personal Details Go Online by Nicholas and Matthew Dunn

Why the Dark Web Could Be a Growing Threat to Your Small Business by Ben Lobel

Wikipedia – Deep Web

WISE-Cluster: Clustering E-Commerce Search Engines Automatically by Q. Peng, W. Meng, H. He, C. Yu


Search Tools Reports: Searching for Text Information in Databases

UK Web Archiving Consortium


FDsys – Search Across Multiple Government Databases

GovInfo – Discover Government Information

King County Library System

NLM Gateway Search

SUMSearch 2 [Health Sciences]


Bright Planet – Deep Web Intelligence


Dieselpoint Java Search and Navigation Software

Dublin Core Metadata Initiative (DCMI)

EEVL Xtra – Cross Database Search

Gold Rush – Database Search Tool


MetaSearch Initiative


Peter’s PolySearch Engines

PBCore – The Public Broadcasting Metadata Dictionary

Search Federal Research and Development 

SRU – Search/Retrieve via URL

The Flamenco Search Interface Project


ALPINE Network – SourceForge: Project

Azureus – Vuze Java Bittorrent Client

BadBlue [Uncensored News]

Between Rhizomes and Trees: P2P Information Systems by Bryn Loban


Bitmessage – P2P Communication Protocol To Send Encrypted Messages

Bit Torrent Official Site and Search Engine

Coral – The Coral P2P Content Distribution Network

Capn’s PHP Gnutella Search [Only code is available for download]

Distributed Search Engines

Distributed Search in P2P Networks

FAROO – P2P Web Search

FilesOverMiles – Browser to Browser File Sharing (P2P)

Filetopia – File sharing tool with public key encryption

Free Haven Project

Frost Project – Freenet Messaging and File Sharing Client

FuzzBox: Tangent Research Artificial Intelligence and Robotics

GNUnet – Secure P2P Networking – Free Software Foundation (FSF)

Grid, Distributed and Cloud Computing Resources 2018

GNU GRUB – Multiboot Boot Loader

Ian Clarke’s Blog

Internet Movie Database (IMDb)

IPFS Distributed Web

MoleSter – A Tiny File-Sharing Application

MusicBrainZ – Open Music Encyclopedia

MysterNetworks – The Evolution of Peer-to-Peer

OpenNap: Open Source Napster Server

P2P and the Future of Private Copying by Peter K. Yu, Michigan State University College of Law

Peer-To-Peer Wikipedia

Peer to Peer File Sharing – P2P Networking

Port Knocking

PowerFolder – P2P Whole Folder Synchronization

Rodi – Tiny P2P Client/Host


Slyck – File Sharing News and Info

Solid – Reshape the Web

Stealth Mode Online Privacy Resources 2019

Swarm – A Transparently Scalable Distributed Programming Language

The Anthill Project

The Freenet Project


Transmission – Fast, Easy and Free BitTorrent Client

Tribler – Privacy Using Our Tor-Inspired Onion Routing


Understanding BitTorrent: An Experimental Perspective by Arnaud Legout, Guillaume Urvoy-Keller, and Pietro Michiardi

WASTE (Secure P2P communication)

YaCy – Distributed P2P Based Web Indexing and Anonymous Search Engine

YouServ – A P2P (peer-to-peer) Web Hosting/File Sharing System

Zebra – Structured Text Indexing and Retrieval

Zilok – Peer To Peer Rental Marketplace


Deep Web

Searching the Internet 2018 – The Primer – 2019 Guide to Searching the Internet

The Virtual Private Library™ and The Deep Web Video by Melissa Barker

RESOURCES – Deep Web Research

6 Dark Web Pricing Trends – It Isn’t Good News 

Academic and Scholar Search Engines and Sources 2019

A Data-Driven Look At Dark Web Marketplaces

AEON (Automatic Evaluation of ONtologies)

AnkaSearch – Meta Search and Deep Web Search Desktop Tool

Anonymous Web Browsing – Wikipedia

An Up-To-Date Layman’s Guide To Accessing The Deep Web

AskReddit – What Are Your Experiences With the Deep Web

Awesome Public Datasets

BASE – Bielefeld Academic Search Engine 

Biznar – Deep Federated Search

Bot and Intelligent Agent Research Resources 2019

BrightPlanet – Deep Web Intelligence

Catalog of U.S. Government Publications (CGP)

Cazoodle – Search, Integrate, and Organize — The Real World

Censys Search Engine for Devices and Networks

CloserLook Search Engine

Creative Commons RDF-Enhanced Search

Cyber Cemetery

CyberGhost – One of the World’s Most Trusted and Secure Virtual Private Networks

Dark Reading

Dark Web Academy – Dark Web Curriculum

Dark Web Intelligence and Monitoring Service – Perfecta

DarkWeb News – Deep Web

DarkWeb News – The Ultimate Dark Web Resource

Data Mining Resources 2019

Datasets for Data Mining and Data Science

DeepDive – Analyze Data On a Deeper Level Than Ever Before


Deep Web – Discover Resources That Help You Mine the Deep or Invisible Web Instead of Just Searching the Surface

Deep Web Links – Massive Deep Websites Links and URLs

Deep Web Research and Discovery Resources 2019

Deep Web Search

Deep Web Sites

Deep Web Technologies – federated search

Deep Web University

Digital Shadows

Directory Resources 2018

eGreenBot 2018 – Green Resources Search Engine

eHealthcare Bot 2019 Deep Meta Search Engine

eMarketing Bot 2019 Deep Meta Search Engine


Engineering Village

EU Open Data Portal

Federated Search Blog

Firefox Monitor

Freely Accessible Databases for the Public

Google Fusion Tables

Google Scholar

How To Do a Deep Web Search and Why You Should

Indiegogo Datasets

Internet Archive

Invisible Library

IOGDS: International Open Government Dataset Search

Kapow Web Collector

Karma – Data Integration Tool

KDnuggets: Data Mining, Web Mining, and Knowledge Discovery Guide

Kickstarter Datasets

Knowledge Discovery 2019

Knowlesys – Use Deep Data Mining To Find the Intelligence

Large-Scale Deep Web Integration: Incomplete Bibliography

Linked Data – Connect Distributed Data Across the Web

LinkingOpenData – W3C SWEO Community Project

List of Academic Databases and Search Engines – Wikipedia

List of Tor Hidden Services from the Wikipedia


Mappa.Mundi Magazine

Mason OER Metafinder

Mednar – Innovative Medical Search

Mining the Deep Web for Economic Data

New Zealand Digital Library

OAI-PMH Implementation Guidelines – Conveying rights expressions about metadata in the OAI-PMH framework


OECD.StatExtracts – Complete Databases Available Via OECD’s iLibrary

Omnity Insight Engine – Everything Connected

OneLook Dictionary Search

Onion Browser – An Open-Source Privacy Enhancing Web Browser for iOS

OnionLand Search Engine

Open Archives Initiative

Open Datasets

Open Educational Resources (OER) Sources 2019 and Search Engine

OpenIndex – Creating a Public Internet Index

Open Source Intelligence

Open Vulnerability Assessment System (OpenVAS)

Privacy Resources 2019

Project Maelstrom – The Internet We Build Next – 2,000 Data Repositories

Recommended Gateway Sites for the Deep Web

ReportLinker: Industry Reports, Company Profiles and Market Statistics

SAO/NASA Astrophysics Data System (ADS)


Science and Technology Sources on the Internet

Scientific and Technical Information Network (STINET)

Science Commons – FirstGov for Science – Government Science Portal – Deep Web Search Engine

Scientific Data Repository – Real Time Visualization and Exploration Techniques

SciTech Connect

Scrapinghub Crawls the Deep Web

Scrapy Webcrawler

SDARTS – A Protocol and Toolkit for Metasearching

SIMILE Widgets – Free, Open-Source Data Visualization Web Widgets and More

Social Buzz Bot 2019 (PDF download)

STN International – Databases in Science and Technology

SurfEasy – Online Privacy

Swoogle – Semantic Bot

SWRC Ontology

Tails – The Amnesic Incognito Live System

TechDeepWeb – How-To Guide to the Deep Web for IT Professionals

Terbium Labs – Matchlight Proactive Security In an Insecure World

The 10 Best Deep Web Search Engines to Explore the Hidden Web by Michelle Fuchs

The Deep Web: Shutdowns, New Sites, New Tools by Vincenzo Ciancaglini

The Enterprise Onion Toolkit

The Graph That Knows the World

The Invisible Internet Project (I2P)

The World Bank – Data

THOR: Deep Web Data Extraction

Top 10 Best Deep Web Search Engines To Explore the Hidden Web

Tor Browser Bundle – Anonymity

TOR For Newbies – When Should You Use It?

Tor Project

Tor Stack Exchange

TRID – The TRIS and ITRD Database (Transportation Research Board)

TunnelBear – Simple, Private, Free Access to the Global Internet

Twitter/Search #deepweb

UC Irvine Machine Learning Repository

UNdata – Data Access System To UN Databases

UNESCO Information Services – Databases

United States Census Bureau Research

Useful Tips and Tools to Research the Deep Web

US Government Web Services and XML Data Sources

Virtual Private Networks Directory of Best Services

Web Data Extractors 2019

WebFountain™ – Analytical engine unstructured data

Web IR & IE

WebScales: Towards a Highly Scalable Metasearch Engine

What Is a VPN – Beginner’s Guide to Virtual Private Networks

Wolfram Data Respository

WTO Statistics Database

Zanran – Search the Web for Data and Statistics

Zeef – Deep Web Links

Bot and Intelligent Agent Research Resources and Sites – Speech Interface for Apps and Devices

Applied Soft Computing

Article Search API – New York Times Articles 1981 to Present

Artificial Intelligence Resources 2019

artoo.js – The Client-Side Scraping Companion

BEN Portal: BioSciEdNet

BotAnalytics – Analytics for Bots

Bot and Intelligent Agent Research Resources 2019


Bots, Blogs and News Aggregators 2019

Caffe – Deep Learning Framework

Chatbase – Analyze and Optimize Bots More Easily

ChatBottle Search

ChatterBot – Convert More Customers Automatically

ChatterBots Resources and Sites 2019

Chatfuel – Build a Facebook Bot Without Coding

Clara – Digital Employee That Schedules Meetings

Common Crawl – Open Repository of Web Crawl Data Composed Of Over 5 Billion Freely Available Web Pages

ContusBot – Build Chatbots Super-Charged With Artificial Intelligence

Contentmine – Text and Data Mining Open Source Tools

cQuery – Content Query Engine

CrawlTrack – Your Web Statistics Tool

Create a Crawler – Extract Data From an Entire Website

Data Mining Resources 2019

Dataminr – Real-time Information Discovery

DataparkSearch Engine – Full-Featured Open Source Web-Based Search Engine

DataRobot – Build Better Predictive Models – Faster

Deep Web Research 2019

Design of a Parallel and Distributed Web Search Engine by Salvatore Orlando, Raffaele Perego, and Fabrizio Silvestri

Dictionary of Algorithms and Data Structures

Diffbot Analyze API

Digital Footprints – Collect Facebook Data

Ejenta – Intelligent Agents for Connected Care

Eliza – The Original ChatterBot

Engati – Bot Innovation

Ethereum Frontier Release – A Decentralized Software Platform

EU Open Data Portal

EventBots by Sciensio

Facepager – Fetching Public Data From Facebook

FAME (Facilitating Agents in Multiculture Exchange) Project

File Information Tool Set (FITS)

Findo – An Intelligent Search Assistant

Foundation for Intelligent Physical Agents

Free Robots.txt Generator and Validator

Friends of ChatBot Coalition

Google Guide

Gupshup – Bot Building Platform

HAIKU – Open Source Operating System That Specificially Targets Personal Computing

How to Make a Web Crawler With Selenium

Huginn – Your Agents Are Standing By

HYPHE – Web Corpus Curation Tool Featuring A Research-Driven Web Crawler

IBM Watson Services

idavatars – Intelligent Digital Avitars

IFR – Federation of Robotics

Imagination Engines – Turn the Web Into Data With Extractors, Crawlers and Connectors

Indexing Robot Crawler Checklist

Indiegogo Datasets

InfoExtractor – Extract Relevant Information from Various Sources Like Blogs, YouTube, and Wikipedia

Institute for Human and Machine Cognition (IHMC)

Intellexer – Custom Built Search Engines, Knowledge Management Tools, Natural Language Processing

Intelligent Information Systems Research Laboratory

International Journal of Agent-Oriented Software Engineering (IJAOSE)

jSEO – Web Crawler For Search Engine Optimization

Keras: The Python Deep Learning Library

Kickstarter Datasets

Knowledge Discovery 2019

LAIR – Laboratory of Applied Informatics Research

List of User-Agents (Spiders, Robots, Crawler, Browser)

Ludwig – Find Your Sentence

Lurchr – I Keep Track of What’s Shared by Your Team So You Can Stay Focused On Work

MindMeld – Advanced Artificial Intelligence To Power the New Generation of Intelligent Conversational Interfaces

Minimal-Intelligence Agents for Bargaining Behaviors in Market-Based Environments by Dave Cliff and Janet Bruten

MIT Media Lab: Software Agents

Modelling and Mining of Network Information Systems

Motion AI – ChatBots and Artificial Intelligence Made Easy

Mozenda Web Agent Builder – Web Data Extraction


NCapture – Capture Web Content

Networks and agents Network (NaN)

NewsBot – Related News At a Click Of a Button

Open Educational Resources (OER) Sources Search Engine

oSkope Visual Search

Oxyus Open Source Search Engine

Postfire – AI Powered Social Media Assistant

ProBot – Automate and Improve Your Workflow

Rank Dynamics – Make Your Search Dynamic

ResearchKit Framework – Medical Research Apps

Robo Brain – Large Scale Computational System That Learns from Publicly Available Internet Resources

ReVerb – Open Information Extraction Software

SoftwareSciensio – AI Specific Events Chatbots

Scrapple – A Framework For Creating Web Scrapers and Web Crawlers

Search Engine Robots

Search Engine Watch News

Search Tools – Information Guides and News

SeerSuite – CiteSeerX Toolkit

Semantic Web

ShoppingBots 2019

Siri – Your Virtual Personal Assistant

Sisense Bot Framework

Smarter Bots 2019

SocialBuzzBot 2019- The Business and Social Intelligence Search Engine for Information Discovery from Social Communities

SocSciBot – Social Sciences Link Analysis Research

Spidering Hacks

STACKS – Social Media Tracker, Analyzer, & Collector Toolkit at Syracuse

Structure and Interpretation of Computer Programs – Video Lectures by HalAbelson and Gerald Jay Sussman

SupportBots – Chatbots for MSP and IT Service Providers

Supybot, A Superb Python IRC Bot

Surf Canyon Search Engine

Swirlds – The Platform for Distributed Applications

Swoogle – Semantic Bot

TextRunner Search – Searches Hundreds of Millions of Assertions Extracted from 500 Million High-Quality Web Pages

The Essential Landscape of Enterprise AI Companies

The Intelligent Software Agents Lab

The Lemur Toolkit – Language Modeling and Information Retrieval Research

The Search Engine Project (TSEP)

The Simon Lavern Page

Transcriptic – Discovery Biology On Demand

UC Irvine Machine Learning Repository

UMBC AgentWeb

UMBC eBiquity

Universe – Measurement and Training for Artificial Intelligence

Web Curator Tool (WCT)

Web Data Extractors 2019

Web Intelligence Consortium

Web IR & IE

WolframAlpha Computational Knowledge Engine – Trillions of Pieces of Curated Data and Millions of Lines of Algorithms

Xenioo – Professional Chatbot Platform for Your Business

Zanran – Search the Web for Data and Statistics

Posted in: Big Data, Business Research, Competitive Intelligence, Discovery, Encryption, Privacy, Search Engines, Search Strategies