Deep Web Research and Discovery Resources 2017

Deep Web Research and Discovery Resources 2017 is a keynote presentation I have been delivering that presents resources from the extensive research I have completed over many years into using the “deep” web. The Deep Web covers trillions of pages of information in various files and formats that the current search engines on the internet either cannot find or have difficulty indexing in their respective search results. Search engines currently index and can locate hundreds of billions of pages. In the last several years, some of the more comprehensive search engines have written algorithms to search the deeper portions of the world wide web by attempting to find files such as .pdf, .doc, .xls, ppt, .ps. and other file formats such as sound and moving images videos, podcasts and film, software, games, and APIs and code hosted in Github. These files are predominately used by businesses to communicate their information within their organization or to disseminate information to external clients and to users, from their organization. Searching for this information using deeper search techniques and the latest algorithms allows researchers to obtain a vast amount of corporate information that was previously unavailable or inaccessible. Research has also shown that even deeper information can be obtained from these files by searching and accessing the “properties” information on these files.

This guide provides a wealth of resources to better understand the evolution and changing tools used for deep web research. It provides researchers with a comprehensive and wide ranging group of “deep web” data, information, documents, code, applications and cutting edge tools to use as key drivers in building queries that result in strategic, actionable knowledge, regardless of what sector you work in.

This guide includes resources hosted and published by government, academic, corporate, advocacy, private enterprise, individual experts, national and international consortium, news organizations, and sponsored research papers. This guide includes resources hosted and published by government, academic, corporate, libraries, advocacy groups, private enterprise, individual experts, national and international consortium, news organizations, and sponsored research papers. [Editor’s note – several of the sources listed require registration for download and/or purchase].

This Deep Web Research and Discovery Resources 2017 report and guide is divided into the following sections:

ARTICLES, PAPERS, FORUMS, AUDIOS AND VIDEOS (Current and Historical)

8 Best Deep Web People Search Engines
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/16323-8-best-deep-web-people-search-engines-updated

99 Resources to Research & Mine the Invisible Web by Jessica Hupp
http://www.collegedegree.com/library/college-life/99-resources-to/

Academic and Scholar Search Engines and Sources
http://www.ScholarSearchEngines.com/

Access the Deep Web and Protect Your Privacy Online with Anonabox by Marco Chiappetta
http://www.forbes.com/sites/marcochiappetta/2016/04/29/access-the-deep-web-and-protect-your-privacy-online-with-the-anonabox/#72cd0dc337c2

All of OCLC’s WorldCat Heading Toward the Open Web by Barbara Quint
http://newsbreaks.infotoday.com/nbreader.asp?ArticleID=16353

An Interactive Clustering-based Approach to Integrating Source Query interfaces on the Deep Web by W. Wu, C. Yu, A. Doan, W. Meng
http://www.cs.binghamton.edu/~meng/pub.d/sigmod04-final.pdf

Annotation for the Deep Web
http://dl.acm.org/citation.cfm?id=1137372

An Up-To-Date Layman’s Guide To Accessing The Deep Web
http://www.fastcolabs.com/3026989/an-up-to-date-laymans-guide-to-accessing-the-deep-web

A Primer on Staying Secure and Anonymous on the Dark Web by Mark Turner
http://www.techspot.com/guides/1292-web-security-anonymizer-primer/

Automatic Extraction of Web Search Interfaces for Interface Schema Integration by H. He, W. Meng, C. Yu, Z. Wu
http://www.cs.binghamton.edu/~meng/pub.d/WWWposterhe.pdf

Automatic Information Extraction From Semi-Structured Web Pages By Pattern Discovery
http://dl.acm.org/citation.cfm?id=640423&dl=ACM&coll=portal

Automatic Meaning Discovery Using Google by Rudi Cilibrasi and Paul M. B. Vitanyi
http://arxiv.org/abs/cs.CL/0412098

Battle of the Secure Smartphones
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17276-battle-of-the-secure-smartphones

Beyond Google: The Invisible Web – Tools for Teaching the Invisible Web
http://library.laguardia.edu/invisibleweb/teachingtools

Bibliomining for Automated Collection Development in a Digital Library Setting: Using Data Mining to Discover Web-Based Scholarly Research Works by Dr. Scott Nicholson
http://scottnicholson.com/pubs/asisdiss.pdf

Bot Research
http://www.BotResearch.info/

BrightPlanet Launches Deep Web Data Feeds: Global News Data Feed Is First Available Data Feed
http://www.brightplanet.com/2015/01/brightplanet-launches-deep-web-data-feeds-global-news-data-feed-first-available-data-feed/

Client-Side Deep Web Data Extraction
http://www.computer.org/csdl/proceedings/cec-east/2004/2206/00/22060158-abs.html

Clustering E-Commerce Search Engines by Q. Peng, W. Meng, H. He, C. Yu
http://www.cs.binghamton.edu/~meng/pub.d/WWWposterPeng.pdf

Common Deep Web and Big Data Questions Answered (Part 1)
http://www.brightplanet.com/2014/11/common-deep-web-big-data-questions-answered-part-1/

Common Deep Web and Big Data Questions Answered (Part 2)
http://www.brightplanet.com/2014/12/common-deep-web-big-data-questions-answered-part-2/

Creating Intelligence from Big Data
http://bigdata.brightplanet.com/creating-new-intelligence-from-big-data

Current Awareness Discovery Tools on the Internet 2017
http://www.zillman.us/white-papers/current-awareness-discovery-tools-on-the-internet/

Dark Web Version of Facebook Shows a New Way to Secure the Web by Tom Simonite
https://www.technologyreview.com/s/532256/dark-web-version-of-facebook-shows-a-new-way-to-secure-the-web/

Data Extraction and Label Assignment for Web Databases
http://www2003.org/cdrom/papers/refereed/p470/p470-wang.htm

Deep Web – Exploring the Secrets of the Hidden Internet by Marcus P. Zillman, M.S., A.M.H.A., – 23 minutes – Internet/Technology Channel
http://www.planetearthradio.com/technology.htm

Deep Web: Legal Due Diligence by Lisa Brownlee
http://federatedsearchblog.com/2015/04/29/deep-web-legal-due-diligence/

Desperately Seeking Web Search 2.0
http://news.netcraft.com/archives/2004/04/23/desperately_seeking_web_search_20.html

Digging Deeper into Deep Web Databases by Breaking Through the Top-k Barrier
http://arxiv.org/abs/1208.3876

DigiCULT Thematic Issue 6
Resource Discovery Technologies for the Heritage Sector, June 2004
http://www.digicult.info/downloads/digicult_thematic_issue6.pdf

Effective and Scalable Metasearch Project
http://www.cs.binghamton.edu/~meng/metasearch.html

Efficient Deep Web Crawling Using Reinforcement Learning
http://link.springer.com/chapter/10.1007%2F978-3-642-13657-3_46

Everything You Need To Know About the Deep Web In One Simple Infographic
http://www.businessinsider.com/everything-you-need-to-know-about-the-deep-web-in-one-simple-infographic-2015-2

Experiences In Crawling Deep Web In The Context Of Local Search
http://dl.acm.org/citation.cfm?id=1460016

Grey Literature
http://en.wikipedia.org/wiki/Gray_literature

Grey Literature Network Service (GreyNet)
http://www.greynet.org/

Here Are the 10 Best Deep Web Search Engines by Kristen Hubby
http://www.dailydot.com/layer8/best-deep-web-search-engines/

How To Access the Dark Web by Brandt Ranj
http://www.independent.co.uk/life-style/gadgets-and-tech/how-to-access-the-dark-web-a7047041.html

How To Access the Dark Web Using Tor by Cozmo
http://mixance.com/how-to-access-the-dark-web-using-tor/

How To Browse the Deep Web Using Tor on iPhone and iPad by Sumeet Sharma
http://www.iappnalysis.com/2015/04/browse-deep-web-on-iphone-ipad-using-tor-browser.html

How To Use the Secure Encrypted Tor Messenger by Ray Walsh
https://www.bestvpn.com/blog/30751/tor-project-releases-secure-encrypted-tor-messenger-how-to-use-it/

Information Retrieval and the Semantic Web by Tim Finin, James Mayfield, Clay Fink, Anupam Joshi, and R. Scott Cost
http://ebiquity.umbc.edu/paper/html/id/185/

In Search of the Deep Web
http://www.salon.com/2004/03/09/deep_web/

IR and IE on the Web – PhD and MSc Dissertations
https://groups.yahoo.com/neo/groups/webir/info
http://www.webir.org/

Journey Into the Hidden Web: A Guide for New Researchers by Ryan Dube
http://www.makeuseof.com/tag/journey-into-the-hidden-web-a-guide-for-new-researchers/

Just the Tip of the Iceberg: Why You Should Be Monitoring the Deep Web
http://www.information-age.com/technology/security/123461668/just-tip-iceberg-why-you-should-be-monitoring-deep-web

Lessons from the Deep Web That Could Lead To a More Secure IoT by Revathl Subramanian
http://blogs.ca.com/2015/04/02/lessons-from-the-deep-web-that-could-lead-to-a-more-secure-iot/?mrm=425878&cid=GLOB-SMM-ABUS-AAR-000002-00000571

Metadata? Thesauri? Taxonomies? Topic Maps! by Lars Marius Garshol
http://www.ontopia.net/topicmaps/materials/tm-vs-thesauri.html

Mining Newsgroups Using Networks Arising From Social Behavior
http://www.almaden.ibm.com/cs/projects/iis/hdb/Publications/papers/www03_social.pdf

Mining the Deep Web: Search Strategies That Work by Lee Ratzan
http://www.computerworld.com/s/article/9005757/Mining_the_Deep_Web_Search_strategies_that_work?pageNumber=1

Mining Topic-Specific Concepts and Definitions on the Web
http://www.cs.uic.edu/~liub/publications/WWW-2003.pdf

NASA Is Indexing the Deep Web to Show Mankind What Google Won’t by Danielle Bronner
http://fusion.net/story/145885/nasa-is-indexing-the-deep-web-to-show-mankind-what-google-wont/

Next Generation TOR is Arriving and It Will Include a Distributed Random Number Generator
http://securityaffairs.co/wordpress/47683/deep-web/distributed-tor-random-number-generator.html

NYU-Poly Researcher Awarded DARPA Contract To Explore the Deep Web by Rhea Kelly
http://campustechnology.com/articles/2015/02/25/nyu-poly-researchers-awarded-darpa-contract-to-explore-the-deep-web.aspx

Onion Browser – An Open-Source Privacy Enhancing Web Browser for iOS
https://mike.tig.as/onionbrowser/

OntoMiner: Bootstrapping and Populating Ontologies From Domain Specific Web Sites
http://www.public.asu.edu/~hdavulcu/VLDB-WS03.pdf

OpenIndex – Creating a Public Internet Index
http://www.openindex.org

Out-googling Google: Federated Searching and the Single Search Box
http://library.marist.edu/ACRL/Foxhunt_demo.html

QProber: Classifying and Searching “Hidden-Web” Text Databases
http://qprober.cs.columbia.edu/

Really Private Browsing: An Unofficial User’s Guide to Tor by Andre Infante
http://www.makeuseof.com/tag/really-private-browsing-an-unofficial-users-guide-to-tor/

Research Beyond Google: 56 Authoritative, Invisible, and Comprehensive Resources
http://oedb.org/ilibrarian/research-beyond-google/

Scientific American: Featured Article: The Semantic Web
http://www.sciam.com/article.cfm?id=the-semantic-web

Search Engine Meeting
http://www.SearchEngineMeeting.net/

Search Engine Technology and Digital Libraries
http://www.dlib.org/dlib/june04/lossau/06lossau.html

Searching the Deep Web by Alex Wright
http://mags.acm.org/communications/200810/?pg=16

Searching the Deep Web
http://www.dlib.org/dlib/january01/warnick/01warnick.html

Searching the Deep Web – Video
http://www.osti.gov/media/DeepWebVideo.html

Searching the Internet 2017 – A Primer
http://www.SearchingTheInternet.info/

Search Interfaces on the Web: Querying and Characterizing by Denis Shestakov
https://www.doria.fi/handle/10024/38506

Seeing through the ‘invisible’ Web
http://usatoday30.usatoday.com/tech/2001/10/15/invisible-web-search.htm

Semantic Web Content Accessibility Guidelines for Current Research Information Systems (CRIS) by A. Lopatenko
http://derpi.tuwien.ac.at/~andrei/AURIS_DE.htm

Structured Databases on the Web: Observations and Implications
http://dl.acm.org/citation.cfm?id=1031584

Testbed for Information Extraction from Deep Web
http://research.microsoft.com/users/nickcr/pubs/yamada_www2004poster.pdf

The 10 Best Deep Web Search Engines to Explore the Hidden Web by Michelle Fuchs
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17217-top-10-best-deep-web-search-engines-to-explore-hidden-web

The Deep Web: Surfacing Hidden Value by Michael K. Bergman
http://quod.lib.umich.edu/j/jep/3336451.0007.104?view=text;rgn=main

The Darknet: A Quick Introduction for Business Leaders by Trey Degrassi
https://hbr.org/2016/12/the-darknet-a-quick-introduction-for-business-leaders

The Deep Web: What’s Lurking in the Underbelly of the Internet? By Michelle Alvarez
http://securityintelligence.com/the-deep-web-whats-lurking-in-the-underbelly-of-the-internet/#.VUIUNmfD9D8

The Future Of News: The Digital Information Librarian
http://www.masternewmedia.org/2004/03/24/the_future_of_news_the.htm

The Hidden Potential of the Web
http://www.theguardian.com/society/2004/apr/21/epublic.technology18

The Invisible Web by Chris Sherman
http://web.freepint.com/go/newsletter/64#feature

The Invisible Web: What it is, Why it exists, How to find it, and Its Inherent Ambiguity
http://www.newworldencyclopedia.org/entry/Deep_Web

The Invisible Web: Where Search Engines Fear To Go
http://www.powerhomebiz.com/vol25/invisible.htm

The New Search Engines Shining a Light On the Deep Web by Carola Frediani
http://kernelmag.dailydot.com/issue-sections/features-issue-sections/10376/how-to-search-deep-web-tor/

The Ultimate Guide to the Invisible Web
http://oedb.org/ilibrarian/invisible-web/

The Virtual Private Library™ and The Deep Web Video by Melissa Barker
http://zillman.blogspot.com/2009/07/virtual-private-library-and-deep-web.html

This Tool Lets You Scan the Dark Web for Your (stolen) Personal Data by Anna K. Sasaki
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17251-this-tool-lets-you-scan-the-dark-web-for-your-stolen-personal-data

Timeline of Events Related to the Deep Web
http://papergirls.wordpress.com/2008/10/07/timeline-deep-web/

Topological Measures and Maps Of the Web
http://informatics.indiana.edu/fil/Web/

Tor Coders Harden the Onion Against Surveillance
https://nakedsecurity.sophos.com/2016/06/23/tor-coders-harden-the-onion-against-surveillance/

TOR For Newbies – When Should You Use It?
http://www.makeuseof.com/tag/tor-for-newbies/

TorVPN by Max Eddy
http://uk.pcmag.com/torvpn/19165/review/lockspin-torvpn

Toward the Semantic Deep Web by James Geller, Soon Ae Chun, and Yoo Jung An
http://www.mendeley.com/catalog/toward-semantic-deep-web/

Towards Automatic Incorporation of Search Engines Into A Large-Scale Metasearch Engine
http://www.cs.binghamton.edu/~meng/pub.d/wi2003.pdf

Traffic-Based Feedback on the Web by Jonathan Aizen, Daniel Huttenlocher, Jon Kleinberg, and Antal Novak
http://www.pnas.org/content/101/suppl_1/5254.abstract

Travel Industry and Deep Web: Exclusive Interview with Marcus P. Zillman
http://plrplr.com/90014/deep-web-and-travel-industry-exclusive-interview-with-marcus-p-zillman/

UMBC – AgentNews
http://agents.umbc.edu/

Understanding Metadata
http://www.niso.org/standards/resources/UnderstandingMetadata.pdf

Understanding the Deep Web In 10 Minutes
http://www.brightplanet.com/2013/03/whitepaper-understanding-the-deep-web-in-10-minutes/

Using the Internet As a Dynamic Resource Tool for Knowledge Discovery 2017
http://www.zillman.us/white-papers/using-the-internet-as-a-dynamic-resource-tool-for-knowledge-discovery/

Web Characterization Activity
http://www.w3.org/WCA/

Web Data Extractors 2017
http://www.WebDataExtractors.com/

Web Pages Search Engine Based on DNS by Wang Liang, Guo Yi-Ping, and Fang Ming
http://arxiv.org/pdf/cs.NI/0403035

WebScales: Towards a Highly Scalable Metasearch Engine
http://www.cs.binghamton.edu/~meng/pub.d/PIreport04.html

What Is The Dark Web? By David Glance
http://www.iflscience.com/technology/what-dark-web/

What Is the Deep Web? A WhatIs Podcast 15 Minute Interview with Marcus P. Zillman
http://zillman.blogspot.com/2006/10/what-is-deep-web.html

What is the Invisible Web? A Crawler Perspective by Natalia Arroyo, Laboratorio de Internet
http://cybermetrics.wlv.ac.uk/AoIRASIST/arroyo.html

Wikipedia – Deep Web
http://en.wikipedia.org/wiki/Deep_web

WISE-Cluster: Clustering E-Commerce Search Engines Automatically by Q. Peng, W. Meng, H. He, C. Yu
http://www.cs.binghamton.edu/~meng/pub.d/PengWIDM04.pdf

CROSS DATABASE ARTICLES

Search Tools Reports: Searching for Text Information in Databases
http://www.searchtools.com/info/database-search.html

The Right Solution: Federated Search Tools by Roy Tennant
http://lj.libraryjournal.com/2003/06/ljarchives/the-right-solution-federated-search-tools/

UK Web Archiving Consortium
http://www.webarchive.org.uk

CROSS DATABASE SEARCH SERVICES

EnergyFiles – Subject Pathways [Oil Gas production and forecasting]
http://energyfiles.com/

FDsys – Search Across Multiple Government Databases
http://www.gpo.gov/fdsys/

King County Library System
http://www.kcls.org/

NLM Gateway Search
http://wwwcf.nlm.nih.gov/hsr_project/home_proj.cfm

SUMSearch 2 [Health Sciences]
http://sumsearch.org/

CROSS DATABASE SEARCH TOOLS

Bright Planet – Deep Web Intelligence
http://brightplanet.com/

Copernic – fee-based desk top seach
http://www.copernic.com/

Dieselpoint Java Search and Navigation Software
http://www.dieselpoint.com/

Dublin Core Metadata Initiative (DCMI)
http://www.dublincore.org/

EEVL Xtra – Cross Database Search
http://www.ariadne.ac.uk/issue44/eevl/

Gold Rush – Database Search Tool
http://goldrush.coalliance.org/

MetaLib
http://www.exlibrisgroup.com/category/MetaLibOverview

MetaSearch Initiative
http://www.niso.org/workrooms/mi

MuseGlobal
http://www.museglobal.com/

Peter’s PolySearch Engines
http://www2.hawaii.edu/~jacso/extra/poly-page.html

PBCore – The Public Broadcasting Metadata Dictionary
http://www.pbcore.org/

Registry of Library Knowledge Bases
http://www.public.iastate.edu/~CYBERSTACKS/KBL.htm

Search Federal Research and Development
http://www.osti.gov/

SRU – Search/Retrieve via URL
http://www.loc.gov/standards/sru

The Flamenco Search Interface Project
http://flamenco.berkeley.edu/

VIAF – The Virtual International Authority File
http://www.oclc.org/viaf.en.html

PEER TO PEER (P2P), FILE SHARING, GRID AND MATRIX SEARCH ENGINES

ALPINE Network – SourceForge: Project
http://sourceforge.net/projects/alpine/

Azureus – Vuze Java Bittorrent Client
http://www.vuze.com/

BadBlue [Uncensored News]
http://badblue.com/

Between Rhizomes and Trees: P2P Information Systems by Bryn Loban
http://firstmonday.org/ojs/index.php/fm/article/view/1182

BigChampagne
http://www.bigchampagne.com/

Bitmessage – P2P Communication Protocol To Send Encrypted Messages
https://bitmessage.org/wiki/Main_Page

Bit Torrent Official Site and Search Engine
http://www.BitTorrent.com/

Coral – The Coral P2P Content Distribution Network
http://www.coralcdn.org/

Capn’s PHP Gnutella Search [Only code is available for download]
http://capnbry.net/gnutella/gs.php

ClearBits – BitTorrent distribution of open licensed media
https://twitter.com/clearbits

Distributed Search Engines
http://www.openp2p.com/pub/t/74

Distributed Search in P2P Networks
http://www.computer.org/csdl/mags/ic/2002/01/w1068-abs.html

FAROO – P2P Web Search
http://www.faroo.com/

FilesOverMiles – Browser to Browser File Sharing (P2P)
http://www.filesovermiles.com/

Filetopia – File sharing tool with public key encryption
http://www.filetopia.org/

Free Haven Project
http://www.freehaven.net

Frost Project – Freenet Messaging and File Sharing Client
http://jtcfrost.sourceforge.net/

FuzzBox: Tangent Research Artificial Intelligence and Robotics
http://tangentresearch.com/news/07252001_p2p_ai.html

GNUnet – Secure P2P Networking – Free Software Foundation (FSF)
https://gnunet.org/

Grid, Distributed and Cloud Computing Resources 2017
http://www.GridResources.info/

GNU GRUB – Multiboot Boot Loader
http://www.gnu.org/software/grub/

Ian Clarke’s Blog
http://blog.locut.us/

infinit – Re-imaging the Way You Send Files
https://infinit.io/

Internet Movie Database (IMDb)
http://www.imdb.com/

MoleSter – A Tiny File-Sharing Application
http://ansuz.sooke.bc.ca/software/molester/

MusicBrainZ – Open Music Encyclopedia
http://www.MusicBrainZ.org/

MysterNetworks – The Evolution of Peer-to-Peer
http://www.mysternetworks.com/

Open Directory – File Sharing
http://dmoz.org/Computers/Software/Internet/Clients/File_Sharing/

Open Directory – MP3 Search Engines
http://dmoz.org/Arts/Music/Sound_Files/MP3/Search_Engines/

OpenNap: Open Source Napster Server
http://opennap.sourceforge.net/

OpenP2P.com
http://www.openp2p.com/

P2P and the Future of Private Copying by Peter K. Yu, Michigan State University College of Law
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=578568

Peer-To-Peer Wikipedia
http://en.wikipedia.org/wiki/Peer-to-peer

Peer to Peer File Sharing – P2P Networking
http://compnetworking.about.com/od/p2ppeertopeer/Peer_to_Peer_File_Sharing_P2P_Networking.htm

Port Knocking
http://www.portknocking.org/

PowerFolder – P2P Whole Folder Synchronization
http://www.powerfolder.com/

Rodi – Tiny P2P Client/Host
http://rodi.sourceforge.net/

Skype
http://www.skype.com/

Slyck – File Sharing News and Info
http://www.slyck.com/

Stealth Mode Online Privacy Resources 2017
http://www.StealthMode.info/

Swarm – A Transparently Scalable Distributed Programming Language
http://swarmframework.org/

The Anthill Project
http://www.cs.unibo.it/projects/anthill/

The Freenet Project
http://freenetproject.org/

ToPeer
http://www.2peer.com/

Transmission – Fast, Easy and Free BitTorrent Client
http://www.transmissionbt.com/

Tribler – A Social Community That Facilitates Filesharing Through P2P
http://www.tribler.org/

TrustyFiles
http://www.trustyfiles.com/

Understanding BitTorrent: An Experimental Perspective by Arnaud Legout, Guillaume Urvoy-Keller, and Pietro Michiardi
http://hal.inria.fr/inria-00000156/en

YaCy – Distributed P2P Based Web Indexing and Anonymous Search Engine
http://www.yacy.net/

YouServ – A P2P (peer-to-peer) Web Hosting/File Sharing System
http://www.bayardo.org/youserv/

Zebra – Structured Text Indexing and Retrieval
http://www.indexdata.com/zebra

Zilok – Peer To Peer Rental Marketplace
http://zilok.com/

PRESENTATIONS

Deep Web
http://whatis.techtarget.com/definition/deep-Web

Deep Web Video
http://www.osti.gov/media/DeepWebVideo.html

From Theory To Practice – Bielefeld Academic Search Engine
http://www.diglib.org/forums/spring2004/presentations/summann-2004-04.pdf

Searching the Internet Whitepaper
http://www.SearchingTheInternet.info/

The Virtual Private Library™ and The Deep Web Video by Melissa Barker
http://zillman.blogspot.com/2009/07/virtual-private-library-and-deep-web.html

RESOURCES – Deep Web Research

AEON (Automatic Evaluation of ONtologies)
http://code.google.com/p/aeon-project/

AnkaSearch – Meta Search and Deep Web Search Desktop Tool
http://www.ankasoftware.com/ankasearch.html

Anonymous Web Browsing – Wikipedia
http://en.wikipedia.org/wiki/Anonymous_web_browsing

An Up-To-Date Layman’s Guide To Accessing The Deep Web
http://www.fastcolabs.com/3026989/an-up-to-date-laymans-guide-to-accessing-the-deep-web

AskReddit – What Are Your Experiences With the Deep Web
http://www.reddit.com/r/AskReddit/comments/lm4dl/reddit_what_are_your_experiences_in_the_deep_web/

BASE – Bielefeld Academic Search Engine
http://www.base-search.net/

Biznar – Deep Federated Search
http://www.deepwebtech.com/2015/08/biznar-a-deep-web-google-alternative/

Bot Research
http://www.BotResearch.info/

BrightPlanet – Deep Web Intelligence
http://www.brightplanet.com/

Catalog of U.S. Government Publications (CGP)
http://catalog.gpo.gov/

Cazoodle – Search, Integrate, and Organize — The Real World
http://www.cazoodle.com/

Censys Search Engine for Devices and Networks
https://censys.io/

Creative Commons RDF-Enhanced Search
http://search.creativecommons.org/

Cyber Cemetery
http://govinfo.library.unt.edu/

CyberGhost – One of the World’s Most Trusted and Secure Virtual Private Networks
http://www.cyberghostvpn.com/

Dark Web Academy – Dark Web Curriculum
https://www.youtube.com/channel/UCJ31aJo8U-ZaRnZ4Y27so_Q

DarkWeb News – The Ultimate Dark Web Resource
https://darkwebnews.com/

Data Mining Resources
http://www.DataMiningResources.info/

DeepDive – Analyze Data On a Deeper Level Than Ever Before
http://deepdive.stanford.edu/

Deep.Dot.Web
http://www.DeepDotWeb.com/

Deep Web – Discover Resources That Help You Mine the Deep or Invisible Web Instead of Just Searching the Surface
http://libguides.msubillings.edu/c.php?g=242182&p=1610131

Deep Web Research and Discovery Resources 2017
http://www.DeepWebResearch.info/
http://www.DeepWeb.us/

Deep Web Search
http://deep-web.org/

Deep Web Sites
http://www.deepweb-sites.com/

Deep Web Technologies – federated search
http://www.deepwebtech.com/

Directory Resources
http://www.DirectoryResources.info/

eFinancial Bot Deep Meta Search Engine
http://www.eFinancialBot.com/

eGreenBot – Green Resources Search Engine
http://www.eGreenBot.com/

eHealthcare Bot 2017 Deep Meta Search Engine
http://www.eHealthcareBot.com/

eMarketing Bot 2017 Deep Meta Search Engine
http://www.eMarketingBot.com/

ENDECA
http://www.oracle.com/us/products/applications/commerce/endeca/overview/index.html

Engineering Village
http://www.engineeringvillage.com

Falcons Semantic Web Search Engine
http://ws.nju.edu.cn/falcons/objectsearch/index.jsp

Federated Search Blog
http://federatedsearchblog.com/

Freely Accessible Databases for the Public
http://www.istl.org/01-winter/internet.html

Google Fusion Tables
http://www.google.com/drive/apps.html#fusiontables

Google Scholar
http://scholar.google.com/

HighWire Press – Largest Repository of Free Full-Text Life Science Articles in the World
http://highwire.stanford.edu/

Internet Archive
http://www.archive.org/

Invisible Library
http://invislib.blogspot.com/

Kapow Web Collector
http://www.automated-info-solutions.com/

Karma – Data Integration Tool
http://www.isi.edu/integration/karma/

KDnuggets: Data Mining, Web Mining, and Knowledge Discovery Guide
http://www.kdnuggets.com/

Knowledge Discovery
http://www.KnowledgeDiscovery.info/

Knowlesys – Use Deep Data Mining To Find the Intelligence
http://knowlesys.com/

Large-Scale Deep Web Integration: Incomplete Bibliography
http://metaquerier.cs.uiuc.edu/webibib.html

Linked Data – Connect Distributed Data Across the Web
http://linkeddata.org/

LinkingOpenData – W3C SWEO Community Project
http://www.w3.org/wiki/SweoIG/TaskForces/CommunityProjects/LinkingOpenData

List of Tor Hidden Services from the Wikipedia
https://en.wikipedia.org/wiki/List_of_Tor_hidden_services

MagPortal
http://www.magportal.com/

Mappa.Mundi Magazine
http://mappa.mundi.net/

Mednar – Innovative Medical Search
http://mednar.com/

Mining the Deep Web for Economic Data
https://www.collectiveip.com/grants/NSF:0207603

New Zealand Digital Library
http://www.nzdl.org/

OAI-PMH Implementation Guidelines – Conveying rights expressions about metadata in the OAI-PMH framework
http://www.openarchives.org/OAI/2.0/guidelines-rights.htm

OAIster
http://www.oclc.org/oaister.en.html

OECD.StatExtracts – Complete Databases Available Via OECD’s iLibrary
http://stats.oecd.org/

Omnity Insight Engine – Everything Connected
https://www.omnity.io/

OneLook Dictionary Search
http://www.onelook.com/

Onion Browser – An Open-Source Privacy Enhancing Web Browser for iOS
https://mike.tig.as/onionbrowser/

Open Archives Initiative
http://www.openarchives.org/

Open Datasets
https://github.com/caesar0301/awesome-public-datasets
https://www.kaggle.com/datasets
https://www.data.gov/
https://www.quora.com/Where-can-I-find-large-datasets-open-to-the-public
https://aws.amazon.com/public-datasets/
https://data.world/
http://data.worldbank.org/

OpenIndex – Creating a Public Internet Index
http://www.openindex.org/

Open Source Intelligence
http://www.oss.net/

Open Vulnerability Assessment System (OpenVAS)
http://www.darknet.org.uk/2015/01/openvas-7-released-open-source-vulnerability-scanner/

Privacy Resources 2017
http://www.PrivacyResources.info/

Project Maelstrom – The Internet We Build Next
http://blog.bittorrent.com/category/labs/

QProber: Classifying and Searching “Hidden-Web” Text Databases – PERSIVAL Project
http://qprober.cs.columbia.edu/

Recommended Gateway Sites for the Deep Web
http://people.hws.edu/hunter/deepwebgate03.htm

ReportLinker: Industry Reports, Company Profiles and Market Statistics
http://www.reportlinker.com/

SAO/NASA Astrophysics Data System (ADS)
http://adswww.harvard.edu/

reSearcher
http://researcher.sfu.ca/

Science and Technology Sources on the Internet
http://www.loc.gov/rr/scitech/resources.html

Scientific and Technical Information Network (STINET)
http://www.loc.gov/flicc/Exemplars/DTIC/DTIC-STINET.PDF

Science Commons
http://creativecommons.org/science

Science.gov – FirstGov for Science – Government Science Portal
http://www.science.gov/

ScienceResearch.com – Deep Web Search Engine
http://www.scienceresearch.com/

SciTech Connect
http://www.osti.gov/scitech/

Scrapinghub Crawls the Deep Web
http://blog.scrapinghub.com/2015/02/24/memex/

Scrapy Webcrawler
http://scrapy.org/

SDARTS – A Protocol and Toolkit for Metasearching
http://sdarts.cs.columbia.edu/

SIMILE Widgets – Free, Open-Source Data Visualization Web Widgets and More
http://simile-widgets.org/

Social Buzz Bot (PDF download)
http://www.SocialBuzzBot.com/

STN International – Databases in Science and Technology
http://www.stn-international.de/

SurfEasy – Online Privacy
https://www.surfeasy.com/

Swoogle – Semantic Bot
http://swoogle.umbc.edu/

SWRC Ontology
http://ontoware.org/swrc/

Tails – The Amnesic Incognito Live System
https://tails.boum.org/

TechDeepWeb – How-To Guide to the Deep Web for IT Professionals
http://www.TechDeepWeb.com/

Terbium Labs – Matchlight Proactive Security In an Insecure World
https://terbiumlabs.com/

Testbed for Information Extraction from Deep Web
http://research.microsoft.com/users/nickcr/pubs/yamada_www2004poster.pdf

The Deep Web: Shutdowns, New Sites, New Tools by Vincenzo Ciancaglini
http://blog.trendmicro.com/trendlabs-security-intelligence/the-deep-web-shutdowns-new-sites-new-tools/

The Invisible Internet Project (I2P)
https://geti2p.net/en/

The World Bank – Data
http://data.worldbank.org/

THOR: Deep Web Data Extraction
http://www.cc.gatech.edu/projects/disl/THOR/

Tor Browser Bundle – Anonymity
https://www.torproject.org/projects/torbrowser.html.en

TOR For Newbies – When Should You Use It?
http://www.makeuseof.com/tag/tor-for-newbies/

Tor Project
https://www.torproject.org/

TorSearcher – Dark Web Search
http://www.torsearcher.com/

Tor Stack Exchange
http://tor.stackexchange.com/

TRID – The TRIS and ITRD Database (Transportation Research Board)
http://trid.trb.org/

TunnelBear – Simple, Private, Free Access to the Global Internet
https://www.tunnelbear.com/

Twitter/Search #deepweb
https://twitter.com/search?q=%23deepweb

UNdata – Data Access System To UN Databases
http://data.un.org/

UNESCO Information Services – Databases
http://www.unesco.org/unesdi/index.php/eng/doc/tous.html

Useful Tips and Tools to Research the Deep Web
http://www.online-college-blog.com/features/100-useful-tips-and-tools-to-research-the-deep-web/

Virtual Private Networks Directory of Best Services
http://www.makeuseof.com/tag/best-vpn-services/

Wall Street Executive Library
http://www.executivelibrary.com/

Web Data Extractors 2017
http://www.WebDataExtractors.com/

WebFountain™ – Analytical engine unstructured data
http://en.wikipedia.org/wiki/IBM_WebFountain

Web IR & IE
https://groups.yahoo.com/neo/groups/webir/info
http://www.webir.org/

WebScales: Towards a Highly Scalable Metasearch Engine
http://www.cs.binghamton.edu/~meng/pub.d/PIreport04.html

WTO Statistics Database
http://stat.wto.org/

Zaba Search – Free People Search and Public Information Search Engine
http://www.zabasearch.com/

RESOURCES – Semantic Web Research

Analyzing Social Networks on the Semantic Web
http://ebiquity.umbc.edu/paper/html/id/202/?EBS=d259cb1bacc16993d8f13615a1925762

DARPA Agent Markup Language
http://www.daml.org/

Deep Search, Wide Search and Everything Else You Should Know About Semantic Search
http://www.dataversity.net/deep-search-wide-search-everything-else-know-semantic-search/

Digital Object Identifier (DOI)
http://www.doi.org/

FOAF Project – A Semantic Web Application
http://www.foaf-project.org/

Foundation for Intelligent Physical Agents (FIPA)
http://www.fipa.org/

GoodRelations Vocabulary – Semantic Web Based eCommerce
http://www.heppnetz.de/projects/goodrelations/

Infomesh’s Semantic Web Introduction
http://infomesh.net/2001/swintro/

International Journal of Metadata, Semantics and Ontologies (IJMSO)
http://www.inderscience.com/jhome.php?jcode=ijmso

Jena – A Semantic Web Framework for Java
http://jena.sourceforge.net/

Journal of Biomedical Semantics
http://www.jbiomedsem.com/

Journal of Web Semantics
http://www.journals.elsevier.com/journal-of-web-semantics

Journal of Web Semantics: Preprint Server
http://www.websemanticsjournal.org/

Knowledge Discovery
http://www.KnowledgeDiscovery.info/

KnowledgeNets
http://wissensnetze.ag-nbi.de/

Language Engineering for the Semantic Web: A Digital Library for Endangered Languages
http://informationr.net/ir/9-3/paper176.html

Magpie – The Samatic Filter and Tool For the Semantic Web
http://projects.kmi.open.ac.uk/magpie/main.html

MetaData at W3C
http://www.w3.org/Metadata/

MindRaider – Semantic Web Outliner
http://mindraider.sourceforge.net/

OASIS – Advancing eBusiness Standards
https://www.oasis-open.org/

Ontology Metadata Vocabulary (OMV)
http://omv2.sourceforge.net/

O’Reilly’s Semantic Web Primer
http://www.xml.com/pub/a/2000/11/01/semanticweb/

pOWL – Semantic Web Development Plattform
http://sourceforge.net/projects/powl/

RDF – Resource Description Framework
http://www.w3.org/RDF/

Rules and Rule Markup Languages for the Semantic Web – RuleML-2003
http://www.informatik.uni-trier.de/~ley/db/conf/semweb/ruleml2003.html

SameAs.org – Interlinking the Web of Data
http://sameas.org/

SAO/NASA Astrophysics Data System (ADS)
http://adswww.harvard.edu/

SemanticDeskTop.org
http://www.SemanticDeskTop.org/

Semantic Knowledge Technologies and Language Computation
http://gate.ac.uk/projects/sekt/

SemanticWeb.org – The Semantic Web Community Portal
http://www.semanticweb.org/

Semantic Web Activity Statement
http://www.w3.org/2001/sw/Activity.html

Semantic Web Application Platform – SWAP
http://www.w3.org/2000/10/swap/

Semantic Web for AURIS-MM
http://derpi.tuwien.ac.at/~andrei/AURIS-MM-plan.html

Semantic Web In Breadth
http://logicerror.com/semanticWeb-long

Semantic Web Primer for Object-Oriented Software Developers
http://www.w3.org/TR/2006/NOTE-sw-oosd-primer-20060309/

Semantic Web Roadmap
http://www.w3.org/DesignIssues/Semantic.html

Semantic Web Search Engine
http://ws.nju.edu.cn/falcons/objectsearch/index.jsp

Semantic Web Search Engine (SWSE)
http://www.swse.org/

Semantic Web Services Challenge
http://www.sws-challenge.org/

Semantic Web – The Voice of Semantic Web Technology
http://www.semanticweb.com/

Semantic Web W3C
http://www.w3.org/2001/sw/

SenseBot – Semantic Search Engine That Finds Sense On the Web
http://www.sensebot.net/

Simile Widgets – Free, Open-Source Data Visualization Web Widgets and More
http://simile-widgets.org/

Sindice – The Semantic Web Index
http://sindice.com/

SourceForge.net: Project Info – OWL API
http://sourceforge.net/projects/owlapi

Swoogle – Semantic Bot
http://swoogle.umbc.edu/

SWRL: A Semantic Web Rule Language Combining OWL and RuleML
http://www.daml.org/2003/11/swrl/

Terbium Labs – Matchlight Proactive Security In an Insecure World
https://terbiumlabs.com/

The Authoritative Resource List for the Semantic Web by Kaila Strong
http://www.verticalmeasures.com/search-optimization/the-authoritative-resource-list-for-the-semantic-web/

The Cover Pages
http://xml.coverpages.org/

The RDF Query Language (RQL)
http://139.91.183.30:9090/RDF/RQL/

The Semantic Web: An Introduction
http://infomesh.net/2001/swintro/

The Semantic Web By Tim Berners-Lee, James Hendler and Ora Lassila
http://www.scientificamerican.com/article.cfm?id=the-semantic-web

The Semantic Web Is Your Friend
http://web.freepint.com/go/newsletter/160#feature

Transforming and Enriching Documents for the Semantic Web by Dietmar Roesner, Manuela Kunze, Sylke Kroetzsch
http://arxiv.org/abs/cs.AI/0501096

uClassify – Free Text Classified Web Service
http://uclassify.com/

Watson Web – Exploring the Semantic Web
http://watson.kmi.open.ac.uk/WatsonWUI/

Web Semantics: Science, Services and Agents on the World Wide Web
http://www.sciencedirect.com/science/journal/15708268

Web Service Modeling Ontology
http://www.wsmo.org/

Wilbur Toolkit for Semantic Web Programming [Project no longer actively maintained]
http://wilbur-rdf.sourceforge.net/

World Wide Web Reference 2017
http://www.WWWReference.info/

XML.com: Semantic Web
http://www.xml.com/pub/rg/Semantic_Web

XML.org
http://www.xml.org/

Yahoo Groups – SemanticWeb
http://groups.yahoo.com/group/semanticweb/

Bot and Intelligent Agent Research Resources and Sites

1st Spot
http://1st-spot.net/topic_agents.html

80legs – Powerful and Economical Service Platform for Crawling and Processing Web Content
http://www.80legs.com/

Agent Construction Tools
http://www.agentbuilder.com/

AgentLink
http://www.AgentLink.org/

Agents
http://aitopics.org/

AgentSheets – Authoring Tool to Create Agents
http://www.agentsheets.com/

ALICEBot
http://www.alicebot.org/

api.ai – Speech Interface for Apps and Devices
http://api.ai/

Applied Soft Computing
http://www.sciencedirect.com/science/journal/15684946

Article Search API – New York Times Articles 1981 to Present
http://developer.nytimes.com/docs/article_search_api

Artificial Intelligence Resources 2017
http://www.AIResources.info/

artoo.js – The Client-Side Scraping Companion
http://medialab.github.io/artoo/

BotAnalytics – Analytics for Bots
http://botanalytics.co/

Bot and Intelligent Agent Research Resources 2017
http://www.BotResearch.info/

Bots, Blogs and News Aggregators 2017
http://www.BotsBlogs.com

ChatBottle Search
https://chatbottle.co/

ChatterBots Resources and Sites 2017
http://www.ChatterBots.info/

Chatfuel – Build a Facebook Bot Without Coding
https://chatfuel.com/

Clara – Digital Employee That Schedules Meetings
https://claralabs.com/

Common Crawl – Open Repository of Web Crawl Data Composed Of Over 5 Billion Freely Available Web Pages
http://www.CommonCrawl.org/

cQuery – Content Query Engine
http://cquery.com/

CrawlTrack – Your Web Statistics Tool
http://www.crawltrack.net/

Create a Crawler – Extract Data From an Entire Website
http://support.import.io/knowledgebase/articles/247570-create-a-crawler

Data Mining Resources 2017
http://www.DataMiningResources.info/

Dataminr – Real-time Information Discovery
http://www.dataminr.com/

DataparkSearch Engine – Full-Featured Open Source Web-Based Search Engine
http://www.dataparksearch.org/

DataRobot – Build Better Predictive Models – Faster
http://www.datarobot.com/

Deep Web Research 2017
http://www.deepwebresearch.info/

Design of a Parallel and Distributed Web Search Engine by Salvatore Orlando, Raffaele Perego, and Fabrizio Silvestri
http://arxiv.org/abs/cs.IR/0407053

Dictionary of Algorithms and Data Structures
http://xlinux.nist.gov/dads/

Digital Footprints – Collect Facebook Data
http://digitalfootprints.dk/

Eliza – The Original ChatterBot
http://www-ai.ijs.si/eliza/eliza.html

Ethereum Frontier Release – A Decentralized Software Platform
https://www.ethereum.org/

Facepager – Fetching Public Data From Facebook
https://github.com/strohne/Facepager

FAME (Facilitating Agents in Multiculture Exchange)Project
http://cordis.europa.eu/projects/rcn/58337_en.html

File Information Tool Set (FITS)
http://fitstool.org/

Foundation for Intelligent Physical Agents
http://www.fipa.org/

Free Robots.txt Generator and Validator
http://www.toolsiseek.com/robots-txt-generator-validator

Friends of ChatBot Coalition
https://www.fcc.ai/

Google Guide
http://www.googleguide.com/

Gupshup – Bot Building Platform
https://www.gupshup.io/

Huginn – Your Agents Are Standing By
https://github.com/cantino/huginn

IBM Watson Services
http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/services-catalog.html

idavatars – Intelligent Digital Avitars
http://idavatars.com/

Imagination Engines
http://www.imagination-engines.com/

Import.io – Turn the Web Into Data With Extractors, Crawlers and Connectors
https://import.io/

Indexing Robot Crawler Checklist
http://www.searchtools.com/robots/robot-checklist.html

InfoExtractor – Extract Relevant Information from Various Sources Like Blogs, YouTube, and Wikipedia
http://www.infoextractor.org/

Institute for Human and Machine Cognition (IHMC)
http://www.ihmc.us/

Intellexer – Custom Built Search Engines, Knowledge Management Tools, Natural Language Processing
http://www.intellexer.com/

Intelligent Information Systems Research Laboratory
http://iis.ist.psu.edu/

International Journal of Agent-Oriented Software Engineering (IJAOSE)
http://www.inderscience.com/jhome.php?jcode=ijaose

jSEO – Web Crawler For Search Engine Optimization
http://codecanyon.net/item/jseo-web-crawler-for-search-engine-optimization/8770392

Knowledge Discovery
http://www.knowledgediscovery.info/

LAIR – Laboratory of Applied Informatics Research
http://lair.unc.edu/

List of User-Agents (Spiders, Robots, Crawler, Browser)
http://www.user-agents.org/index.shtml

Lurchr – I Keep Track of What’s Shared by Your Team So You Can Stay Focused On Work
https://lurchr.com/

MindMeld – Advanced Artificial Intelligence To Power the New Generation of Intelligent Conversational Interfaces
https://www.mindmeld.com/

Minimal-Intelligence Agents for Bargaining Behaviors in Market-Based Environments by Dave Cliff and Janet Bruten
http://www.hpl.hp.com/techreports/97/HPL-97-91.html

MIT Media Lab: Software Agents
http://agents.media.mit.edu/index.html

Modelling and Mining of Network Information Systems
http://www.mathstat.dal.ca/~mominis/index.html

Motion AI – ChatBots and Artificial Intelligence Made Easy
http://motion.ai/

Mozenda Web Agent Builder – Web Data Extraction
http://www.mozenda.com/

MultiAgent
http://www.MultiAgent.com/

MySpiders
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.21.3013

NCapture – Capture Web Content
http://www.qsrinternational.com/products_nvivo_add-ons.aspx

Networks and agents Network (NaN)
http://cnets.indiana.edu/groups/nan/

NewsBot – Related News At a Click Of a Button
https://getnewsbot.com/

Nomibot – Bots Scour the Web To Bring You What You Want
http://nomibot.com/

Open Source Web Information Retrieval (OSWIR05)
http://www.emse.fr/OSWIR05/

Oxyus Open Source Search Engine
http://sourceforge.net/projects/oxyus/

ResearchKit Framework – Medical Research Apps
https://github.com/ResearchKit

Robo Brain – Large Scale Computational System That Learns from Publicly Available Internet Resources
http://robobrain.me/

Scrapple – A Framework For Creating Web Scrapers and Web Crawlers
https://pypi.python.org/pypi/scrapple

Search Engine Robots
http://www.jafsoft.com/searchengines/webbots.html

Search Engine Watch News
http://www.searchenginewatch.com/

Search Tools – Information Guides and News
http://www.searchtools.com/

SeerSuite – CiteSeerX Toolkit
http://sourceforge.net/projects/citeseerx/

Semantic Web
http://www.semanticweb.org/

ShoppingBots 2017
http://www.ShoppingBots.info/

Siri – Your Virtual Personal Assistant
http://www.apple.com/ios/siri/

Smarter Bots 2017
http://www.SmarterBots.com/

SocialBuzzBot – The Business and Social Intelligence Search Engine for Information
Discovery from Social Communities
http://www.SocialBuzzBot.com/

SocSciBot – Social Sciences Link Analysis Research
http://socscibot.wlv.ac.uk/

Spidering Hacks
http://www.oreilly.com/catalog/spiderhks/

Spinn3r: RSS Content, News Feeds, News Content, News Crawler and Web Crawler APIs
http://spinn3r.com/

STACKS – Social Media Tracker, Analyzer, & Collector Toolkit at Syracuse
https://github.com/bitslabsyr/stack

Structure and Interpretation of Computer Programs – Video Lectures by Hal Abelson and Gerald Jay Sussman
http://groups.csail.mit.edu/mac/classes/6.001/abelson-sussman-lectures/

SupportBots – Chatbots for MSP and IT Service Providers
http://www.supportbots.io/

Swoogle – Semantic Bot
http://swoogle.umbc.edu/

TextRunner Search – Searches Hundreds of Millions of Assertions Extracted from 500 Million High-Quality Web Pages
http://openie.cs.washington.edu/

The Intelligent Software Agents Lab
http://www.cs.cmu.edu/~softagents/

The Lemur Toolkit – Language Modeling and Information Retrieval Research
http://www.lemurproject.org/

The Search Engine Project (TSEP)
http://freecode.com/projects/tsep

The Simon Lavern Page
http://www.simonlaven.com/

TSEP – The Search Engine Project
http://www.tsep.info/

UMBC AgentWeb
http://agents.umbc.edu/

UMBC eBiquity
http://ebiquity.umbc.edu/

Universe – Measurement and Training for Artificial Intelligence
https://universe.openai.com/

Web Curator Tool (WCT)
http://webcurator.sourceforge.net/