Recommended Gateway Sites for the Deep Web

And Specialized and Limited-Area Search Engines

 Michael Hunter    7/1/2009
Web version:  http://people.hws.edu/hunter/deepwebgate09.htm

 

The Deep, or Invisible Web requires interaction to display --  dynamically-created pages, real-time information and databases.  Currently estimated to be over 100 times larger than the surface web,  the Deep Web houses billions of documents in databases and other sources, over 95% of which are available to the public.  As crawler-based search engines cannot access these documents reliably, specialized sources such as these currently provide our best access.

 

General Gateways   |    Humanities   |   Social Sciences  

Science and Technology   |   Health Sciences  
Business and Government  |   Reference, Popular Culture   |  Other


General Gateways:

 

Invisible Web Directory (highly recommended)  http://www.invisible-web.net/

            An excellent gateway to some of the best research-oriented invisible web resources available.  Arranged by category and sub-category. Moderate in size with sites well selected and organized.  Maintained by Chris Sherman and Gary Price.

 

Intute    http://www.intute.ac.uk
                A well-annotated listing of Deep Web resources. Compiled and maintained by over 70 educational and research organizations, largely from the UK. Hubs include Science and Technology, Arts and Humanities, Social Sciences and Health and Life Sciences.
* 

Infomine  (highly recommended)           http://infomine.ucr.edu

            U. of California at Irvine’s directory of excellent academic research sites now includes search capability for all databases found on the sites.

 

Librarians' Internet Index  (highly recommended)    http://lii.org

            Librarian-created collection with extensive site reviews.  Includes technology, home, people and recreation as well as scholarly sites.  Searchable by title, URL or description.


Oaister  (highly recommended)    http://www.oaister.org/

Catalog of over 22 million documents from scholarly and archival digital repositories and special collections worldwide.  Includes full-text journal articles, conference papers and reports as well.

WWW Virtual Library   (highly recommended)   http://vlib.org/

          Includes traditional disciplines, area studies, minority studies and other academic and research areas.  Excellent annotations with dates of coverage.

 

Dmoz.org   (recommended)                  http://dmoz.org/

                The librarian-created Open Directory offers this comprehensive listing of reviewed, quality websites and databases by subject area. 

                   

IncyWincy          http://www.incywincy.com/

                Covers over 100,000 embedded databases, identifying those relevant to the search and

searches them. 

 __________________________________________________

Humanities:

 

WWW Virtual Library   (highly recommended)   http://vlib.org/

          Includes traditional disciplines, area studies, minority studies and other academic and research areas.  Excellent annotations with dates of coverage. 

 

Infomine      http://infomine.ucr.edu/

            The University of California’s premier resource offers a listing of over 140 specialized and general search engines and services.  Choose “Internet Enabling Tools” at the homepage and type search engines and a subject in the search box.

 

Arts and Humanities   (highly recommended)     http://www.intute.ac.uk/artsandhumanities

            Covers arts, languages and literatures, philosophy, history, religion, archaeology, American studies and other related areas.  Maintained by Intute.

 

Voice of the Shuttle     (highly recommended)   http://vos.ucsb.edu/

            Top choice for Humanities resources.  Maintained by Alan Liu of UC Santa Barbara.  Includes Anthropology, Area Studies and Minority Studies as well as other Humanities disciplines.

 

Shakesearch     http://www.rhymezone.com/shakespeare/

            Full-text search of all works by Shakespeare with links to related sources

 

History Matters  (highly recommended)   http://historymatters.gmu.edu/

            Features  media-rich resources for teaching U.S. history at the secondary and college levels.  The associated site www.history is a collection of quality sites with extensive descriptions.

____________________________________________________
Social Sciences:

Rutgers University Research Guides:  Social Science and Law  (highly recommended) 

            http://www.libraries.rutgers.edu/rul/rr_gateway/research_guides/research_guides.shtml#social_sciences

            Excellent source, covering Environmental Studies, Women’s Studies, Latin American Studies, Library & Information Science and Alcohol Studies as well as the traditional social science disciplines.

 

WWW Virtual Library   (highly recommended)   http://vlib.org/

          Includes traditional disciplines, area studies, minority studies and other academic and research areas.  Excellent annotations with dates of coverage. 

 

Infomine      http://infomine.ucr.edu/

            The University of California’s premier resource offers a listing of over 140 specialized and general search engines and services.  Choose “Internet Enabling Tools” at the homepage and type search engines and a subject in the search box.


Statistical Resources on the Web        http://www.lib.umich.edu/govdocs/stats.html

            One of the best, national and international in scope, with links to U. of Michigan’s excellent Foreign Governments Document Collection.

 

Social Science Information Gateway (UK & Europe)       http://www.intute.ac.uk/socialsciences/

            Offers articles, reports, databases and organizations in major social science fields, business and women’s studies.  Most resources are UK and European-based.  The Grapevine feature accesses academic departments, latest conferences and resumes.  Part of Intute.

 

SocioSite      http://www.sociosite.net

          Sociology gateway maintained by the University of Amsterdam.  Extensive and well-structured subject access to a wide range of sociological topics.  Includes more specialized gateway sites, association home pages, articles and conference papers.  Predominantly English language sources.

 

Psychcrawler   (highly recommended)   http://www.psychcrawler.com/

            A product of the American Psychological Association created to provide quick access to quality Internet content in the field of psychology. 

_____________________________________________

Business and Government

 Rutgers University Research Guides:  Business  (highly recommended) 

            http://www.libraries.rutgers.edu/rul/rr_gateway/research_guides/busi/business.shtml

          Excellent source for Business News, Market Research, Company Profiles, Job Searching and Financing Your Business.  National and international in scope.

 

USA Gov      (highly recommended; formerly FirstGov)       http://usa.gov

            New government information gateway.  Comprehensive and well maintained.

 

Direct Search   (recommended)          http://www.freepint.com/gary/direct.htm

Compiled by Gary Price of George Washington University, this gateway concentrates on research resources.  Large site and somewhat difficult to use, but strong in business, economics, news and speech transcripts.  Not currently updated.

               

GPO Access  (recommended)             http://www.access.gpo.gov/

                Best interface for deep web content on US Government websites with full-text databases

   

_________________________________________________

Science and Technology:

 SciCentral      http://scicentral.com

           Gateway site to directories, databases, e-journals and metasites.  Organized by broad subject area (Biosciences, Health Sciences, Physics, Chemistry, Earth & space and Engineering.) and resource type.  Strong in breaking science news with links to current research highlights

 

Science.gov      http://www.science.gov

            Searchable database with descriptions of sites that are key entry points to U.S. government science and technology resources

 

Scientific Search Engines      http://www.scientific-search-engines.com/

            Actually a gateway to quality search services and sites in all academic disciplines in English and German.  Maintained by LLEK in Germany.

  

Scirus  (highly recommended)        http://www.scirus.com/

            Created by Elsevier Science, this gateway covers scientific, technical and medical sources with over 60 million pages, listing Web resources as well as the fee-based Elsevier Science Collection.  Excludes sites with no scientific content.  Includes pdf and  postscript files and indexes full text of all documents.  Fine Advanced Search features.  Classified into broad subject areas that can be searched individually.  Customization offered

 

Martindale’s Reference Desk  (highly recommended)   http://www.martindalecenter.com

                Maintained by James Martindale of UC-Irvine, a gateway to high-quality sources in a wide range of reference topics, especially strong in sciences and health sciences.

 

WorldWideScience  (highly recommended)  http://worldwidescience.org

Global science gateway to journal citations and articles, reports and other publications from over 50 national and international scientific databases.

WWW Virtual Library  (highly recommended)    http://vlib.org/Science.html

            Comprehensive metasites for biosciences, health, earth sciences, physics and chemistry.  Maintained by universities and research institutions.  (For agriculture, computer science and eingeneering see http://vlib.org/.

 

Biology Links      (highly recommended)     http://mcb.harvard.edu/BioLinks.html

                Includes data banks, model organisms, biological software directories and other related gateway sites.  Maintained by the Dept. of Microbiology at Harvard.

 

Internet Directory for Botany     (highly recommended)      http://www.botany.net/IDB

            Rich collection of databases, links and specialized data resources.  Maintained by an international team of scholars. 

 

Calculators On-line Center   (highly recommended)   http://www.martindalecenter.com/Calculators.html

                The most comprehensive collection of this type on the Internet. 

 

Infomine      http://infomine.ucr.edu/

            The University of California’s premier resource offers a listing of over 140 specialized and general search engines and services.  Choose “Internet Enabling Tools” at the homepage and type search engines and a subject (eg. Science) in the search box.

   

Statpages         http://StatPages.org

                Web pages that perform calculations, probabilities, plotting and other interactive programming environments. 

______________________________________________

Health Sciences:

 

 Healthfinder      (highly recommended)     http://www.healthfinder.gov/

            The National Institutes of Health maintain access to over 100 health-related limited-area engines and database search pages from government and other sites as well.  Consumer-oriented information covering prevention and self-care, infants, teens, senions, fraud and complaints and much more  Enter search engines in the search box at the URL given above.

 

Martindale’s Health Sciences Guide   (highly recommended)

  http://www.martindalecenter.com/HSGuide.html

                A true Virtual Medical Center covering general and specialized medical fields as well as allied areas such as Dental, Medical Law, Nutrition, Pharmacy and Veterinary.  Includes online courses, tutorials, databases and video files.

 

RxList                 http://rxlist.com/

            In-depth drug information covering over 4,000 U.S products.  Each entry contains description, clinical pharmacology, indications, usage, contraindications, warnings, precautions, drug interactions,adverse reactions, drug abuse and dependence, dosage and administration and animal pharmacology.

 

WWW Virtual Library  (highly recommended)    http://vlib.org/Medicine.html

          Strong in traditional medical areas, allied health fields and alternative medicine.  (For medical research see Biosciences section of the Virtual Library

_____________________________________________

Reference, Popular Culture, Lifestyle:

 

Martindale’s Reference Desk  (highly recommended)   http://www.martindalecenter.com

                Maintained by James Martindale of UC-Irvine, a gateway to high-quality sources in a wide range of reference topics.   Excellent collection of free online foreign language dictionaries.

  

WWW Virtual Library   (highly recommended)   http://vlib.org/

          Includes encyclopedias, dictionaries, biographical sources, geographical sources, statistics, almanacs and yearbooks.  Excellent annotations with dates of coverage.

 

 Beaucoup  (highly recommended)      http://www.beaucoup.com/

            Collection of over 2,500 searchable databases and search engines. Emphasis on lifestyle, media and people.

 

Foundation Center Online      http://fdncenter.org/

                Includes an excellent “Online Library” of resources on foundations and non-profits and grant-seeking as well as a directory of grantmakers from all types of business and private sources.

 

Internet Public Library  http://www.ipl.org/

            One of the best for public library needs, including exhibits, news, teens, youth collections and tutorials on the research process.  Now includes guides formerly known as Argus Clearinghouse.

  

_____________________________________________

Other: 

           

Dmoz.org   (recommended)               http://dmoz.org/

                The librarian-created Open Directory offers this comprehensive listing of blogs by subject area.  Path:  Computer/Internet/On the Web/Weblogs

               

 Virtual Gumshoe         http://virtualgumshoe.com/

            Gateway to locator databases, from Adoption Resources to Voters Records.  Associations and individuals covered extensively.

 

NewPages                        http://www.newpages.com/

            Portal to information on independent bookstores, independent publishers, literary periodicals, alternative newsweeklies and more. 

  

Disinformation      http://www.disinfo.com/

            Collects and indexes the most “hidden” and subversive material on the ‘Net.  Over 30 million documents.

____________________________________

Country or Region-Specific:

 

Beaucoup  (highly recommended)      http://www.beaucoup.com/

            Collection of over 2,500 searchable databases and search engines

 

Search Engine Colossus       (highly recommended)     http://www.searchenginecolossus.com/

            Over 920 search engines and subject directories, organized by country with brief descriptions of scope, language and origin of each.

 

Country Search Engines    http://www.philb.com/countryse.htm

            Includes subject metasites focused on individual countries as well as search engines and subject directories.  Up-to-date, but without annotations.

 

Main Portals       http://www.mainportals.com/#countries

            A listing of major, portalized search services in 84 countries

 

Nationsonline      http://www.nationsonline.org/oneworld/

            Smaller listing of search services in 45 countries, with each designated as a search engine or subject directory (index).

 

Official City Sites      http://officialcitysites.org/    (highly recommended)

            Covers cities in the U.S, Canada, UK, France, Germany, Netherlands, Australia, New Zealand and Japan.

 

Country-specific Information Sources

 Nationsonline      http://www.nationsonline.org/oneworld/

          A wealth of up-to-date information on individual countries, cultural, political, social and econcomic.

 

ELDIS        http://www.eldis.org

            Information sources on development and the environment, country-specific and international in scope