Open Source Linux Search Engines

Web data extraction (web data mining, web scraping) tool. It leverages well proved XML and text processing techologies in order to easely extract useful data from arbitrary web pages.

SphinxSearch is an extension for MediaWiki to replace the built-in search engine with a Sphinx search engine backend.

Aperture is a Java framework for extracting and querying full-text content and metadata from various information systems (file systems, web sites, mail boxes, ...) and the file formats (documents, images, ...) occurring in these systems.

Aracnis is a Java based framework for building distributed web spiders. These spiders can be used to accomplish a variety of tasks, for example, screen-scraping and link integrity checking.

Arts is a collection of perl scripts that allow you to input text into a quick and dirty knowledge base. It's designed to save information that is generated by email away for safekeeping in an organized web index.

C-squares is an easily implemented method for storage, querying and display of spatial data locations, based on a hierarchical, grid-based representation of the Earth' surface. Source code for encoding, decoding, mapping, etc. is provided via this site. Additional support is available by contacting the system developer, Tony.Rees@marinespecies.org; see also the c-squares home page at http://www.cmar.csiro.au/csquares/ .

DoxMentor4J is a standalone cross platform Web/Ajax based documentation library that is fully searchable and may be hosted in the file system, in an archive or embedded in the Java classpath.

'Extrakt' provides you with an plugin-based, easy to use PHP framework for fetching dynamic content from other pages on the internet, passing parameters either by GET or by POST. A demo application is available at: http://www.linuxuser.at/vuln-search

Frosttie (FROnt-end SchemaTron Text Internet Engine) takes XHTML pages and processes them with various user-definable filters such a W3C's WAI, Section 508 (US) web usability compliance, ad removal, etc. It can be used with zKnowMan.

projet Web Open Source qui permet de référencer les logiciels en langue française (style linuxapps et freshmeat). Il est conçu en PHP3 et MySql et utilise les templates.

Monitors webpages for changes and emails output with differences to subscribers. Permits user accounts and registration. PHP/MYSQL.

MAOS (Meta-Attribute Object Store) is a light-weight Java library / framework implementing simple Object persistence using search-engine technology

The Informa library provides a convenient Java API for handling news channels and metadata about them. Different syntax formats (RSS 0.91, 1.0, 2.0 and Atom 0.3, 1.0) for feeds are supported. Also support for channel information descriptions (OPML) avail

Open Digital Assets Repository System. This is php application for storing large amount of assets (> 100 000). Each asset can consist of several files and can have different versions. This kind of application is also called: AMS, CMS.

That project aims at providing a clean API, and the corresponding C++ implementation, for parsing travel-focused requests (e.g., "washington dc beijing monday r/t +aa -ua 1 week 2 adults 1 dog").

OpenEphyra is an open framework for question answering (QA). It retrieves answers to natural language questions from the Web and other sources. Visit http://www.ephyra.info/ for more details and information on joining this open research initiative.

OpenMKS is a search & navigational tool for large multimedia collections. With pluggable functionality and a core subsystem supporting the z39.50 ZING Community SRW search & retrieval specification, it can be run either as a Servlet or as a Web Service.

PHP World Portal is being developed as the framework for JLS Web Development's site. After each module is completed it will be released as open source for the public. The core framework will be released by 1/23/04.

PHPCrawl is a high configurable webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

Project B is a platform for various Bible programs using Java. It will support desktop applications like the On-Line Bible and Sword, there is a Servlet interface, some add-in macros for MS Word. Other interfaces are in development.

A threaded Web graph (Power law random graph) generator written in Python. It can generate a synthetic Web graph of about one million nodes in a few minutes on a desktop machine. It implements a threaded variant of the RMAT algorithm.

A fast way to rate the reading challenging level of book or text. Unlike well known reading metrics such as Fog, Kincaid, SMOG, ARI, Flesch, and Coleman-Liau readability this metric takes into account far more factors and is standarized against a corpus

Dr. Micheal Kay: "Saxon 8.7 is the first release to be released simultaneously by Saxonica on the Java and .NET platforms." MDP: Mission accomplished! Saxon for the .NET platform from Saxonica is now available and supported via the http://saxon.sf.net

The library does quick search of dictionary words in arbitrary input strings. Known problems are known. :) See appropriate section in documentation. Also, only ASCII words and strings for now.

ViewVC is a web-based version control repository browser, currently supporting CVS and Subversion repositories. Project development is now hosted at https://github.com/viewvc/viewvc/.