Outline

• Information Retrieval and Applications
• Basic Architecture
• Probability Ranking Principle

Information Retrieval and Applications

• IR concerned with representing, searching, and manipulating large collections of text and data.
• Common applications include:
  • Web search
  • Desktop, email, and filesystem search
  • Document Management Systems for businesses.
  • Digital Libraries.
  • Document Filtering such as News Aggregation.
  • Text Clustering and Categorization
  • Summarization
  • Question Answering
  • Multimedia Retrieval

Basic Architecture

• A user has an information need called a topic and issues a query to the IR system to try to satisfy this need.
• A query typically consists of a small number of search terms (may or may not be words -- could be dates wildcards, etc).
• The user's query is processed by a search engine. This may be on the user's machines, or may be on one or more machines accessed over an internet.
• A major task of the search engine is to maintain and provide access to an inverted index for a document collection, the "database" of the search engine. This index provides a mapping between terms and documents that contain those terms. The size of the inverted index is typically about the same size as that of all the documents which are stored. So one has to be very clever in order to get information out of it quickly.
• The search engine processes the query and computes a ranked list of results according to a score calculated for each found document. It then attempts to remove duplicates from the results so far and finally returns what's left to the user.

Documents and Update

• For this purposes of this class when we say the word document, we mean it as a generic term to refer to any self-contained unit that can be returned to the user as a search result.
• For example, it might be an e-mail message, a web page, a news article, or even a video.
• When sub-parts of a larger object may be returned as an individual search results, we refer to the sub-part as an element. For example a paragraph or page from a book
• When arbitrary text passages, video segments, or similar material may be returned from a larger object it is called a snippet.
• In our picture from the last slide, you notice that documents can be added and deleted from the search engine's document collection. How this is done is called the update model of the IR system.

Performance Evaluation

• Two common ways to evaluate an IR system's performance are: efficiency and effectiveness.
• Efficiency is measured in terms of time (seconds per query) and space (bytes per document).
• Response time (time between query and results) and throughput (queries per second) are two statistics which are often cited for efficiency.
• As example, while at Mozilla, I heard a talk from the Wikipedia team and they said in 2008 they were handling 30,000 - 40,000 hits per second -- not all of these though would be queries to the search field. As Wikipedia is a top ten site, I would guess that Google handles within a order of magnitude of this amount.
• Other measures of efficiency include such things as the power consumed by the machines needed to generate the search results and their overall carbon footprint.
• Effectiveness is harder to measure than efficiency since it depends on human judgment.
• One commonly used notion of effectiveness is that of relevance. A document is considered relevant to a given query if its content satisfy the information need represented by the query.
• To determine this an assessor reviews document/topic pairs and assigns a relevance value. These values might be either binary or graded.

Probability Ranking Principle

The fundamental goal of relevance rank is often expressed in terms of the Probability Ranking Principle:

If an IR system's response to each query is a ranking of the documents 
in the collection in order of decreasing probability of relevance, then 
the overall effectiveness of the system to its users will be maximized.

• This serves as a guiding principle that underlies much research, but nevertheless overlooks such things as the size and scope of the document.
• The specificity of the document measures the degree to which its contents are focused on the information need. For example, the document might satisfy the need but also contain lots of other garbage.
• The exhaustivity of the document reflects the degree to which it covers the information related to the topic.
• The novelty of a document in a document list reflects how much new information the n-th search result give to the user given the previous results.