Searching XML Data

There are a few different techniques used for search XML data stored in a traditional RDBMS:
1. Full-Text Search - We use an existing full-text index on the columns with the xml data. I.e., we don't consider the structure of the document at all, but we might support looking up of documents containing tags like h1. or the like.
2. Keyword-Based Search - certains keyword or fields are extracted from the XML document and indexed. This might include meta information such as file name, author name, date, etc.
3. Structured Search - i.e., search that makes use of the XML document structure. Such search queries might be represented in a language like XQuery.

XQuery

XQuery is a query language that lets you query and transform XML data into a variety of other data formats.
XQuery 1.0 became a W3C recommendation in 2007, and the most recent version is XQuery 3.1 from 2017.
It uses XPath to refer to parts of documents, but then allows one to cycle over objects satisfied by such an expressions, filtering, ordering, and transforming them to a final output.

One common way to formulate an XQuery statement is to use a FLWOR (pronounced "flower") instruction:

FOR $variable IN expression 
LET $variable:=expression 
WHERE filtercriterion 
ORDER BY sortcriterion 
RETURN expression

A given FLWOR expression might not involve all of these clauses.
Let's take a look at some examples of how FLWOR statements work...

Setting up a FLWOR Experiment

To experiment with FLWOR, XQUERY statements, I downloaded the Base X XML database.
This is an light-weight, open source, BSD-licensed, XML database system written in Java.
It was initially created by Christian Grun in 2005.
Internally, it uses a tabular format to stores XML data, much like our node example, but it does all the work to handle XQuery for us.
According to DB-Engine Rankings, it is the most popular Native XML/non-multi-model, XML database.
After downloading it, one can click on the BaseX.jar file to launch it.
Clicking on the New Database icon, let's you choose a folder to make a new XML database in:

From the editor pane I then made a winecellar.xml file and a winereview.xml file:

//winecellar.xml
<?xml version="1.0" encoding="UTF-8"?>
<winecellar>
<wine winekey="1">
<name>Jacques Selosse Brut Initial</name>
<year>2012</year>
<type>Champagne</type>
<grape percentage="100">Chardonnay</grape>
<price currency="EURO">150</price>
<geo>
<country>France</country>
<region>Champagne</region> </geo>
<quantity>12</quantity>
</wine>
<wine  winekey="2">
<name>Meneghetti White</name> <year>2010</year>
<type>white wine</type>
<grape percentage="80">Chardonnay</grape>
<grape percentage="20">Pinot Blanc</grape>
<price currency="EURO">18</price>
<geo>
<country>Croatia</country>
<region>Istria</region> </geo>
<quantity>20</quantity> </wine>
</winecellar>

//winereview.xml
<?xml version="1.0" encoding="UTF-8"?>
<winereviews>
<winereview winekey="1">
<author>Chris Pollett</author>
<title>Great</title>
<description>This wine was great!</description>
<rating>5</rating>
</winereview>
<winereview winekey="2">
<author>Chris Pollett</author>
<title>Sucked</title>
<description>This wine was horrible!</description>
<rating>1</rating>
</winereview>
</winereviews>

FLWOR expressions

The previous slides steps succeeded in making my database, I could now do some FLWOR XQuery queries.
To make a query in the editor, I would create a new file a name related to thhe query and the extension .xq
Unlike our books examples, Base X expects query keywords to be in lower case.
As a first example query consider:
```
let $maxyear := 2012
return doc("winecellar.xml")/winecellar/wine[year<$maxyear]
```
This illustrates how the let clause allows one to make a local variable assignment, and how we use that assignment to return nodes with a return clause that involves an XPath.
For clauses in a FLWOR expression allow us to iterate over nodes returned by an XPath expression. The order by clause is used to control the order used by this iteration. For example,
```
for $wine in doc("winecellar.xml")/winecellar/wine
order by $wine/year ascending
return $wine
```
Where clauses allow us to filter nodes that result from for-let-order clauses according to some condition. So far, our return clauses have just returned the results of a node or XPath expressions, they can also be used to output xml document with new tags built out of the results of an XPath expression, thus, allowing us to transform the output. For example,
```
for $wine in doc("winecellar.xml")/winecellar/wine
where $wine/price < 20 and $wine/price/@currency="EURO"
return <cheap_wine> {$wine/name, $wine/price}</cheap_wine>
```

It is also possible to join multiple documents by using multiple for statement either by repeating the for keyword or by comma separting the documents we'd like to cycle over.

for $wine in doc("winecellar.xml")//wine
for $winereview in doc("winereview.xml")//winereview
where $winereview/@winekey=$wine/@winekey
return <wineinfo> {$wine/name, $winereview/rating} </wineinfo>

Final Remarks on XQuery

Postgres doesn't support XQuery in its implementation of SQL/XML.
DB2 and MSSQL do.

A common format for a query might be:

SELECT XMLQUERY(some_xquery)
FROM table_list
WHERE XMLEXISTS(some_other_xquery))

We can also use a similar syntax for UPDATE and INSERTs

Quiz

Which of the following is true?

The DOM API models XML documents internally as trees.
A Schema-Oblivious mapping of XML only works for XML data that was originally similar to a table.
The xmlagg function in SQL/XML is used to count the number of nodes an XPath expression returns.

Semantic Search, RDF and SPARQL

Once the web established itself, it was natural to try to get semi-autonomous, web-based agents that could access web resources and negotiate transactions on behalf of an end user.
For example, one might have a booking program that could first determine using the web, what hotels exist in a city. For each hotel, it could determine from web data its price range and availability on a set of dates, let a user decide on the hotel. Finally, it could complete the booking process.
Standard HTML web-pages are not suited to easily determine this kind of information, so XML languages that use tags to carry semantic information for this more robust kind of interaction were developed, the so-called semantic web.
The Resource Description Framework is an XML language developed by the W3C starting in 1997-1999 to provide a data model for objects on the web.
Its most recent version is RDF 1.1.
An RDF model consists of a sequence of statements in a subject-predicate-object format.
SPARQL (pronounced sparkle) is a query language used to query RDF models.
It first appeared in 2008 and the most recent version is SPARQL 1.1 from 2013.

RDF

Consider the subject-predicate-object triples:

(Bart, Name, Bart Baesons)
(Bart, Likes, Meneghetti White)
(Meneghetti White, Tastes, Citrusy)
(Meneghetti White, Pairs, Fish)

These triples roughly capture the idea that Bart is the name of the object Bart Baeson who likes Meneghetti White wine which tastes ciitrusy and pairs with fish.
In RDF, subject, predicate, objects are given URI's so that they refer to unique objects on the web. For example, the first triple written as URIs might be:
```
(www.kuleuven.be/Bart.Baesens, http://mywineontology.com/#term_name, 
"Bart Baesons")
```
Notice the Predicate column refers to a web document that has sections for a collection of possible predicates. This collection of possible predicates we might use is referred to as an ontology.
The Dublin Core Ontology and Friend of a Friend Ontology are examples of possible ontologies.

Serializing RDF

RDF can be serialized in a number of different ways, one of which is using XML. Below is an example XML document for the triples we hhad earlier:

<?xml version="1.0"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/TR/PR-rdf-syntax/" 
xmlns:myxmlns="http://mywineontology.com/">
<rdf:Description rdf:about="www.kuleuven.be/Bart.Baesens"> 
<myxmlns:name>Bart Baesens</myxmlns:name>
<myxmlns:likes rdf:resource="www.wine.com/MeneghettiWhite"/> 
</rdf:Description>
<rdf:Description rdf:about="www.wine.com/MeneghettiWhite"> 
<myxmlns:tastes>citrusy</myxmlns:tastes>
<myxmlns:pairs rdf:resource="http://wikipedia.org/fish"/> 
</rdf:Description>
</rdf:RDF>

Another way to write RDF is to use N3. In it, we just list out triples with a period at the end:
```
Bart name "Bart Baesons".
```
If we have two triples which repeat the same subject we can use a semi-colon and not repeat the subject:
```
Bart Name "Bart Baesons";
     Likes "Meneghetti White".
```
A comma is used if the subject and predicate are repeated:
```
Bart Father "Jane",
            "Sally".
```

Extensions to RDF

RDF provides a simple mechanism to create linked data by associating three web resources with each other.
RDF Schema (RDFS) is an extension of RDF that allows one to create classes of properties (the predicate portion of RDF) and then subclass them. For example, to say x is of a type B in RDF we might use a triple: x rdf:type B. To say B is a subtype of C in RDFS, we might use the triple: B rdfs:subClassOf C.
OWL (the Web Ontology Language) further extends RDFS with new capabilities for producing types from existing types. For example, the ability to create inverse types, to create super classes, to express transitive properties, to create restrictions, to create unions, and to create intersections.
Triples in RDF are often stored in a triple store such as MarkLogic or a Apache Jena. These typically come with an inferencing engine.

For example, given the triples:

:Surgeon rdfs:subClassOf :Staff
:Surgeon rdfs:subClassOf :Physician
:Kildare rdf:type :Surgeon

we can infer:

:Kildare rdf:type :Staff
:Kildare rdf:type :Physician

SPARQL

SQARQL is one language for extracting triples from a triple store.

It works by matching graph patterns against RDF graphs. For example,

PREFIX: mywineont: <http://mywineontology.com/> 
SELECT ?wine
WHERE {?wine mywineont:tastes "Citrusy"}

Here a PREFIX statement is used to make an abbreviation for a URI.
The names beginning with ? represent variables.
So in the above, the WHERE pattern represents a triple we would like to match against, filling in for the variable ?wine.
The SELECT clause says what to return if we find a match.

It is completely legal to put several triples within the {} using N3 notation:

PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT ?name 
       ?email
WHERE
  {
    ?person  a          foaf:Person .
    ?person  foaf:name  ?name .
    ?person  foaf:mbox  ?email .
  }

We can also see from the above it is legal to return and bind to more than one variable:

XQuery, RDF, SQARQL

Outline