Introduction

Last day, we were continuing our discussion of XML as a semi-structured data model.
We gave a second way to specify the structure of set of XML documents: using XML Schemas. (Such a spec is somewhat similar to a CREATE TABLE statement in the relational model.)
We then talked about stylesheet transformations via XSLT that allow us to map documents in one XML language to documents in a different XML language.
Finally, we introduced XPath as a means to express a collection of nodes in an XML document.
We now continue with a description of how to do this in XML...

XPath Expressions

Generally, an XPath expression starts from a context node then navigates down the document tree to the collection of nodes that are desired.
For example,
```
doc("winecellar.xml")/winecellar/wine
```
matches all wine nodes (and their contents) in the winecellar.xml file which start from the whole document, then immediately go through a winecellar sub-tag then have a wine tag.
We can use the expression // to skip multiple levels in the tree. So winecellar//wine would match any wine element that was a descendant of a winecellar element.
We can use wild-cards like /*/ to match a single tag name without specifying what it is.
If we are interested in the 2nd wine element children of a particular winecellar, we could use the syntax:
```
doc("winecellar.xml")/winecellar/wine[2]
```
We can place requirements on the values of attributes of a element by using [...] notation. For example:
```
doc("winecellar.xml")/winecellar/wine[price > 20]/name
```
returns the names of all wine nodes with price attribute greater than 20.
Finally, if you want the value of an attribute to be returned rather than the contents of a node, you can use @. For example, //img/@src to get all the url's from img tags in an HTML document.

XPath

The Yioop search engine used for the discussion board for this class can be configured to scrape news feed data into its data model from different XML or HTML search sources.
The configurations to do this use XPath.
Below is an example of what this configuration might look like for the Associated Press top news stories HTML Page.

URL:

https://apnews.com/apf-topnews

Channel:

//article[contains(@class, 'feed')]

Item Separator:

//div[contains(@class, 'FeedCard')]

Title:

//div[contains(@class, 'CardHeadline')]//h1

Description:

//div[contains(@class, 'content')]

Link:

//a[contains(@data-key, 'story-link')]

In-Class Exercise

View the source HTML for today's slides.
Come up with XPath expressions to retrieve the following:
- The titles of each slide.
- The URL's of each stylesheet linked to by today's slides.
- The contents of the third bullet point on each slide.
Post your solution to the Sep 2 In-Class Exercise Thread.

Processing XML Documents

As we just saw, information in an XML document can be used by an application for further processing.
The figure above shows the steps typically involved in this.
The first step maps the XML document to a common XML format used by the application. This might be done using XSLT.
The XML Parser then checks if the result is well-formed and supplies pertinent information, often using either a DOM or SAX API, from the document to the application.

DOM API

The DOM (Document Object Model) API is a tree-based API that represents the XML document as a tree in internal memory.
For example, the document:
```
<wine>
<name>Meghhetti White</name>
<year>2010e</year>
</wine>
```
would be represented by the figure above.
DOM provides various classes and method for navigating through the tree and to do various operations such as adding, moving, or removing elements from the tree.
As the whole document is held in memory at one time, it often uses a lot of space for larger documents.

SAX API

The SAX (simple API for XML) API is an event-based API.

Using SAX, the XML document from the previous slide would be represented by a stream of events:

start document
start element: wine
start element: name
text: Meneghetti
end element: name
start element: year
text: 2010
end element: year
end element: wine
end document

This stream can be directly passed on to the application which will use an event handler to process the event.
So access with SAX is more path-in-tree-like than whole tree-like.
Since no tree is built it tends to use less memory than DOM.
SAX tends to be good for simple sequential operations on a document or for random access to a part of a document, but it performs worse if you need to do have data manipulation.
Both DOM and SAX have library implementations for a variety of languages: Java, Python, C++, PHP, Javascript, etc.
For developers this means one can create one's own XML language relatively easily and then use these pre-built parsers to do data-manipulations for documents written in your new language.

Storage of XML Documents

One simple way to store XML documents is as a collection of files on a web server in much the same way that HTML pages are often stored and served.
This can be cumbersome in terms of maintenance, data manipulation, and structured search.
A better alternative is to use a document management system aimed at editing, storing, and maintaining complex hierarchically structured documents.
Such systems are often built on top of a traditional DBMS and we need to write how to do the mapping between the XML and the underlying database system.
A third alternative is to use a XML Server or XML DBMS specifically designed to store and manipulate XML Documents. (MarkLogic, Virtuoso, etc.)
Finally, we can store XML data in a usual RDBMS using either a document or data-oriented manner. We now look at this more detail.

RDBMS Data Storage Approaches for XML

Document-Oriented Approach - an XML document in either a BLOB (binary large object) or CLOB (character large object column). If we use the latter, XML documents might still be searchable if we create a full text index on the column (either use the native full indexes or an external such index such as Sphinx). As we will, see some DBMS's support an XML column type that makes it easier to use XPath expressions on the XML column. This approach tends to be easy to implement, but not to mesh very well with traditional SQL processing and optimization.
Data-Oriented Approach - an XML document is decomposed into its constituent data parts which are then spread across a set of connected tables in a process called shredding. The DBMS or separate middleware is responsible for mappings between XML and tables. An advantage of this approach is it allows for finer SQL manipulation/constraint of elements in the stored documents. Some XML features such as nesting can often be implemented using the CREATE TYPE commmand, etc of Object-Relational DBMSs. A drawback of this approach is that for complicated document structures, any manipulation, querying, etc might involve the computation os many non-trivial joins.
Combined Approach (aka partial shredding) an XML document has some of its subtrees shredded whereas others are stored as BLOBs or CLOBs. SQL Views are then typically used to reconstruct the entire document. For example, consider a car insurance claim XML document. The details of the claimant and the car might be stored in a traditional relation so they can be easily search using SQL. The description of the accident, photos, etc from the XML document might all be stored as BLOBs and CLOBs.

Differences Between XML and Relational Data

The base object of the relational model is the relation which consists of 0 or more tuples. The order of the storage of these tuples in the relation does not matter. The order of the attributes of a tuple is also flexible as we can project them (list them in a SELECT line in SQL) in whatever order we want.
In the XML model, order of nodes can matter. For example, it would be weird to take the paragraph in a XHTML document and output them backwards.
To implement ordering when mapping between the models we typically would need to add an extra attibute to the table that says the position of the column or row with respect to the order. For some orderings in a ORDBMS, we might be able to use a list collection type.
RDBMS databases by design support at least first normal form: so we don't have nested relations. For example, an emplotee cannot have a children column that acts as a subtable listing all of an employees children. XML data would allow us to have Employee document with a children node containing a list of children. ORDBMS support columns which are sets or lists of some table type.
XML data constrained by just a DTD (and hence PCDATA) does not support atomic types such as INT, BOOL, etc.
XML data can have anomalies in structures that is not all tags or tag attributes appear in every document; whereas, for a relation all columns must exist even if they are null. Adding new tags might be okay in XML, but adding new columns might require rebuilding an entire table in the relational model.

Mappings Between XML Documents and Relational Data - Table-based Mapping

If the original XML original XML document is like a table then a direct mapping between XML and relations is possible. For example, if the XML data was in the form:

<database>
<row><column> data </column> <column> data ... </column> <column> data </column></row>
<row><column> data </column> <column> data ... </column> <column> data </column></row>
...
</database>

or in the form:

<database>
<row column_1="data"  column_2="data" ... column_n="data"/>
<row column_1="data"  column_2="data" ... column_n="data"/>
...
</database>

This kind of mapping is often used when you want to transfer data from one RDBMS to a different one.

XML Storage and Mappings

Outline