Introduction to XML

Recall that HTML was originally specified as an SGML (Standard Generalized Markup Language) doctype.
HTML only provides limited semantic information about a document. You can tell if something is in <h1> tags that is probably important, but not much else.
Starting in 1998 a stripped-down version of SGML called XML (extensible markup language) was developed to make it easier to create new tag-based mark-up languages where the tags can be used to carry whatever semantic information is desired.

The Syntax of XML

A new XML language can be specified in one of several ways (give a DTD - Document Type Definition, using XML Schemas, using Relax-NG, etc.).
We will look at the simplest way, which is with a DTD.
Other ways, allow for more precise control than DTDs would what can be in tags and attributes.
To create a new language with a DTD one specifies:
1. What tag elements exist in the language
2. What sub-elements or data a given element is allowed to contain
3. What attributes an element has and what their values can be.
4. What abbreviation entities are allowed in the language.
Tags are case-sensitive in XML, and every tag must have a close tag, although the abbreviation <element /> works as an implicit close tag.

XML Document Structure and Entities

Let's look at XML document:

<patient type="out of state">
     <![CDATA[This is data that will not be parsed by the XML parser 
      even if it has tags in it like this: <tag>]]>
<name><first>John</first><last>Smith</last></name>
<insurerID>&kaiser;</insurerID>
</patient>

Two files are usually associated with such a document:
1. one specifies what tags will be used with this document
2. another specifies how those tags should be styled.
One can abbreviate parts of a document using entities. For example, &kaiser; above might abbreviate some string of numbers.
Associated with any XML document is a document entity. This can be thought of as the root of the tree associated with the tags of the document. This entity thus can represent the whole document or the parts of it which do not change.
Entities can also be used to reference binary data such as images.

Document Type Definitions

A DTD is used to specify:
1. what tags will be used in a document
2. what kind of data or subtags these tags can hold
3. what attributes the tags can have and what values for these attributes are legal
4. what entities can occur in the document
DTDs can be internal (occur in the XML document itself) or external (occur in some separate file).
A DTD consists of a sequence of declarations enclosed in the block of a DOCTYPE markup declaration. Items 1-4 above correspond to the declaration of type <!ELEMENT ...> (1 and 2), <!ATTLIST ...>, and <!ENTITY ...>
There is also a tag called <!NOTATION ...>which is sometimes used.

<!ELEMENT ..>

Basic syntax of the tag is:

<!ELEMENT element_name 
   (list of names of child elements) SYSTEM Location NDATA NotationName >

Some examples:

<!ELEMENT memo (from, to, date) > 
---- SYSTEM and NDATA don't have to used

<!ELEMENT dept_script SYSTEM "dept.php" NDATA "php" >
<!NOTATION php SYSTEM "/usr/bin/php" >

<!ELEMENT person(parent+, age, spouse?, sibling*) > 
---- + is one or more, ? is optional, and * is 0 or more

<!ELEMENT element_name (#PCDATA) > 
---- pcdata = parsable character data can also use EMPTY 
for no sub-tags or character data or ANY if 
you want to allow everything

<!ATTLIST ..>

The general way to specify a list of attributes is:

<!ATTLIST element_name 
attribute_name_1 attribute_type [default_value] ... attribute_name_n attribute_type [default_value]  >

For example,

<!ATTLIST airplane places CDATA "4">
<!ATTLIST airplane engine_type CDATA #REQUIRED> ---- must have the field
<!ATTLIST airplane price CDATA #IMPLIED> ---- no default value is given
<!ATTLIST airplane manufacturer CDATA #FIXED "cessna"> 
   --- all instances must have the same value

We can define multiple attributes in the same ATTLIST:

<!ATTLIST img src CDATA #REQUIRED
                 width  CDATA #REQUIRED
                 height CDATA #REQUIRED
                 alt    CDATA #IMPLIED
>

We can also list possible values rather than leave it as general as just CDATA:
```
<!ATTLIST horizontal_rule alignment (left|center|right) #IMPLIED >
```

<!ENTITY ..>

Entities come in two flavors:
- General entities that can be referenced anywhere in an XML document
- Parameter entities which can be referenced only in markup declarations
The general form of an entity declaration is:
```
<!ENTITY [%] entity_name "entity value" >
```
% is used when it is a parameter entity

For example,

<!ENTITY cp "Chris Pollett" >
<!ENTITY cool_pic SYSTEM "/usr/local/cool_pic.jpg">

Example DTD

<?xml version = "1.0" encoding ="utf-8" ?>
<!-- planes_for_sale.dtd  -->
<!ELEMENT planes_for_sale (ad+) >
<!ELEMENT ad (year, make, model, color, price?, seller) >
<!ELEMENT year (#PCDATA)>
<!ELEMENT make (#PCDATA)>
<!ELEMENT model (#PCDATA)>
<!ELEMENT color (#PCDATA)>
<!ELEMENT price (#PCDATA)>
<!ELEMENT seller (#PCDATA)>
<!ATTLIST seller license CDATA #REQUIRED >
<!ENTITY c "cessna" >

Internal versus External DTDs

There are two ways to associate a DTD with an XML document:

Internally --

<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE planes_for_sale.dtd [ <!--DTD for planes_for_sale-->]>
<!-- The planes_for_sale document -->

Externally --

<!DOCTYPE planes_for_sale SYSTEM "planes_for_sale.dtd" >

Here is an example complete document that would validate against our planes_for_sale.dtd of a couple slides back

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE planes_for_sale SYSTEM "planes_for_sale.dtd" >
<planes_for_sale>
    <ad>
        <year>1960</year>
        <make>&c;</make>
        <model>150</model>
        <color>white</color>
        <seller license="12345678">Duran Duran</seller>
    </ad>
    <ad>
        <year>1945</year>
        <make>Douglas Aircraft Company</make>
        <model>DC-3</model>
        <color>red</color>
        <seller license="ABCDEFG">Gyro Gearloose</seller>
    </ad>
</planes_for_sale>

Namespaces

It is sometimes useful to mix and match markup from several XML languages into one file.
For instance, you might want to have mark-up for HTML as well as mark-up for SVG, a vector graphics language, and MathML, a language for mathematics.

One could do this with a declaration like:

<div xmlns="http://www.w3.org/1999/xhtml"
     xmlns:svg="http://www.w3.org/2000/svg"
	xmlns:mml="http://www.w3.org/1998/Math/MathML" >

Now to use an html tag within this div I don't need a prefix.
To use the svg tag rect, I might write <svg:rect ...>
This solves the problem where one might have two languages both with the same name for a tag. For intances, both with a <table> tag.

XML and CSS

Most modern browsers are completely happy to style any tag provided there is some style-sheet information given for it:
```
plane {display:block; border 3px;}
```
To associate a stylesheet with an XML document we use the syntax:
```
<?xml-stylesheet type="text/css" href="mystyles.css" ?>
```

Quiz

Which of the following statements is true?

setTimeout is used to set up a repeating timer in Javascript.
The regex pattern /\s/ matches any single non-whitespace character.
private visibility can be faked in Javascript using the module pattern.

Beginning AJAX

AJAX stands for Asynchronous Javascript and XML.
Asynchronous means that one does not need to wait for the response to an HTTP request to do something one can use Javascript to have several outstanding HTTP requests and still make things happen in the browser.
Further one doesn't have to reload the entire page when the results of a request are returned.
The XML in AJAX is because the results of a request are usually XML, JSON, or YAML data which is then formatted by some Javascript code.
The basic key to AJAX is the ability for Javascript to make an HTTP request. This is done with the XMLHttpRequest object.
As an example, consider the following test news items:
http://www.cs.sjsu.edu/faculty/pollett/174.2.22s/test-news.html
We can use AJAX to get these news items from our server and and embed them in the current page with the div tag with id "test-news" below:
Over the next couple slides we'll discuss how the code for this works.

Step-by-Step I

To create an XMLHttpRequest one could simply write in Javascript:
```
request = new XMLHttpRequest();
```
If we need to set up HTTP request headers we can use:
```
request.setRequestHeader("name", "value");
```

Step-by-Step II

To open a connection back to the server the Javascript came from one can then do:

request.open(theHTTPmethod, theURL, theAsync flag)
or
request.open(theHTTPmethod, theURL, theAsync flag, username, password)

For example,

request.open("GET", "test-news.html", true);
request.onreadystatechange = function()
{
    switch(request.readyState)
    {
        case 0:// handle uninitialized case
        case 1: // handle open but no send case
        case 2: // handle send but no response case
        case 3: // handle response is being downloaded case
        case 4: // handle response has completed being downloaded case 
    }
}

The asynchronous flag says whether or not the Javascript can continue executing (true) or if it must wait for a response (false).
At this point no data has been sent yet.

Step-by-Step Continued III

In case 3 or 4 above, request.responseText or request.responseXML will contain the data that has been returned by the server
In case 3 or 4 above, request.getAllResponseHeaders() can be used to get the HTTP response headers.
Sometimes it is useful to start a timer in case 3, using setTimout. If the request is taking too long so that the timer gets called, one can then abort the request using: request.abort.

Typically, the code in 3 or 4 above would put the response text into some tag in our document. This could be done with:

document.getElementById("myDivTag").innerHTML =  request.responseText; 
//or we could do...
myDiv = document.getElementById("myDivTag")
if(myDiv.firstChild)
{
	myDiv.removeChild(myDiv.firstChild);
}
myDiv.appendChild(document.createTextNode(request.responseText));

For our starting example, the code looked like:

request = new XMLHttpRequest();
request.onreadystatechange = function()
{
    switch(request.readyState) {
        case 4:
           document.getElementById("test-news").innerHTML =  request.responseText; 
        break;
    }
}
request.open("GET", "test_news.html", true);
request.send();

Step-by-Step Continued IV

Once we have set up our callback function we are ready to send data to the server.
To do this we can do:
```
request.send(null);
```
Note: send's argument can be used if using POST method to send the posted data
That's it! We just sit back then and wait for our callback to be called.

Parsing What Comes Back

After doing a request.open(), one can set the value of request.responseType.
By default this is null, if one sets it to 'document', then when the response comes, request.responseXML will hold an HTMLDocument object (provided the data was HTML). You can access from javascript objects within this document the same way one can access a web page. I.e., use getElementsByTagName, getElementById, nodeValue, firstChild, etc.
For XML data, one wants the content type sent by the server to be text/xml, then request.responseXML will then again have a document object that can be manipulated with standard methods.
We haven't talked about JSON, but for JSON, we don't need to set the responseType after opening the request. Once the request completes, we can instead just call:
```
my_object = JSON.parse(request.responseText);
```

Fetch API

Since about 2017, most browsers support a simpler API, called the Fetch API, for doing the kinds of requests we just showed:
```
fetch(some_url).then(
    (response) => {
        return response.json();
    }
);
```
The then clause is used to set up a call back that is executed after the attempt at retrieving some_url.
This is an example of a promise pattern which we will talk about in more detail in an upcoming lecture.

Web Services

One important use of AJAX and PHP is to allow you to write web services.
A web service is a programming interface which can be invoked over HTTP.
The first attempts to standardize such services made use of things like WSDL, SOAP, UDDI, etc. XML languages which tended to violate the KISS (keep it simple stupid) principle.
Some simpler web service interfaces have been written by major companies using XML-RPC, JSON-RPC and REST.
A XML-RPC document is a XML document which specifies a remote procedure call, i.e., which function of which object to invoke on some server; or it specifies the response of such a call. It later evolved into the more complicated and still evolving SOAP (Simple Object Access protocol)
JSON-RPC is like XML-RPC but uses JavaScript Object Notation -- basically, a snippet of Javascript code for an object.

REST

REST stands for Representational State Transfer. It is a technique for writing web services developed by Roy Fielding in 2000.
The idea is that an application state/method is viewed a resource. Each resource has a URL. There is a well defined way to tack on to this URL a query to invoke the function and return results. For example, the Yioop News Service might be invoked with a line like:
```
http://www.yioop.com/s/news?f=rss&limit=20&num=30
```
The resource in this case is "http://www.yioop.com/s/news" and the arguments are (f, limit, num) which control the returned value from the resource.

In general, a web service for an object might be given by

url/my_obj/method1?arg_1_for_method1=value_1&
    arg_2_for_method1=value_2...
url/my_obj/method2?arg_1_for_method2=value_1&
    arg_2_for_method2=value_2...
...

JSON

Stands for Javascript Object Notation.
It is commonly used for sending data when REST is used by Javascript can immediately use the data.

Primitive types in JSON are written as you expect:

12.3  -- an example Number
"hi there" -- an example String
true -- an example Boolean, other possibility false

Arrays are written in square brackets and comma separated:
```
[1, 4, 9]
```
Objects are written in braces and the name value pair are separated by a colon:
```
{"bob": 29,  "sally" : 35}
```
JSON data can be assigned to an object with the syntax:
```
myObj = eval( "(" + data + ")" );
```

XML versus JSON versus POSH

JSON is very popular because JSON data can be quickly converted into objects in a variety of languages: Javascript (obviously), PHP, Python, Java, Rust, etc.
Two arguments for the continued use of XML are:
1. Better integrity checking that the data meets some kind of format. I.e., We can check for example whether Patient data conforms to a Patient DTD using a general XMLParser. For JSON Patient data, such integrity checks tend to eb more ad hoc.
2. XML has better tools for extracting portions of a document such as XPath and XQuery.
If you are less concerned about (1) and more interested in (2), then you might just want that your tags are open and closed correctly, and your tag attributes are well-formed. In which case, you might try embedding semantic information directly in HTML or HTML fragments and use the ubiquitous parsers available for HTML. This is often called the Plain Old Semantic HTML (POSH) approach.
This approach is further supported by the fact that HTML 5 views any tag name beginning with x- as a legal HTML tag name and any attribnute beginning with data- as a legal HTML attribute. So the following would validate as a legal HTML 5 code fragment:
```
<x-patient data-insurer="kaiser" >
<x-name >Bob Newhart</x-name >
</x-patient>
```

GraphQL Web Services

Often web serives just request data, rather than invoke methods which change something on the server.
REST API often tend to be awkward at expressing complicated queries to retrieve data such as what one might expect from a language like SQL.
GraphQL is a web service query language original developed at Facebook in 2012, and since moved to its own non-profit foundation.
Web Service requests consists of POST'd JSON-like objects with "blanks" (field value comma separators missing) representing the data one wants:
```
{
  hero {
    name
    height
    mass
  }
}
```

GraphQL responses consist of JSON objects with the missing field values filled in:

{
  "hero": {
      "name": "Luke Skywalker",
      "height": 1.72,
      "mass": 77
  }
}

To define allowed queries one can specify JSON types, which partly goes towards making something like a DTD for JSON. For example,

type Hero {
  name: String
  height: Number
  mass: Number
}
//as another example
type Project {
  name: String
  tagline: String
  contributors: [User] //array of User objects
}

XML, AJAX, Web Services

Outline

Introduction to XML

The Syntax of XML

XML Document Structure and Entities

Document Type Definitions

<!ELEMENT ..>

<!ATTLIST ..>

<!ENTITY ..>

Example DTD

Internal versus External DTDs

Namespaces

XML and CSS

Quiz

Beginning AJAX

Step-by-Step I

Step-by-Step II

Step-by-Step Continued III

Step-by-Step Continued IV

Parsing What Comes Back

Fetch API

Web Services

REST

JSON

XML versus JSON versus POSH

GraphQL Web Services