Introduction

Last week, we were talking about NoSQL (Not-only SQL) databases.
We said there are four kinds of NoSQL databases: Key-Value Stores, Tuple and Document Stores, Column-Oriented Databases, and Graph-based Databases.
We looked at Key-Value Stores and how nodes could be used to determine who should store a given key-value pair using consistent hashing and a gossip algorithm.
We talked about replication of nodes and redundancy.
We also talked about eventual consistency and BASE versus ACID properties of NoSQL databases.
Finally, we said a key-value store where the value is always a vector is called a tuple store, and a key-value store where the value is semi-structured data such as JSON, JSONB, or XML is a document store.
For document stores, we said the key is often reflected as one of the fields in the document associated with the key. (_id attribute in MongoDB)
Today, we look at document stores in more detail.

Filters and Queries

Just as with key value stores, the primary key can rapidly be retrieved from a document store.
Document stores often have many common attributes in their document collection and so also have ways to retrieve documents based on simple filters on these attributes.
To see this in action we now look at MongoDB, a popular document store.

MongoDB

MongoDB was created by 10gen software in 2007.
It was open-sourced in 2009.
Since 2013, 10gen is called MongoDB Inc and since 2017 it is listed on the NASDAQ.
You can download MongoDB Community Server from the MongoDB Website.
In my case, I got a .tgz file which I extracted to /usr/local/mongodb and chown'd to myself so I could be lazy.
The README explains how to get the Compass GUI for MongoDB if desired.
We are interested in the bin folder which has the two programs mongod (server) and mongo (client).
I used mkdir to make a folder /usr/local/mongodb/data/db .

Then I ran the server using:

mongod --dbpath /usr/local/mongodb/data/db/

Using the Mongo Client and Mongo Databases

To experiment with databases in mongo db, I ran the mongo client shell:
```
mongo
```
From the command prompt that results, you can see a list of commands using:
```
help
```
Mongo arranges data into databases which are in turn arranged into collections.
To see a current list of databases you can use the command:
```
show dbs
```
To connect to use a database you type:
```
use name_of_desired_databases
```
Even if the database isn't created yet, you can use the above command to switch to a given database.
As soon as a collection is created within the database the database will be created.
To get rib of the current database we can use the command:
```
db.dropDatabase();
```

Making Collections Mongo

To create a collection you insert an item into it:

db.employees.insertOne({name : "Chris Pollett", born: 1970, salary: "too low"});

The above adds one document to the employees collection (creating collection and database as needed). Notice documents are JSON or JSONB objects.

You can insert many items in one go using insertMany with an array of JSON objects:

db.employees.insertMany([
    {name : "Alice", born: 1990, salary: "just right"},
    {name : "bob", born: 2000, salary: "too high"}
]);

Alternatively, you can create a collection using:
```
db.createCollection(name, options);
```
The client allows you to make temporary variable assignments which you then use later:
```
my_options = {capped:true, size: 10000000, max:50};
db.createCollection("foo", my_options);
```
Here capped size there is a maximum size of the collection, size is its size in bytes, max is the maximum number of documents.
To get help on collection commands you can use:
```
db.name_of_collection.help()
```
To see all the current collections in a database, we can use the command:
```
show collections
```
To get rid of a collection we can use the command:
```
db.collection_name.drop() 
```

Filter Criteria and their use in Search, Deletes and Updates

The find(criteria) method is used to search for elements in a collection:

db.employees.find({}); //return all employees documents
db.employees.find({born:1990, salary : "just right"}); 
   // all employees born in 
   //1990 whose salary is "just right"
db.employees.find({born: {$gt : 1975}});  
    // all employees born after 1975
    // also have $lt, $lte, $gte; logical operators 
    //(use arrays) $and, $or, $not
db.employees.find({and: [{born: {$gt : 1975}},
       {born: {$lt : 2000}}
    ]}); 
db.employees.find({}, {name:1, _id:0}); 
    // here we have project criteria where we list
    // fields we want with value 1, and those we don't with value 0

Filter Criteria are also used for removing and updating items:

db.collection.remove(criteria)
db.collection.update(filter, update, options)

For example,
```
db.employees.remove({born: 1970}) 
```

Quiz

Which of the following is true?

Partitioning data over servers is a process known as sharding.
Cassandra is typically configured without the use of seed nodes for its gossip algorithm.
Consistent hashing requires nodes to be arranged in either a star topology or using a Peterson graph.

Using MongoDB from an Application

The Java Driver for MongoDB can be found on the Maven Repository.

Below is an example Java program that connects to a test MongoDB database, that illustrates some of the filtering operations we just saw but from an Java application:

import org.bson.Document;
import com.mongodb.MongoClient;
import com.mongodb.client.FindIterable;
import com.mongodb.client.MongoDatabase;
import java.util.ArrayList;
import static com.mongodb.client.model.Filters.*;
import static java.util.Arrays.asList;
public class MongoDBExample
{
    public static void main(String... args) {
        MongoClient mongoClient = new MongoClient();
        MongoDatabase db = mongoClient.getDatabase("test");
        // Delete all books first
        db.getCollection("books").deleteMany(new Document());
        // Add some books
        db.getCollection("books").insertMany(new ArrayList<Document>() {{
            add(getBookDocument("My First Book", "Wilfried", "Lemahieu",
                12, new String[]{"drama"}));
            add(getBookDocument("My Second Book", "Seppe", "vanden Broucke",
                437, new String[]{"fantasy", "thriller"}));
            add(getBookDocument("My Third Book", "Seppe", "vanden Broucke",
                200, new String[]{"educational"}));
            add(getBookDocument("Java Programming for Database Managers",
                "Bart", "Baesens", 100, new String[]{"educational"}));
        }});
        // Perform query
        FindIterable<Document> result = db.getCollection("books").find(
            and(
                eq("author.last_name", "vanden Broucke"),
                eq("genres", "thriller"),
                gt("nrPages", 100)));
        for (Document r: result) {
            System.out.println(r.toString());
            // Increase the number of pages:
            db.getCollection("books").updateOne(
                new Document("_id", r.get("_id")),
                new Document("$set",
                    new Document("nrPages", r.getInteger("nrPages") + 100)));
        }
        mongoClient.close();
    }
    public static Document getBookDocument(String title, String authorFirst,
        String authorLast, int nrPages, String[] genres) {
        return new Document("author",
            new Document()
                .append("first_name", authorFirst)
                .append("last_name", authorLast))
            .append("title", title)
            .append("nrPages", nrPages)
            .append("genres", asList(genres));
    }
}

Remarks on Example

The above program first connects to a test database, then deletes the books collection if it exists.
Next it adds four books to the books collection.
It uses the auxiliary method getBookDocument to build the appropriate bson (jsonb) object.
Then it searches the books collections for books whose author's last name was "vanden Broucke" which are thrillers of more than 100 pages.
For each such book (in this case only one), it uses an update command to add 100 pages to the length of the book.

Aggregation

Most NOSQL stores support queries involving, aggregation, ordering, and limit clauses.

For example, in MongoDb, we can add to our basic query additional method calls:

db.my_collection
  .find(some_criteria)
  .sort({field_name1:1, field_name2:-1, ...})
  .skip(50)
  .limit(5); // find returns a cursor, 
             // which we modify to sort according to listed fields (1 = ASCENDING, -1=DESCENDING
             // then return the 50th through 55 result

We can use the aggregation() function to do aggregations like compute a sum:

// imagine salary had been reals rather that the strings
// we used as a joke, we could calculate
// for each year the total salary for that year for  everyone 
// born after 1975 sorted by totals in descending fashion as:
db.employees.aggregate([
   { $match: { born: {$gt : 1975} } },
   { $group: { _id: "$born", total: { $sum: "$salary" } } },
    { $sort: { total: -1 } }
])

In Java, we might do an aggregate query such as:

// Perform aggregation query
AggregateIterable<Document> result = db.getCollection("books")
    .aggregate(asList(  
        new Document("$group",
            new Document("_id", "$author.last_name") 
                .append("page_sum", new Document("$sum", "$nrPages")))));
for (Document r: result) {
	System.out.println(r.toString()); 
}

which returns the total number of pages written by each author:

Document{{_id=Lemahieu, page_sum=12}}
Document{{_id=vanden Broucke, page_sum=637}}
Document{{_id=Baesens, page_sum=100}}

Document Store Indexes

You might be wondering how we can iterate over collections?
Aren't document stores key value stores where each keys is hashed to some independent value with no relation to the other?
Typically, in a document store we have for each collection a specialized document listing all of the items in that collection.
One can think of this as an index. For MongoDB, that is why we always have the _id field.
Many document stores support indexes on other document fields to speed up querying that field (otherwise, we have to scan through the whole list of documents given for the _id index, very much like a table scan).

To create an index in MongoDB we can use the command:

my_criteria = {title: "text", description: "text"};
db.my_collection.createIndex(my_criteria);

The above creates full text indexes on the title and description attributes of my_collection, allowing queries like:
```
db.my_collection.find( { $text: { $search: {"database mongodb"} } } )
```
to look up titles or descriptions with the string "database mongodb" in it.

Embedded Documents, Duplication, and Avoiding Duplication

One aspect missing from document stores is that of relationships between tables.
For example, in our Java code, we had book documents each with author information. What if we wanted to store more information about each author?

One way to solve this problem is to model each author as a more complicated document. So:

{
"title": "Databases for Beginners", "authors": ["J.K. Sequel", "John Smith"], "pages": 234
}

becomes

{
    "title": "Databases for Beginners", "authors": 
       [{"first_name": "Jay Kay", "last_name": "Sequel", "age": 54},
       {"first_name": "John", "last_name": "Smith", "age": 32} ], 
    "pages": 234
     
}

This technique is called using embedded documents.
The drawback is that we duplicate information whenever we have a new book with the same author.
Alternatively, we can store foreign keys to a author collection in the book document.
The drawback of this is, it will tend to mean you need to do nested queries in your apps to marshal author information for each book one wants to return.

More Document and Tuple Stores

Outline