Benchmarking

Our next topic is going to be how to scale our web sites.
Before we get into this, it is useful to look at how we can measure how well our web site can currently handle traffic.
Apache comes with a useful tool for benchmarking called Apache Bench.
One would typically run it from the command line with a line like:
```
ab -nSOME_NUM -cCONCURRENCY URL
```
The n controls the number of requests you are making, the c controls the number of requests to make at the same time.

Here is an example with output:

|fg5:search_yioop:105>ab -n2000 -c3 "http://localhost/git/yioop/?q=google"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        Apache/2.2.24
Server Hostname:        localhost
Server Port:            80

Document Path:          /git/yioop/?q=google
Document Length:        328 bytes

Concurrency Level:      3
Time taken for tests:   1.199 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Non-2xx responses:      1000
Total transferred:      553000 bytes
HTML transferred:       328000 bytes
Requests per second:    834.27 [#/sec] (mean)
Time per request:       3.596 [ms] (mean)
Time per request:       1.199 [ms] (mean, across all concurrent requests)
Transfer rate:          450.54 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    3  59.1      0    1084
Processing:     0    0   0.1      0       1
Waiting:        0    0   0.1      0       1
Total:          0    4  59.2      0    1085

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      1
  99%      1
 100%   1085 (longest request)

Scaling a Web-site

Today we will look at what's needed to move your site from handling a few thousand hits a day to thousands of hits per second.
Rough traffic (2008) - addons.mozilla.com (3000-6000 hits/second); wikipedia.org, a top ten site (30,000 -40,000 hits/second). (I know this is old, but I am reasonably sure of the numbers, and would be roughly ballpark for a large website.)
Scaling is a term that is often used in a vague sense.
To make this term precise, we will say a web-site is scalable if:
- It can accommodate increased usage
- It can accommodate an increased dataset
- It is maintainable.
Throwing more hardware at a problem might help solve the first two points, but if the additional hardware makes the system to hard to manage three will fail.
Let's look at an example web-site architecture in terms of scalability.

Example Web-site Architecture

Organization of a website show squids, http servers, memcached server, databases.

Quiz

Which of the following statements is true?

When the XmlHttpRequest object's readyState property is 4, the server's response to a sent request has been completely received.
In REST, the remote procedure call is specified by sending the server an XML message with the name of an object and method.
Target blank attacks, as described in class, can allow an attacker to hack a user's account without social engineering.

Database Part of the Architecture

The diagram on the last slide only shows one master DB and many slaves.
The slaves would be read only and the master usually supports reads and writes. The slaves could be kept in sync with the master using rsync.
To handle more data might have several master DBs and horizontally partition the data between them.
We will mainly look-at how to handle increase usage.

ORMCache Part of the Architecture

If our web-site is built using an MVC/MVA pattern, then one of the common sources of database queries will be marshaling data for our models.
Let's briefly consider how this is done.
Typically, models are used to get objects into and out of a database.
As an example, a user might make many posts to a discussion board.
This is an example of a one-to-many relationship.
We might have two database tables user and post. To represent this relationship the post table might have a column uid as a foreign key reference into the user table.
Now a User object might be modeled as PHP associative array consisting of the name, and other fields of the user, one of these fields being an array of posts objects:
```
Array( "name" => "Bob", ..., "posts" => Array([0] => Array("title" => "Dark and Stormy Night", ...), [1]=>...))
```
An object relational mapping is a mapping between these kind of objects and database tables.
Besides one-to-many relationship, one might need to handle many-to-many, belongs-To, and one-to-one relationships.
In pretty much all cases, marshaling an object requires several database queries, which can be slow, so may be sped up by caching.
An ORM Cache such as Redis or Memcache solves this problem by providing a RAM-based key value store for objects that have been previously marshaled and which have not been written to.

Reverse Proxy Part of the Architecture

The reverse proxy in front of the architecture from a couple slides back may provide several functions:
1. Act as a cache for previously calculated whole web pages that have not changed.
2. Choose which web server should handle a request (might have several machines all running the identical web app).
3. Hardware support for things like SSL.
The reverse proxy in front of everything is typically implemented using Varnish, Squid, or a hardware specific system.
An advantage of a layer seven network appliance hardware over a Squid is that it can do SSL in hardware. On the other hand, hardware is harder to change.
This completes our brief overview of our architecture.
I'd like to point out that many products nowadays run virtualized and cloudified as well.

Content Delivery Networks

One problem with even an architecture like that provided a couple slides back, is that traffic all still has to come to your subnetwork of the internet and this could lead to a bottleneck.
To scale so that geographic effects don't mess with your ability to provide content, you often would also use a content delivery network.
Akamai is one early example of such a network. There is also Cloudflare. Amazon has Cloudfront and Microsoft has Azure.
CDN's are often implemented using a distributed hash table.
Like a usual hash table, such a table stores key value pairs.
Unlike a hash table, there is also a distance function d(k1, k2) which can be used to compare the distance between two keys.
A key might contain information about a url, an ip address (geographic info), and a replication number; a value might be a web page.
Each node in the network also has a key associated with itself.
Nodes in such a DHT network usually are arranged in some kind of topology like a ring with additional edges to keep the number of hops between any two nodes logarithmic.
When a request comes in to one node in this network, it checks if any neighbor has distance closer to the key, if so it let's the closest neighbor try to handle the request; otherwise, it handles it.

C10k Problem and its Variants

Ideally one would like to run ones web app on as few machines as possible because it will be easier to manage.
The C10k refers to 10,000 simultaneously open connections on a single machine and was coined by Kegel in 1999.
Modern variants replace k with some number and M.
Originally, 10,000 connections was a bottleneck for Apache web servers because the server uses multiple concurrently running modules, each in its own process, each of which in turn runs multiple threads.
Sockets connections require file handles. The operating system usually places a limit on the number of threads and file handles that a process is allowed to have open. The limits on threads are often caused by how much memory one has and Apache threads can be heavy.
Once these limits have been exhausted, blocking occurs, slowing down the server.
One approach to avoid this issue is to use event-driven I/O and not spawn threads or processes. This reduces the memory footprint of the server and then one can use ulimit or some other such construct to raise the limit on the allowed open connection for a process.
This is the approach the nginx (Engine X) web server takes and it is also the approach of Node.

Node.js -- Introduction

Node.js (sometimes I'll just say Node) is an open-source, Javascript runtime environment for executing Javascript on the server.
It was originally developed by Ryan Dahl in 2009 and uses an event-driven approach (remember promises).
It was designed to solve a problem that most web apps tend to either write sequential code that blocks while waiting for I/O or incurs multiple execution stacks if dealing with more than one thread.
To install Node, on Ubuntu Linux one can type:
```
sudo apt install nodejs
```
On Windows or a Mac you can download an installer from https://nodejs.org/en/download/
Or a Mac, install Homebrew. Then type:
```
brew install node
```
You can test your installation, by opening a shell and typing either node or nodejs (I am assuming your path variables are set up correctly).
This is the node REPL (read, eval, print loop).

You should see a prompt ">" at which you can type Javascript commands:

> a = 10
10
> a + 5
15
> console.log('yo')
yo
undefined
> .help
.break    Sometimes you get stuck, this gets you out
.clear    Alias for .break
.editor   Enter editor mode
.exit     Exit the repl
.help     Print this help message
.load     Load JS from a file into the REPL session
.save     Save all evaluated commands in this REPL session to a file
> .exit

Example Node application

var http = require('http'); // module pattern object for creating a server
   // If want ssl use require('https') and add key info

http.createServer(function (request, response) {
    response.writeHead(200, {'Content-Type': 'text/plain'}); 
    // 1st arg is HTTP response type, 2arg is HTTP headers as JSON object
    response.end('Node is working\n');
    // body of response
}).listen(8888);

console.log("Server is up!");

Example Parsing the Query String

var http = require('http');
var fs = require('fs');

http.createServer(function (request, response) {
    var name = require('url').parse(request.url, true).query.name;
    // gets name from the query string
    if (name === undefined) {
        name = "myphoto";
    }
    if (name == 'myphoto') {
        var file_name = name + '.jpg';
        fs.stat(file_name, function (error, status) {
            //callback when promise ready. I.e., have stat info about file
            if (error) {
                console.error(error);
                response.writeHead(404, {'Content-Type': 'text/plain'});
                response.end("myphoto.jpg not available!");
            } else {
                var image = fs.readFileSync(file_name);
                response.contentType = 'image/jpeg'; 
                    //alternative was to set Content-Type
                response.contentLength = status.size;
                response.end(image, 'binary'); // send JPEG photo
            }
        });
    } else {
        response.writeHead(200, {'Content-Type': 'text/plain'}); 
        response.end('Name was:' + name);
    }
}).listen(8888);

console.log("Server is up!");

Node's Package Manager

npm stands for Node Package Manager, and it is Node's equivalent to Composer we saw for PHP.
If you used one of the Windows or Mac installers then npm should come bundled with it, although you might need to adjust your path variables to use it.

You can also install it with the command:

On Linux:
sudo apt install npm
On Mac:
brew install npm

To install a package you can use the command install:
```
sudo npm install some_package
```
One can tack a "-g" if one wants to do the operation globally rather than for the node package you are defining. You can add --save if you want to add the package to the list of dependency of the project/package you are deifning.
To update Node modules one can type:
```
sudo npm update -g
```

Express.js

Express.js is a web application framework built on top on Node.
Suppose we wanted to build a web app.
We could make a folder and switch into it:
```
mkdir webapp
cd webapp
```

Using npm we can create a new package and entry point into our project:

npm init
// bunch of prompts, use default, except
entry point: (index.js)

At this point, we should get a package.json file.
Next we can install express.js (--save adds it to the list of dependencies in the package.json):
```
npm install express --save
```
Modules are saved in the node_modules sub-folder.

Express index.js file Example

var express = require('express')
var app = express()

// can add different routes
app.get('/', function (req, res) {
  res.send('This is an express app using route /')
})

app.listen(8888, function () {
  console.log('Server up!')
})

We can then run the project by typing at a shell prompt:

node index.js

Scaling Websites, Node.js

Outline