Scaling a Web-site

Today we will look at what's needed to move your site from handling a few thousand hits a day to thousands of hits per second.
Rough traffic (2008) - addons.mozilla.com (3000-6000 hits/second); wikipedia.org, a top ten site (30,000 -40,000 hits/second). (I know this is old, but I am reasonably sure of the numbers, and would be roughly ballpark for a large website.)
Scaling is a term that is often used in a vague sense.
To make this term precise, we will say a web-site is scalable if:
- It can accommodate increased usage
- It can accommodate an increased dataset
- It is maintainable.
Throwing more hardware at a problem might help solve the first two points, but if the additional hardware makes the system to hard to manage three will fail.
Let's look at an example web-site architecture in terms of scalability.

Example Web-site Architecture

Organization of a website show squids, http servers, memcached server, databases.

Remarks on Architecture

The diagram on the last slide only shows one master DB and many slaves.
The slaves would be read only and the master usually supports reads and writes. The slaves could be kept in sync with the master using rsync.
To handle more data might have several master DBs and horizontally partition the data between them.
We will mainly look-at how to handle increase usage.
On Wednesday we'll talk about how to handle caching of web-pages using Squids.
An advantage of a layer seven device like a Netscaler, etc hardware over a Squid is that it can do SSL in hardware. On the other hand, hardware is harder to change. Many products nowadays run virtualized and cloudified as well.
We will focus today on how to handle increased database traffic where most of the traffic comes in the forms of reads.

Content Delivery Networks

One problem with even an architecture like that provided a couple slides back, is that traffic all still has to come to your subnetwork of the internet and this could lead to a bottleneck.
To scale so that geographic effects don't mess with your ability to provide content, you often would also use a content delivery network.
Akamai is one early example of such a network. Amazon has Cloudfront and Microsoft has Azure. One also has things like BitTorrent.
One example of a free CDN is the Coral CDN.
To cache my web page,
```
http://www.cs.sjsu.edu/faculty/pollett/
```
in this network I simple make a request to:
```
http://www.cs.sjsu.edu.nyud.net/faculty/pollett/
```
i.e., I tack on .nyud.net to the domain name.
CDN's are often implemented using a distributed hash table.
Like a usual hash table, such a table stores key value pairs.
Unlike a hash table, there is also a distance function d(k1, k2) which can be used to compare the distance between two keys.
A key might contain information about a url, an ip address (geographic info), and a replication number; a value might be a web page.
Each node in the network also has a key associated with itself.
Nodes in such a DHT network usually are arranged in some kind of topology like a ring with additional edges to keep the number of hops between any two nodes logarithmic.
When a request comes in to one node in this network, it checks if any neighbor has distance closer to the key, if so it let's the closest neighbor try to handle the request; otherwise, it handles it.

Object Relational Mappings

If our web-site is built using the MVC pattern that we have been using on the homeworks, then one of the common sources of database queries will be marshaling data for our models.
Let's briefly consider how this is done.
Typically, models are used to get objects into and out of a database.
As an example, a user might make many posts to a discussion board.
This is an example of a one-to-many relationship.
We might have two database tables user and post. To represent this relationship the post table might have a column uid as a foreign key reference into the user table.
Now a User object might be modeled as PHP associative array consisting of the name, and other fields of the user, one of these fields being an array of posts objects:
```
Array( "name" => "Bob", ..., "posts" => Array([0] => Array("title" => "Dark and Stormy Night", ...), [1]=>...))
```
An object relational mapping is a mapping between these kind of objects and database tables.
Besides one-to-many relationship, one might need to handle many-to-many, belongs-To, and one-to-one relationships.
In pretty much all cases, marshaling an object requires several database queries, which can be slow, so may be sped up by caching.

Quiz

Which of the following statements is true?

We said you should use the HTTP X-Frame-Options header to prevent XSS attacks.
PHP functions beginning with mb_, such as mb_substr, provide support for multi-byte character encodings like UTF-8.
gettext is used to download code from a remote website.

Memcached

Memcached (http://memcached.org/) allows you to create an in memory cache of data.
A memcache can be on a separate server from either your database or your webserver.
Your web app connects to the memcache daemon and checks if an item such as an object that might have been obtained from a database using an ORM mapping, is in the memcache. If it is you can get them quickly, if it isn't then you go to the database.
In PHP you can access a memcache daemon using the PECL extension:
http://pecl.php.net/package/memcache.
To install this you type
```
pecl install Memcache
```
and then add the line
```
extension=memcache.so
```
(or .dll on Windows) to your php.ini file and restart everything.

Running memcached

To run the memcached daemon one can type a line like:
```
memcached -d -m 50 -l 127.0.0.1 -p 11211
```
Here:
- -d -- as daemon
- -m -- number of megabytes
- -l -- location
- -p -- port
The original versions of memcached didn't have any security protocol, so the -l option is important -- it is allowing only traffic from the loopback network to connect to memcached.

Demo of connecting to memcached as a human - set

Memcached has two protocols for interacting with it: a text-based and a binary format.
We will demo connecting to memcache using telnet and the text-based protocol.

To connect and do a set we could do:

telnet 127.0.0.1 11211
set foo 0 0 6
lalala
STORED

Here set is the command.
foo is the key which we can use to look up the value we stored later.
The first 0 is a 16 or 32 bit (1.2.1 or higher), decimal written flag which could be used to optionally store info.
The second zero indicates the expiry time in seconds -- 0 means never expire.
The 6 indicates the number of bytes we'll store not including the final carriage return linefeeds which terminate the data block.
After these value we have a \r\n and then we begin the data block which is terminated with a \r\n.
Finally, memcached responds with STORED.
Other storage commands include: add, replace, append, and prepend.
There are also commands: delete, to delete entries, incr, decr to increment decrement the value associated with a key, and stats to output various statistics.

Demo of connecting to memcached as a human - get

To retrieve stored values, you can use either get followed by one or whitespace separated keys, or gets followed by one or more whitespace separated keys.
The server will respond with:
```
VALUE key flags bytes\r\n
data block\r\n
```
one for each value returned, followed by an END\r\n after the last value.
Continuing the example from the last slide we can type and get the response:
```
get foo
VALUE foo 0 6
lalala
END
```

Some Remarks on Memcached

Using CURL it would be possible to write a pure PHP program that connects with a memcache to store/retrieve values.
Instead, people usually use one of two PHP extensions: Memcache (older) or Memcached written in C.
These extensions are used because they allow for persistent (between php requests) connections to a memcache.
Typically, memcache's are used as a distributed cache. If one has 50Mb free on each of ten machines, one adds the ten servers to the memcache objects in PHP and it manages balancing and storing the key-value amongst these machines. This effectively would give you 500MB of total memory storage.
As of 2008, Facebook was making use of 15TB of memcache storage.
This distributed nature of memcached and the fact that is language agnostic, distinguishes it from things like APC (the alternative PHP cache -- will be built-in to PHP) which provide both op-code caching and single machine object caching.
Memory in memcached is allocated using a slab system with slabs of various sizes. When a given pool of slabs fills an LRU algorithm, is used to determine who is bumped from memory.
It should be noted that there is a 1MB maximum on the size of an object you can store with a given key.
As one common use of memcached is to store objects from ORM, this limit usually does not cause a problem. The PHP extensions do the job of serializing/unserializing these objects before putting them in/retrieving them from a memcache.

Using memcached in PHP

To use memcached within PHP one needs to create a memcache object:
```
$memcache = new Memcache();
```

Next one needs to set-up the memcache server:

$memcache->addServer($host, $port, $persistent, $weight, $timeout, $retry_interval);

To add something to the cache one could use a command like:

$memcache->set($key, $val, $flag, $expire); 
    // $flag is whether to compress the item in the store
    //$expire - Unix timestamp, or seconds from present. 0 -don't expire

To read from the cache one can use a line like:
```
       
$memcache->get($key);
```

Example Memcache Test

<html>
<head><title>Memcache Test</title></head>
<body>
<h1>Test</h1>
<?php
  $memcache = new Memcache();
  $memcache->addServer("localhost", 11211, true, 1, 1, 5);
  $memcache->set("bob", "value", null, 60);
  echo "<p>".$memcache->get("bob")."</p>";
?>
</body>
</html>

Scalability

Outline

Scaling a Web-site

Example Web-site Architecture

Remarks on Architecture

Content Delivery Networks

Object Relational Mappings

Quiz

Memcached

Running memcached

Demo of connecting to memcached as a human - set

Demo of connecting to memcached as a human - get

Some Remarks on Memcached

Using memcached in PHP

Example Memcache Test