Scaling a Web-site

On Monday we started talking about scaling web-sites to more data, more requests, and how to easily manage it all.
We talked about Memcache as a means to cache objects that had been marshalled from several database tables.
We showed how to get a Memcached daemon to listen for requests
We gave an example as a human connecting to a Memcache daemon and using its text protocol to stored lalala.
Today, we begin by looking at the protocol to retrieve a value stored with a key in Memcached, then we look at Squids configured as a reverse proxy servers.
This will allow us hopefully to serve static pages as often as possible rather than having to re-compute scripts again and again.
Let's recall the web-site architecture we had from Monday...

Example Web-site Architecture

Organization of a website show squids, http servers, memcached server, databases.

Demo of connecting to memcached as a human - get

To retrieve stored values, you can use either get followed by one or whitespace separated keys, or gets followed by one or more whitespace separated keys.
The server will respond with:
```
VALUE key flags bytes\r\n
data block\r\n
```
one for each value returned, followed by an END\r\n after the last value.
Continuing the example from the last slide we can type and get the response:
```
get foo
VALUE foo 0 6
lalala
END
```

Some Remarks on Memcached

Using CURL it would be possible to write a pure PHP program that connects with a memcache to store/retrieve values.
Instead, people usually use one of two PHP extensions: Memcache (older) or Memcached written in C.
These extensions are used because they allow for persistent (between php requests) connections to a memcache.
Typically, memcache's are used as a distributed cache. If one has 50Mb free on each of ten machines, one adds the ten servers to the memcache objects in PHP and it manages balancing and storing the key-value amongst these machines. This effectively would give you 500MB of total memory storage.
As of 2008, Facebook was making use of 15TB of memcache storage. As of 2013/2014, Facebook was still using memcache, but I don't have any statistics on hom much.
Twitter uses a variant of Memcache called Fatcache.
The distributed nature of memcached and the fact that is language agnostic, distinguishes it from things like APC (the alternative PHP cache -- built-in to PHP) which provide both op-code caching and single machine object caching.
Memory in memcached is allocated using a slab system with slabs of various sizes. When a given pool of slabs fills an LRU algorithm, is used to determine who is bumped from memory.
It should be noted that there is a 1MB maximum on the size of an object you can store with a given key.
As one common use of memcached is to store objects from ORM, this limit usually does not cause a problem. The PHP extensions do the job of serializing/unserializing these objects before putting them in/retrieving them from a memcache.

Using memcached in PHP

To use memcached within PHP one needs to create a memcache object:
```
$memcache = new Memcache();
```

Next one needs to set-up the memcache server:

$memcache->addServer($host, $port, $persistent, $weight, $timeout, $retry_interval);

To add something to the cache one could use a command like:

$memcache->set($key, $val, $flag, $expire); 
    // $flag is whether to compress the item in the store
    //$expire - Unix timestamp, or seconds from present. 0 -don't expire

To read from the cache one can use a line like:
```
       
$memcache->get($key);
```

Example Memcache Test

<html>
<head><title>Memcache Test</title></head>
<body>
<h1>Test</h1>
<?php
  $memcache = new Memcache();
  $memcache->addServer("localhost", 11211, true, 1, 1, 5);
  $memcache->set("bob", "value", null, 60);
  echo "<p>".$memcache->get("bob")."</p>";
?>
</body>
</html>

Squid

Is a caching proxy available at:
http://www.squid-cache.org/
It originally developed from Harvest Cache Daemon project at UCSD. The latter project started in the 1990s.
We are interested in using squids as a reverse proxies: The idea is clients (browsers), connect to our site and hit our squid, the squid determines if it already has a non-expired copy of the page requested in its cache, if it does, it returns it; otherwise, it forwards the request to one of a collection of webservers who runs the requested script. The returned result in this second case is then cached in the squid.

Setting up squid

We first need to download and compile the squid src's. (./configure; make; sudo make install)
Next we need to edit the squid.conf file in the squid ${squid}/etc/squid.conf
Then one can type the command:
```
/usr/local/squid/sbin/squid -z
```
This just makes the cache directories (${squid}/var/cache/hex_number), if you don't already have them .
Then one can run squid with:
```
/usr/local/squid/sbin/squid -NCd1 #for debugging
```
without -NCd1 if don't want debugging.

Testing Squid

To just test out squid, we will set it up first as a proxy cache.
The idea of a proxy cache is we want a cache for our LAN of any websites off of the LAN.
The squid.conf has the ability to finely control which requests through it will be allowed.
You can set up acl (access control lists). The default acl in squid.conf looks like:
```
acl localhost src 127.0.0.1/32
```
We next say what http stuff localhost is allowed to do by adding the following directive:
```
http_access allow localhost 
```
Next you can start your squid server, and either request web pages using the squidclient or set up a proxy of localhost and port 3128 in your browser.

For example:

${squid}/bin/squidclient http://pollett.org/
HTTP/1.0 200 OK
Date: Wed, 24 Nov 2010 20:57:21 GMT
Server: Apache/2.2.15 (Unix) mod_ssl/2.2.15 OpenSSL/0.9.8l DAV/2 PHP/5.3.3 SVN/1.6.5
X-Powered-By: PHP/5.3.3
Cache-Control: max-age=60
Expires: Wed, 24 Nov 2010 20:58:21 GMT
MS-Author-Via: DAV
Content-Length: 2231
Content-Type: text/html
X-Cache: MISS from chris-polletts-macbook-pro.local
Via: 1.0 chris-polletts-macbook-pro.local (squid/3.0.STABLE10)
Proxy-Connection: close

<!DOCTYPE html
PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
...

Notice in the above we had a cache miss.
To see the actual cache hits and misses you can look in ${squid}/var/logs/access.log
In Firefox to Configure the proxy server for to Preferences::Advanced::Network::Connection::Settings. Do manual proxy settings, use localhost as the host and 3128 as the port.

Configuring Squid as a Reverse Proxy

To match the sample website architecture we want the squid to be on the main machine people come into our website at.
Then we want it to cache pages that will be coming from our web servers.

We need to edit the lines:

http_port (probably set to port 80 from default 3128)
httpd_accel_host (set to virtual if have more than one web server; otherwise, to ip address of the web server)
httpd_accel_port (port number of web servers 80)
httpd_accel_single_host ( on or off depending on how many webservers you have,)
httpd_accel_uses_host_header off

Load Balancing Machines

You can also set up squid to do the load balancing between your web servers using directives like:

cache_peer ip.of.server1 parent 80 0 no-query round-robin
acl sites_server_1 dstdomain www.mysite.com
http_access allow sites_server_1

cache_peer ip.of.server2 parent 80 0 no-query round-robin
acl sites_server_2 dstdomain www.mysite.com
http_access allow sites_server_2

Benchmarking

Apache comes with a useful tool for benchmarking called Apache Bench.
One would typically run it from the command line with a line like:
```
ab -nSOME_NUM -cCONCURRENCY URL
```
The n controls the number of requests you are making, the c controls the number of requests to make at the same time.

Here is an example with output:

|fg5:search_yioop:105>ab -n2000 -c3 "http://localhost/git/yioop/?q=google"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        Apache/2.2.24
Server Hostname:        localhost
Server Port:            80

Document Path:          /git/yioop/?q=google
Document Length:        328 bytes

Concurrency Level:      3
Time taken for tests:   1.199 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Non-2xx responses:      1000
Total transferred:      553000 bytes
HTML transferred:       328000 bytes
Requests per second:    834.27 [#/sec] (mean)
Time per request:       3.596 [ms] (mean)
Time per request:       1.199 [ms] (mean, across all concurrent requests)
Transfer rate:          450.54 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    3  59.1      0    1084
Processing:     0    0   0.1      0       1
Waiting:        0    0   0.1      0       1
Total:          0    4  59.2      0    1085

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      1
  99%      1
 100%   1085 (longest request)

Let's Experiment

Pick two of the pages of your Hw3 (not Hw4) solution. For each page, get a transcript of doing 1000 requests to that page using Apache Bench. Try three different settings for the concurrency 1, 10, 100.

Post what you observed to the May 4 Discussion Thread.

Multimedia and HTML 5

Over the next week we are going to look at some of the multimedia features of HTML.
HTML 5 has optimizations to make it work well with phones and tablets.
In particular, it can make use of a devices sensors like current location and the accelerometer and it has a local storage mechanism.
The canvas tag allows drawing of 2d objects.
HTML 5 supports audio and video playback.
Web 3D provides an interface to OpenGL 3D graphics from HTML 5
WebRTC provides support for video conferencing.
We start with how to make our web apps a little more mobile friendly, along the way learning a little jQuery...

Mobile HTML 5

A mobile app created with HTML 5 is essentially a web page which employs features of HTML 5 to make it render nicely and to render rich content in a mobile device.
Such an app typically makes use of the Javascript language for networking as well as for persistence.
When coding such an app, people typically make use of an existing Javascript framework (a library written in Javascript and CSS) to provide a palette of pre-made, useful widgets which are cross-platform. Otherwise, you would have to code these yourself.
For this class, we are going to make use of jQuery Mobile.

A Template HTML 5 Document with jQuery Mobile

<!DOCTYPE html>
<html> 
    <head> 
    <title>Hello HTML 5</title> 
    <meta name="viewport" content="width=device-width, initial-scale=1"> 
    <link rel="stylesheet" href="http://code.jquery.com/mobile/1.2.0/jquery.mobile-1.2.0.min.css" /> 
    <script src="http://code.jquery.com/jquery-1.8.2.min.js"></script> 
    <script src="http://code.jquery.com/mobile/1.2.0/jquery.mobile-1.2.0.min.js"></script></head> 
<body> 

<div data-role="page">

    <div data-role="header">
        <h1>My First HTML APP!</h1>
    </div><!-- /header -->

    <div data-role="content">    
        <p>This is an HTML 5 app.</p>        
    </div><!-- /content -->

</div><!-- /page -->

</body>
</html>

Hello HTML 5 app.

The Head of a Document

In our example, the head of the document also uses the script tag to include the jQuery framework, and the jQuery mobile framework.
The link tag in our example is used to include jQuery's Cascading Style Sheet file. Stylesheets are used to specify how a tag should render in the browser.
In our example, we have the name attribute given the value viewport. This is a new HTML 5 name value.
The viewport of a browser is the virtual window area in which it renders a web page. This may be larger than the devices screen size (for example, you might need to scroll left or right or up down on the desktop to see the whole viewport).
The viewport affects how lines wrap, etc.
The area of the browser which is not rendered into ( the area containing the url, etc) is often called browser chrome).
The reason for the viewport meta tag is on a mobile device we often don't have scroll-bars -- we use pinch and zoom. The default viewport on a iOS device has a width of 980. Safari draws this whole width within the actual screen-width of 320 pixels/640pixels (on a 3GS on iphone 4 or higher) by scaling. There is more variety on android devices but similar numbers hold.
The content attribute associated with the name attribute viewport in the meta directive says how the browser should render the page within the viewport. For example, width=device-width. Says that the viewport width should be the actual devices width rather than 980. initial-scale controls the zoom level of the page when initially loaded (1 means no zoom).
We could have given a number of pixels rather than device-width. We could also have specified height as device-height or some number of pixels. Other possible directives are minimum-scale, maximum-scale, as well as user-scalable (yes/no as values).

Remarks on the Body of our Example

Notice in our example, the div tag has a data-role attribute. data-* attributes are user-defined attributes which were introduced with HTML 5.
HTML 5 supporting Javascript engines have a document object model including a dataset field for their elements. This allows Javascript to easily get at elements that have data-* attributes and manipulate them. This is what jQuery is using as part of how it draws things.
In CSS, we can also select such tags using selectors like div[data-role=header] .

Where to Find Examples

The jQuery Mobile Getting Started page has a lot of quick examples you can try out to get started with creating HTML 5 apps.
We conclude by showing one of the widgets from that page for displaying unordered lists.

The code you would put within your data-role="content" div tag would look like:

<ul data-role="listview" data-inset="true" data-filter="true">
    <li><a href="#">Red</a></li>
    <li><a href="#">Green</a></li>
    <li><a href="#">Blue</a></li>
    <li><a href="#">Violet</a></li>
    <li><a href="#">Ultra-Violet</a></li>
</ul>

This would render as this Unordered List Page.

More Memcache, Squid, Benchmarking, More HTML 5

Outline