Outline

• Multimedia Formats
  • JPEG
  • MPEG
  • MP3
• Quiz
• Network Security

Introduction

• Last week, we began talking about the presentation layer.
• Its role is to prepare/encode data such as integers, dates, html, xml, images, video, etc. from a process on one host so that it can be transported over the internet to a potentially different host where it can be decoded and made available to a process running there.
• We gave a taxonomy of presentation format design choices: (1) What data types will it support? (2) What conversion architecture strategy is used? (3) Will the data be tagged or untagged? (4) For RPC-protocols, are the stubs compiled or interpreted?
• We then went over some example presentation formats: XDR - used by SUN-RPC (RPC for NFS), ASN1 - used by SNMP, NDR - used by DCE (RPC for DCOM, ActiveX, Corba), XML - used for web services, most office suites.
• We then said most web traffic is multimedia data, and so it is important to understand not only these formats, but also the underlying compression technologies connected with them.
• We classified compression algorithms into two types: lossless (the original data can be exactly reconstructed from the compressed source), or lossy in which data close to, with respect to some metric, the original data can be reconstructed from the compressed string.
• We considered five lossless compression techniques: Huffman coding, run length encoding, differential pulse code modulation (DPCM), Delta-coding, and dictionary-based methods such as Lempel Ziv.
• Then we began looking at multimedia formats.
• For images, we said that such formats often need to specify a color space such as RGB (red, green, blue, the most common format), or YUV (brightness, blue projection, red projection).
• We used our discussion on compression and color spaces to then describe the GIF format.
• Today, we begin by looking at JPEG before moving to video formats.

JPEG

• JPEG (Joint Photographic Experts Group, 1992) was developed over several years beginning in 1986.
• It was based on patented work by companies such as IBM, Mitsubishi, AT&T.
• JPEG specifies a codec that defines how an image is compressed into a stream of bytes and decompressed back into an image, but not the file format used to contain that stream. The most common such file formats are EXIF and JFIF.
• The command line tool cjpeg is often considered a reference implementation of jpeg. Mozjpeg is a drop in replacement which produces higher compression but still outputs a jpeg stream.

JPEG Encoding

• Unlike GIF, JPEG does not reduce the number of colors in the starting image to those from a color palette.
• Instead, JPEG starts off by converting the RGB colors you get from a uncompressed image format like BMP, PPM, or what you get out of a digital camera to YUV color space.
• The equations to do this are:

  Y = 0.299R + 0.587G + 0.114B  (Brightness)
  U = (B-Y) * 0.565  (Projection on blue component of chrominance)
  V = (R-Y) * 0.713 (Projection on red component of chrominance)
  

• As an example white in RGB, (255, 255, 255), gets mapped to (255, 0, 0), but the mapping to Y does not use an equal share of R, G, B. The values 0.299, 0.587, and 0.114 are based on human abilities to perceive these colors of lights.
• The above transformation is invertible. I.e., we can map back from YUV to RGB (Exercise).
• The reason for doing this transformation is that human eyes are more sensitive to differences in brightness (an average human eye has 120 million rod cells) than color (an average human eye has 6 million cone cells and these require more light to activate). So to reduce image size, we are going to use more bits for Y than U and V.
• So far, though, we can imagine we started with an image as a sequence of pixels each consisting of a byte for R, G, B values and have mapped this to a sequence of pixels each consisting of a byte for Y, U, V values.

Compressing YUV Components

• Subsampling is used to compress U, V data. That is, we average U and V for a group of pixels and send that.
• To illustrate, suppose we started with a 16 x 16 array. We would keep a 16 x 16 array of Y bytes, for U and V we treat four adjacent pixels as a group and average them to a single value. So for these components we'll send 8 x 8 arrays as in the picture above.
• More precisely, for each 2x2 square in the original we send four Y values (4 bytes), 1 U value (1 byte), and 1 V value (1 byte). So what original would have been 12 bytes (if used 3 bytes for each of Y, U, V) is compressed to 6 bytes - a 50% reduction.

JPEG Compressing - Overall Flow

• After subsampling we have 3 grids of pixels, and we further compress each of these separately.
• The above diagram illustrates the steps of this further compression: DCT, Quantization, and Encoding
• We consider each of these in turn.

DCT Phase

• A grid of 8x8 values from one of the three Y, U, V channels can be viewed as a kind of elevation map/surface.
• To specify this surface, we can, for each `(x,y)` value, give its height. Alternatively, we could write the surface in terms of adding or subtracting 2D cosine functions of various frequencies.
• Going from point height representation to sum of 2D cosine representation is done by the Discrete Cosine Transformation (DCT). The reverse map is called the inverse DCT.
• The DCT was developed for signal processing by Nasir Ahmed in 1972 based on earlier Fourier Transform Techniques. The reason cosines rather than sines are used is that they tend to match boundary values better for discrete N x N squares of values.
• The DCT and inverse map are: $$DCT(i,j) = \frac{1}{\sqrt{2N}} C(i)C(j) \sum_{x=0}^{N-1} \sum_{y=0}^{N-1} \mbox{pixel}(x,y) \\ \times cos\big[\frac{(2x +1)i\pi}{2N}\big] cos\big[\frac{(2y +1)j\pi}{2N}\big]$$ $$\mbox{pixel}(x,y) = \frac{1}{\sqrt{2N}} \sum_{i=0}^{N-1} \sum_{j=0}^{N-1} C(i)C(j)DCT(i,j) \\ \times cos\big[\frac{(2x +1)i\pi}{2N}\big] cos\big[\frac{(2y +1)j\pi}{2N}\big]$$ where `C(x) = 1/sqrt(2)` when `x=0` and 1 when `x > 0`. For JPEG, `N=8`.
• Here for each `(i,j)`, `C(i)C(j)cos[\frac{(2x +1)i\pi}{2N}] cos[\frac{(2y +1)j\pi}{2N}]` is a particular 2D cosine function. The image above shows what these functions look like.
• DCT(i,j) represents the coefficient we want to multiply the `(i,j)`th such function by after the conversion.
• Using DCT on each 8 x 8 block of the original images produces 8 x 8 blocks of 2D-cosine coefficients. Notices lower frequency 2D cosine functions correspond to averages over more pixels. Location (0,0), called the DC coefficient, corresponds to an average over the whole grid.
• The remaining 2D cosine function coefficients are called AC coefficients (alternating coefficients).

Quantization Phase

• DCT is lossless (assuming infinite precision arithmetic), but as we observed on the last slide the DC and lower frequency AC components typically will contain the most information about a typical block of 8x8 pixels.
• So to compress data, we'd like to use fewer bits for higher frequency coefficients and more for lower frequency coefficients.
• Another issue is the 8 x 8 array of (x,y) bytes of the previous stage has now become an 8 x 8 array of floats.
• To get back to bytes and also to use fewer bits for higher frequencies, we will divide a float in the (i,j) coordinate by the (i,j) entry in the table above and round.
• This process is called quantization.
• More precisely, the formulas for quantization are:

  QuantizedValue(i,j) = IntegerRound(DCT(i,j), Quantum(i,j)) where
  IntegerRound (x) = 
     Floor(x + 0.5) if x >= 0
     Floor(x - 0.5) if x < 0
  

• For example, if DCT(0,0) was 25, then using the table above the quantized value would be Floor(25/3 + 0.5) = 8. To go back to the original value we can use the table above to get `8 times 3 = 24`, close to the original value.

Encoding Phase

• The final phase of JPEG writes out each 8 x 8 macroblock in a linear format using the path in the diagram above.
• Given the quantization values used on the last slide, often only coefficient from the top left corner of the 8 x 8 block are non-zero.
• To further compress the sequence of values, run length encoding is employed.
• To actually write the values a Huffman code is used.
• Because the DC coefficient usually has most of the information about an 8 x 8 block from the source image, and because images tend to change slowly between 8x8 blocks, each DC coefficient is encoded as its difference from the previous DC coefficient (delta code).
• JPEG has a number of variations that control things like amount of compression.
• Control of compression is typically done by using a different quantization table.
• JPEG often achieves compression ratios of 30:1 over an uncompressed image, depending on which quantization table used.

Quiz

Which of the following is true?

1. The handling of packets marked with the Assured Forward PHB is done using RIO.
2. XDR uses a receiver-makes-right conversion strategy to convert between machine architectures.
3. NDR uses a canonical-intermediate-form conversion strategy to convert between machine architectures.

Video Compression (MPEG)

• MPEG (Moving Picture Expert Group) is an alliance of ISO and IEC working groups founded in 1988 by Yasuda and Chiariglione.
• They have developed a series of video and audio formats. We are going to focus on MPEG1, which is a starting point anyway for the more advanced formats.
• As we discuss MPEG1, we will ignore the audio signal that would also typically be encoded with the video.
• Muybridge's horse photo experiments from 1878 showed that video can be viewed as a succession of still images (aka frames).
• For video, we usually encode at 30 frames per second (fps), although sometimes people use 60fps or 120fps. Old school movies were at 24 fps, NTSC and VHS were at 29.97 fps.
• Each frame in a video could be compressed using a DCT-technique like JPEG.
• The difference between two such frames is typically small though, so MPEG uses a variety of techniques to achieve further compression.

Frame Types

• To encode in MPEG1, the input sequence of video frames is compressed into three types of frames I-frames (intrapicture), P-frames (predicted picture), and B-frames (bidirectional predicted picture)
• Each encoded frame begins with information specifying its type (either I, P, or B).
• To a first approximation I-frames are source frames encoded in format very similar to JPEG except rather than use 8x8 macroblocks, it uses 16x16 macroblocks.
• A B-frame, `F_c` is encoded as a compressed sequence of 16 x 16 macroblocks. These macroblocks might be the same as an I-frame macroblock, if the video is changing too much in that macroblock, or might be represented as a 4-tuple with components:
  1. a coordinate for the macroblock in the frame,
  2. a motion vector `(x_p, y_p)` relative to the previous reference I frame (denoted `F_p`),
  3. a motion vector `(x_f, y_f)` relative to the next P or I frame (denoted `F_f`),
  4. a delta `delta(x,y)`, for each pixel in the macroblock computed using: `F_c(x,y) = (F_p(x+x_p, y+y_p) + F_{f} (x+x_f, y+y_f))/2 + delta(x,y)`
• P frames are encoded similarly, except only using `F_p` and not `F_f`.

More MPEG details

• To figure out, during encoding, what should be the motion vectors for a given macroblock is done using heuristic techniques used for motion estimation.
• P- and B-frames are typically mainly 0's so can be highly compressed.
• The input video sequence of the previous slide produces a sequence of frame types: I B B P B B I.
• For decoding purposes, we need to have the reference I frame to decode a P frame and the reference I and P frames to decode a B frame. So we would transmit the frames above as IPBBIBB.
• The MPEG spec does not say how many P- or B-frames to use. You can have a valid MPEG1 file using only I-frames.
• Typically, the compression ratio of I-frames is 30:1, and P and B frames 90:1 to 150:1.

Transmitting MPEG

• MPEG specifies the format of a video stream which could be stored in a file or sent over a network.
• We next look at how such a stream might be sent over a network using MPEG main profile (think of a profile as like a version or a variant).
• A main profile MPEG stream has a nested structure.
• The outermost level, contains a sequence of groups of pictures (GOPs) separated by a SeqHdr.
• A sequence is terminated by a SeqEndCode (0xb7).
• The SeqHdr specifies the size of each frame in the GOP measured in both pixels and number of macroblocks, the interpicture periods measure in microseconds, and two quantization matrices for use within the macroblocks: one for use with I-frame block, one for use with B and P-frame blocks.
• It is possible to change the quantization table and frame rate at GOP boundaries to adapt the stream to network circumstances.
• Each GOP begins with a GOPHdr followed by the set of frames making up the GOP. This header contains: the number of pictures and when the pictures should play relative to the beginning of the video.
• Each picture/frame, begins with a PictureHdr and a set of slices that partition the picture (horizontal lines). The PictureHdr identifies the picture as an I, P, or B frame and gives a picture specific quantization table.
• The SliceHdr gives the vertical position of the slice plus another opportunity to change the quantization table, this time by a constant factor rather than by giving a new table.
• The SliceHdr is followed by the actual macroblocks. The header for a macroblock says where it comes from in the picture.
• When splitting mpeg into packets, if TCP is being used, then TCP decides when it has enough bytes to send an IP datagram; if UDP is being used, typically, one chooses macroblock boundaries as packet split points.
• If MPEG is being used for video conferencing typically B-frames are not used when encoding, sometimes P are also not used.

Audio Compression (MP3)

• MPEG also defines standards for compressing audio.
• One such standard, MP3, can be used to compress the audio portion of a movie and the MPEG standard also says how audio is interleaved with compressed video in a single stream.
• We assume that audio is sampled from some source say at a rate of 44.1kHz (one sample every 23 microseconds - CD quality).
• Each sample is 16 bits measure of sound intensity. This means that a stereo (2-channel) audio stream results in a bit rate of
  `2 times 44.1 times 1000 times 16= 1.41 Mbps`.
• By comparison, telephone quality voice is sampled at 8kHz with 8 bit samples, resulting in a bit rate of 64kbps.
• Synchronization and error correction actually require 49 bits rather than 16 bits per sample, so we actually need a bit rate of: `49/16 \times 1.41 = 4.32 Mbps`.
• MP3 uses techniques similar to those used by MPEG video. It splits the audio into some number of frequency sub-bands (just like YUV channels in images) based on how human perceive sound.
• Each sub-band is split into blocks (like macroblocks) that can vary in length from 64 to 1024 samples.
• Each block is transformed using a modified DCT, quantized, and Huffman encoded, just as for MPEG video. Rather than 2D cosine basis functions, we instead used 1D cosine basis functions in our DCT.

Network Security

• If someone were watching the packets destined for some process on some host, they would able to read the content of the packets.
• To prevent spying, we now look at techniques to encrypt data so that even if we can see the contents of a packet, as they are encrypted, we cannot determine what they originally said.
• Let's list some capabilities we might want for a secure network communication protocol:
  • A protocol that allows for encrypting of packet contents is said to afford confidentiality.
  • If even the quantity and destination of communication is hidden, then we say the protocol assures traffic confidentiality.
  • A protocol that guarantee messages cannot to tampered with is said to provide integrity.
  • A protocol that ensures a process is communicating with who it thinks it is supposed to be communicating with is said to support authentication.
• Here are some other issues which are related to network security:
  • If the network communication is affecting the state of a remote server (say a web server), we might want that a given authenticated user can only do what they are allowed to do by the server's owner. The problem of controlling an authenticated user's access to a service is called access control.
  • A malicious source can also send too much data to a destination preventing other people from accessing it (Denial of Service). So we are also interested in the degree to which an application is usable on the internet, its availability.
  • The internet also makes available malicious code, malware. If this code can replicate itself over the internet without the need of a human, say by exploiting vulnerabilities in servers, it is called a worm. If it spreads itself through files it is called a virus.

Ciphers

• We now look at techniques to ensure confidentiality on the web. I.e., encryption.
• Encryption transforms a message in such a way that it becomes unintelligible to any party that does not have the secret of how to reverse the transformation.
• The sender applies an encryption function to the original plaintext message resulting in a ciphertext message which is sent over the internet.
• The receiver applies a secret decryption function -- the inverse of the encryption function -- to recover the original message.
• The transformation represented by an encryption function and its corresponding decryption function is called a cipher.
• Nowadays, the encryption and decryption function are parameterized by a key such that only the key, and not also the encryption and decryption algorithms, needs to be kept secret.