Stream Processing

A streaming video can be viewed as a potentially infinite sequence of images.

An input device can be viewed as a potentially infinite sequence of characters.

Such sequences can be represented in Scala as streams (now called lazy lists).

Internally, a stream is represented as a linked list of cells representing some prefix of the sequence. However, the tail of the last cell is a promise to compute more cells if they're needed.

For example:

scala> val nums = LazyList(10, 20, 30, 40, 50)
val nums: scala.collection.immutable.LazyList[Int] = LazyList(<not computed>)

scala> nums.head
val res0: Int = 10

scala> nums
val res1: scala.collection.immutable.LazyList[Int] = LazyList(10, <not computed>)

scala> nums.tail.head
val res2: Int = 20

scala> nums
val res3: scala.collection.immutable.LazyList[Int] = LazyList(10, 20, <not computed>)

Notes:

·       Nums is a finite stream. Logically, it consists of five elements.

·       However, it is implemented as a single cell, Stream(10, ?), where ? represents a promise to compute more cells if needed.

·       The tail of this cell is the cell Stream(20, ?), where ? represents a promise to compute still more cells if needed.

·       When we re-inspect nums we now see two cells, the ones just computed.

·       A promise is a thunk (remember these from lazy evaluation?) A thunk is a frozen function call. In this case the thunk represents the ability to compute the next cell of the stream.

We can define infinite streams by a kind of never-terminating recursion:

def makeFibs(fib1: Int, fib2: Int): LazyList[Int] = fib1 #:: makeFibs(fib2, fib1 + fib2)

And now we create the list of all Fibonacci numbers:

val fibs = makeFibs(0, 1)

Notes:

·       makeFibs is a recursive function. It calls itself, but there is no base case. Each call has bigger inputs than the last.

·       a #:: b is stream-cons. It creates a cell with head = a and tail = a function which when called will create the next cell.  

·       In other words, the fatal recursive call doesn't happen. Instead, it gets put on ice for later. We only do as many recursive calls as the user requests. This is an example of Demand-Driven Programming: Only compute what is needed.

Inspecting the fifth element of the stream, fibs(4), executes the recursive call four times. So now the first five elements of the stream have been thawed out:

scala> fibs(5)
res56: Int = 5

scala> fibs
res57: LazyList(0, 1, 1, 2, 3, 5, <not computed>)

They're all there:

scala> fibs(10)
res43: Int = 89

scala> fibs(20)
res45: Int = 10946

scala> fibs(30)
res46: Int = 1346269

scala> fibs(40)
res47: Int = 165580141

scala> fibs(50)
res49: Int = -1109825406

Don't believe me? Try this:

Scala> for(fib <- fibs) print(fib + ", ")

(Save your work, first!)

We can also use map and filter on streams:

scala> val evenFibs = fibs.filter((n: Int)=>n%2 == 0)
evenFibs: scala.collection.immutable. LazyList [Int] = LazyList (2, ?)

scala> evenFibs(5)
res2: Int = 2584

But not reduce. (Why?)