Outline

• String Compression with CFGs
• Other CFG algorithms

Grammar-Based Compression Algorithms

• Besides their use in defining parsable programming languages, CFGs, can also be used in text compression algorithms.
• The Liv Zempel family of compression algorithms (Liv Zempel 1977 and 1978, Welch 1984) can be viewed as an example of this approach.
• The idea is given a string we want to compress, we try to come up with a small CFG that generates only that string. Then we encode the string by its grammar written in some suitable encoding for grammars.
• There are brute force algorithms (i.e., slow) algorithms to find the smallest CFG for a given string.
• LZ 78 is fast and known to be able to approximate the smallest grammar to within a factor `Omega(n^(2/3)/log n)` as lower bound and to within a factor an `O((n/log n)^(2/3))` factor as an upper bound.
• It is known to approximate the smallest grammar to within a `8569/8568` factor is NP-hard (Lehman and Shelat 2002).
• Charikar, et al. ( IEEE Transaction of Information Theory. 2005. pp. 2554--2576) give a compression algorithm that approximates the smallest grammar to within a factor of `O(log(n/g^star))` where `g^star` is the size of the smallest grammar for that string.
• We will present a simpler to describe grammar-based compression algorithm known as SEQUITUR (Nevill-Manning, Witten, Maulsby, 1994, 1997) which runs in linear time and space and also approximates the smallest grammar to within a `Omega(n^(1/3))` factor as a lower bound and `O((n/log n)^(3/4))` as an upper bound.

SEQUITUR

• For this algorithm we will make use of an auxiliary string `mbox(Aux)` that contains the results of the compression process so far.
• We will use a hash table called `mbox(Hash)` that supports deletion which will have pairs of symbols that appear in `mbox(Aux)`. This will allow fast look up of these pairs.
• We will also have a table of productions we have generated so far. The right hand side of these productions will always have length at most `2` before the final step of this algorithm is applied.
• At the start of the algorithm `mbox(Aux)` is empty, `mbox(Hash)` is empty, and our production table is empty. Let `w= w_1, ldots, w_n` be the string we'd like to compress. The algorithm proceeds on this input as follows:
  1. For `i = 1` to `n`: Add `w_i` to the end of `mbox(Aux)` and look at the last two symbols of `mbox(Aux)`. These might be a mix terminals and variables. If:
     1. The last two symbols can be found on the right-hand side of some production in our table, replace in `mbox(Aux)` these two symbols by the left hand side variable of the production in question. Mark the production as having been used more than once.
     2. The last two symbols occur somewhere else earlier in `mbox(Aux)` (we can check by looking up these two symbols in `mbox(Hash)`). Replace the two symbols with a new variable and a new production to our table consisting of this new variable going to these two symbols. Delete these two symbols from `mbox(Hash)`. Add to `mbox(Hash)` the two symbols starting from where we replaced the earlier occurrence.
     We then place the last two symbols of `mbox(Aux)` in `mbox(Hash)`. We perform the above two checks in sequence and might do both and repeat if we can still apply either.
  2. Remove rules that are only used once in the grammar by compressing them. So if `A->BC`, `B->de`, and `C->fg` and the latter two rules are only used once, then we delete these rules and add the rule `A->defg`.
  3. Finally, we output the grammar consisting of the rules in our production table together with a new start variable S that is used in the rule `S -> mbox(final value of Aux)`.

Example of Compressing with SEQUITUR

• Suppose the input string was `\a\b\b\a\b\a\b\b\a\b\b\a\b\b`. The following table shows the state of `mbox(Aux)` and the productions table after each letter is read:

  Letter	`mbox(Aux)`	Production Table	`mbox(Hash)`
  a	a	`{}`	`{}`
  b	ab	`{}`	`{ab}`
  b	abb	`{}`	`{ab, \b\b}`
  a	abba	`{}`	`{ab, \b\b, ba}`
  b	abbab becomes AbA	`{A->ab}`	`{Ab, bA}`
  a	AbAa	`{A->ab}`	`{Ab, bA, Aa}`
  b	AbAab becomes AbAA	`{A->ab` marked}	`{Ab, bA, \A\A}`
  b	AbAAb becomes BAB	`{A->ab` marked, `B->Ab}`	`{\B\A, \A\B}`
  a	BABa	`{A->ab` marked,`B->Ab}`	`{\B\A, \A\B, \B\a}`
  b	BABab becomes BABA becomes CC	`{A->ab` marked, `B->Ab`, `C->BA}`	`{\C\C}`
  b	CCb	`{A->ab` marked, `B->Ab`, `C->BA}`	`{\C\C, \C\b}`
  a	CCba	`{A->ab` marked, `B->Ab`, `C->BA}`	`{\C\C, \C\b, ba}`
  b	CCbab becomes CCbA	`{A->ab` marked, `B->Ab`, `C->BA}`	`{\C\C, \C\b, \b\A}`
  b	CCbAb becomes CCbB	`{A->ab` marked, `B->Ab` marked, `C->BA}`	`{\C\C, \C\B}`

• We output the grammar `{A->ab`, `B->Ab`, `C->BA`, `S->\C\Cb\B}`
• As a crude, not too efficient example of how we could encode this... If we used `R` to denote the start of a rule, # followed by a sequence of digits for `n` to denote the `n` variable, and let `t` denote the terminal `t` unless `t` is `R`, '#', '\', or a digit in which case we use `\\t` is used to denote `t`, then the string `\a\b\b\a\b\a\b\b\a\b\b\a\b\b` might be compressed as RabR#0bR#1#0R#2#2b#1.
• Note we don't have to list the left hand side variable for a rule, because each variable appears at most once on the left hand side and we can count `R`'s to figure out which rule we are.
• It works better on longer strings. For example, a string of 64 `a`'s might be encoded as RaaR#0#0R#1#1R#2#2R#3#3R#4#4, a string of length 28.

Closure Properties of CFL

CFL are closed under union

Proof idea: Let `G` and `H` with start symbols `S` and `T` respectively, be two CFGs for the CFL's we want to take the union of. Make a new grammar with the same alphabet, with the union of the two grammars productions (after renaming) together with the new rules `S'->S|T` where `S'` is the new start variable.

CFLs are closed under intersection by regular languages

Proof idea: use the cartesian product construction on a PDA for the CFL together with a DFA for the regular language.

CFLs are not closed under intersection or complementation

Proof: The languages `{a^nb^nc^m | n, m ge 0}` and `{a^mb^nc^n | n, m ge0}` are both context-free. Their intersection is `{a^nb^nc^n | n ge 0}` which is not by the pumping lemma. CFLs are not closed under complementation as using deMorgan rules, intersection can be defined from union and complementation.

Algorithms for CFLs

• We have already given an algorithm (CYK) for checking if a string is in a CFLs.
• We have also given algorithms for checking for useless variables in grammars, as well as, for eliminating `epsilon`-rules from grammars.
• Here are a couple of more things we have CFG algorithms for:

There is an algorithm, which given a grammar `G` written down formally as a 4-tuple, can decide whether or not `L(G)` is empty.

To do this, the algorithm check if the start variable is useless. If it is we know the language is empty; otherwise it is not.

There is an algorithm to check given G whether or not L(G) is infinite.

To do this, we first eliminate `epsilon`-rules, unit-productions, and useless symbols from `G`. We then construct a graph where `(A,B)` is an edge for two variables `A`, `B` in the graph iff `A->xBy` for some production in `G`. If there is a cycle in this graph then `C=>^star uCv` for some variable `C` in the original grammar. As there are no useless symbols in this grammar, we must have `C=>^star z`, for some string `z` of terminals. Hence, also, `S=>^star sCt =>^star szt, S=>^star sCt =>^star suCvt =>^star suzvt`, etc. Thus, one can argue there is a cycle in the graph iff `G'`s grammar is infinite.