Lecture 5: Notes

open arrays

A function or procedure parameter can have an array type with no bounds:

function sum(a: array of integer): integer;

This looks like a dynamic array type, but in this context this is an open array parameter. You can pass either a static or dynamic array to a function that expects an open array.

Just like dynamic arrays, open arrays are always indexed starting from 0, even if their source array has a different indexing base. For example:

procedure first(a: array of integer);
begin
  writeln('low = ', low(a), ', high = ', high(a), ', first = ', a[0]);
end;

var
  a: array[1..5] of integer = (2, 4, 6, 8, 10);

begin
  first(a);
  ...

This program will print

low = 0, high = 4, first = 2

You may pass a partial array to an open array parameter. For example, in the program above we could call

first(a[3..5]);

This will print

low = 0, high = 2, first = 6

const parameters

A function cannot modify a parameter that is preceded with const. This means that Pascal does not need to make a local copy of the value that is passed. In particular, this makes passing an array much more efficient, since internally it can be passed by reference.

For example:

function sum(const a: array of integer): integer;

var
  s: integer = 0;
  v: integer;
begin
  for v in a do
    s := s + v;
  sum := v;
end;

recursive array sum

The sum function below computes the sum of an array, recursively. It uses an open array parameter, and passes a partial array in the recursive call to itself.

function sum(const a: array of integer): integer;
begin
  if length(a) = 1 then
    sum := a[0]
  else sum += a[0] + sum(a[1 .. high(a)]);
end;

var
  a: array[1..3] of integer = (2, 4, 6);

begin
  writeln(sum(a));
end.

Whenever we write a recursive function, there is a base case and a recursive case.

The base case is a problem instance that is so small that we can solve it trivially. In the function above, the base case is when length(a) = 1.

In the recursive case, our function calls itself recursively, passing it a smaller instance of the problem we are trying to solve. The it uses the return value from the recursive call to construct a value that it itself can return. In the recursive case in this example, we call sum(a[1 .. high(a)], passing an array that is one element smaller than the array we received.

running time

If we did not specify const in the function signature above, then the function’s time complexity would follow this recurrence:

T(n) = T(n–1) + O(n)

That’s because Pascal would copy the partial array passed on each recursive call, since by default the callee receives a local copy. That partial array has n – 1 elements and the time to copy it is proportional to its length, so the copy takes O(n) time.

The solution to the recurrence above is T(n) = O(n²). One way to see that is to consider the related recurrence

T(n) = T(n - 1) + n

We can solve this in closed form:

T(n) = n + (n–1) + (n–2) + … + 2 + 1 = n (n + 1) / 2 = O(n²).

On the other hand, since we did specify const above, Pascal can pass the partial array directly without copying it. So the recurrence is

T(n) = T(n–1) + O(1)

with solution

T(n) = O(n)

So the function runs in linear time, just like an iterative version of the same function would.

out parameters

A parameter preceded with out lets a function return a value by setting a variable that is passed by reference. This is similar to var, but does not let the function receive a value from the caller. For example:

// Return the first and last character of a string
function firstAndList(s: string; out first: char; out last: char);
begin
  first := s[1];
  last := s[length(s)];
end;

out parameters are available only if you enable Delphi mode.

maximum difference, recursively

Suppose we want to write a function to find the maximum difference between any two elements of an array of integers. A poor algorithm to do this would compare every pair of integers, and would run in O(n²) . A better way is to pass over the array, looking for the highest and lowest value. Then we can simply subtract.

Here is a program to do this, using out parameters to return the highest and lowest value from a recursive function:

{$mode delphi}

uses math;

procedure hiLo(const a: array of integer; out hi, lo: integer);
begin
  if length(a) = 1 then  // base case
    begin
      hi := a[0];
      lo := a[0];
    end
  else  // recursive case
    begin
      hiLo(a[1 .. high(a)], hi, lo);
      hi := max(a[0], hi);
      lo := min(a[0], lo);
    end;
end;

function maxDiff(const a: array of integer): integer;
var
  hi, lo: integer;
begin
  hiLo(a, hi, lo);
  maxDiff := hi - lo;
end;

var
  a: array[1..5] of integer = (8, -5, 4, 12, -2);
begin
  writeln(maxDiff(a));
end.

Tower of Hanoi

The Tower of Hanoi is a well-known puzzle that looks like this:

The puzzle has 3 pegs and a number of disks of various sizes. The player may move disks from peg to peg, but a larger disk may never rest atop a smaller one. Traditionally all disks begin on the leftmost peg, and the goal is to move them to the rightmost.

Supposedly in a temple in the city of Hanoi there is a real-life version of this puzzle with 3 rods and 64 golden disks. The monks there move one disk each second from one rod to another. When they finally succeed in moving all the disks to their destination, the world will end.

The world has not yet ended, so we can write a program that solves a version of this puzzle with a smaller number of disks. We want our program to print output like this:

move disk 1 from 1 to 2
move disk 2 from 1 to 3
move disk 1 from 2 to 3
move disk 3 from 1 to 2
…

To solve this puzzle, the key insight is that a simple recursive algorithm will do the trick. To move a tower of disks 1 through N from peg A to peg B, we can do the following:

Move the tower of disks 1 through N-1 from A to C.
Move disk N from A to B.
Move the tower of disks 1 through N-1 from C to B.

The program below implements this algorithm:

procedure move(n: integer; fromPeg: integer; toPeg: integer);
begin
  if n = 0 then exit;
  move(n - 1, fromPeg, 6 - fromPeg - toPeg);
  writeln('move disk ', n, ' from ', fromPeg, ' to ', toPeg);
  move(n - 1, 6 - fromPeg - toPeg, toPeg);
end;

var
  n: integer;

begin
  readln(n);
  move(n, 1, 3);
end.

Note that the body of this recursive function calls itself twice. This is the first recursive function we've seen that calls itself more than once.

In general, if the code in a recursive function calls itself only once, then it is usually straightforward to implement its behavior using iteration instead of recursion. But if it makes multiple recursive calls as in this instance, usually it would be difficult to implement the same functionality iteratively, since the recursive calls occur in a non-linear branching pattern. It is not so easy to solve the Tower of Hanoi puzzle without recursion.

We can compute the exact number of moves required to solve the puzzle using the algorithm above. If M(n) is the number of moves to move a tower of height n, then we have the recurrence

M(n) = 2 ⋅ M(n–1) + 1

The solution to this recurrence is, exactly,

M(n) = 2ⁿ – 1

Similarly, the running time of our program above follows the recurrence

T(n) = 2 ⋅ T(n–1) + O(1)

And the program runs in time T(n) = O(2ⁿ).

It will take 2⁶⁴ - 1 seconds for the monks in Hanoi to move the golden disks to their destination tower. That is far more than the number of seconds that our universe has existed.

visual Hanoi

Here is a program that animates the Tower of Hanoi using text-mode (CRT) graphics.

visual_hanoi.pas

abcd x N

Let's use recursion to write a procedure that prints out all strings of length N containing only characters from the set {'a', 'b', 'c', 'd'}. We will denote this set of strings by S_n.

As always, with recursion we need a base case and a recursive case. The base case is easy: if N = 0, then there is only one such string, namely the empty string. In other words, S₀ = { ∅ }.

For the recursive case, notice that for n > 0, every string in S_n consists of one of the characters {'a', 'b', 'c', 'd'} followed by a string in S_n-1. In other words,

S_n = { c + s | c ∈ { 'a', 'b', 'c', 'd'} ∧ s ∈ S_n-1 }

One possible approach would be to write a recursive function that returns all the strings in S_n:

type 
  stringArray = array of string;

function abcd(n: integer): stringArray;
var 
  a: stringArray;
  c: char;
  s: string;
begin
  if n = 0 then
    begin
      setLength(a, 1);
      a[0] := '';
    end
  else
    begin
      setLength(a, 0);
      for c := 'a' to 'd' do
        for s in abcd(n - 1) do
          begin
            setLength(a, length(a) + 1);
            a[length(a) - 1] := c + s;
          end;
    end;
  abcd := a;
end;

This function directly imitates the definition of S_n above. It works, but there are two problems with this approach:

It is highly inefficient, since it constructs a new array of strings to return at each recursive call. The size of the largest of these arrays and the number of arrays created are both exponential in N.
It takes some effort to construct these arrays, with several calls to setLength in the function above.

So (in Pascal at least) this approach is generally not recommended. Instead, since we only want to print out the strings, we will write a procedure rather than a function, returning nothing at all. We will add a prefix argument that accumulates strings as recursive calls are made. And we will print out each string in the base case. Here is how this looks:

// Write all n-character strings with characters 'a'..'d', prefixing each with (prefix)
procedure abcd(prefix: string; n: integer);
var 
  c: char;
begin
  if n = 0 then writeln(prefix)
  else
    for c := 'a' to 'd' do
      abcd(prefix + c, n - 1);
end;

This procedure is much shorter and far more efficient. When we call it, we will provide an initial empty prefix:

abcd('', 3);

Symmetrically, you can use a suffix argument to accumulate a string in reverse order as recursive calls are made. Here is the procedure above, using a suffix argument instead:

// Write all n-character strings with characters 'a'..'d', suffixing each with (suffix)
procedure abcd(n: integer; suffix: string);
var 
  c: char;
begin
  if n = 0 then writeln(suffix)
  else
    for c := 'a' to 'd' do
      abcd(n - 1, c + suffix);
end;

In this instance either a prefix or suffix works, since we can easily build the string in either direction. Sometimes one approach might work better than the other, and occasionally both a prefix and suffix might be useful.

printing string permutations

Let's now write a procedure to write all permutations of a string s, which we will denote by P_s. We can do this recursively based on these observations:

In the base case, s is the empty string ''. Then P contains only the empty string, i.e. P = { '' }.
In the recursive case where length(s) > 0, each permutation consists of some character c ∈ s followed by a permutation of (s – c), which is the string that results from removing c from s. In other words,

P_s = { c + t | c ∈ s ∧ t ∈ P_{s – c} }

The code below implements this, using a prefix argument to write each accumulated string in the base case. Note that the call to stuffString returns a copy of s with s[i] removed.

uses strutils;

// Write all permutations of s, preceded by the given prefix.
procedure permute(prefix: string; s: string);
var
  i: integer;
begin
  if length(s) = 0 then writeln(prefix)
  else
    for i := 1 to length(s) do
      permute(prefix + s[i], stuffString(s, i, 1, ''));
end;

The running time of permute follows the recurrence

T(n) = n ⋅ T(n–1) + O(n²)

where n = length(s). That’s because there are n recursive calls, each of which reduces the string length by 1, and because there are n calls to stuffString, each of which takes time O(n).

The solution to the recurrence above will be at least as large as O(n!), since the simpler recurrence

T(n) = n ⋅ T(n-1)

is O(n!). In fact, I believe that the first recurrence above is also O(n!), because the n² term is completely dominated by the factorial growth (though I would need to think more about how to prove that). In any case, there are n! permutations and each one of them takes time O(n) to print out, so the entire program can not run in time less than O(n ⋅ n!).

(Generally speaking, if a program runs in exponential or factorial time then we are not so interested in whether there is an additional factor of n, since practically speaking n must always be quite small to be able to run the program in any reasonable amount of time.)

Up to N

Let's now write a procedure that takes an integer N and writes out all increasing sequences of integers that begin with 1 and end with N. For example, if N = 5 the result might look like this:

At first, we might hope to write a recursive procedure with this signature, following the pattern in the examples above:

procedure upToN(prefix: string; n: integer);

Unfortunately this will not work directly. The problem is that a recursive call to this procedure passing (n - 1) will not result in a set of strings that we can usefully transform into a solution for the case (n).

To solve this, then, we must transform and/or generalize the problem so that it can be solved recursively. Once we have done so, we can use the generalized solution to solve our original problem. This is a tactic that we will commonly need to employ.

There are several ways to generalize this problem to be recursively solvable. Here is one possible approach.

Consider the sequences above, removing the 1s and 5s that appear at the beginning and end of each sequence.

These are ascending sequences that contain some of the numbers 2 .. 4.

We can generate these sequences recursively. Let S_i,j be the set of ascending sequences of integers k, where i ≤ k ≤ j. Then we have a recursive pattern:

If j < i, then there are no numbers i ≤ k ≤ j, so S_i,j contains only the empty sequence. This is the base case.
Otherwise, we can construct S_{i, j} recursively from S_{i+1, j}. Specifically, for every sequence s in S_i+1,
j we can either prepend the element i to s, or we can decline to prepend i. In either case we have a sequence in S_{i, j}. In other words, if the symbol '::' denotes prepending an element to a sequence then

S_{i, j} = { i :: s | s ∈ S_{i+1, j} } ∪ S_i+1,_j

Following this idea, here is a recursive procedure that will print out these sequences:

procedure iToJ(prefix: string; i: integer; j: integer);
begin
  if j < i then writeln(prefix)    // base case
  else
    begin
      iToJ(prefix + intToStr(i) + ' ', i + 1, j);  // prepend i
      iToJ(prefix, i + 1, j);                      // do not prepend i
    end;
end;

Now let's modify this procedure to solve the problem as originally stated, in which case the sequence should always contain both 1 and N. Beginning with 1 is easy: we can simply provide the number 1 as the original prefix to which all strings will be appended. To ensure that each sequence contains N, we can modify the procedure so that the base case will print the number N. The final version looks like this:

procedure iToN(prefix: string; i: integer; n: integer);
begin
  if i = n then writeln(prefix, n)
  else
    begin
      iToN(prefix + intToStr(i) + ' ', i + 1, n);  // prepend i
      iToN(prefix, i + 1, n);                      // do not prepend i
    end;
end;

procedure upToN(n: integer);
begin
  iToN('1 ', 2, n);
end;

Now a call to upToN(5) will yield the output at the top of this section.

products of integers

Let's look at one more recursive example. We'd like to write a program that reads an integer N and writes out all the distinct sets of integers ≤ N whose product is N. For example, if N = 18 then the output should be

We can make a first attempt along the following lines. If j divides N, then we can recursively determine the sets of integers whose product is (N / j). Each such set together with j is a set that multiplies to N. Here is this attempt:

procedure products(prefix: string; n: integer);
var
  i: integer;
begin
  if n = 1 then
    writeln(prefix)
  else
    for i := n downto 2 do
      if n mod i = 0 then
        products(prefix + intToStr(i) + ' ', n div i);
end;

Now if we call products('', 18) we get the following output:

All of these sets do multiply to 18. But we have a problem: many sets appear multiple times in the list. We only want to print out each distinct set once.

We can ensure that each set is distinct by allowing only decreasing sequences to appear in our list. That way will print each set only once, in decreasing order. One approach would be to keep our existing recursive structure and just adjust the base case to print only decreasing sequences. But that would be awkward and inefficient.

There is a better way: we can force each sequence to decrease by generalizing our recursive procedure. Suppose that j divides N. Then when we make a recursive call to determine the sequences of integers whose product is N / j, we would like this to generate only sequences containing integers that are all less than or equal to j. That way, when we prepend j to any such sequence we will still have a decreasing sequence.

We can achieve this by adding a parameter max: integer that constraints the integers to be considered. The new version of our procedure looks like this:

procedure products(prefix: string; n: integer; max: integer);
var
  i: integer;
begin
  if n = 1 then
    writeln(prefix)
  else
    for i := max downto 2 do
      if n mod i = 0 then
        products(prefix + intToStr(i) + ' ', n div i, i);
end;

Now if we call products('', 18, 18) we get the output at the top of this section.

recursive binary search

In lecture 3 we saw an implementation of the binary search algorithm, which searches a sorted array to see whether a particular element (called the key) is present. Here it is again:

function binary_search(a: array of integer; key: integer): boolean;
  var
    lo, hi, mid: integer;
  begin
    lo := low(a);
    hi := high(a);
    while lo <= hi do
      begin
        mid := (lo + hi) div 2;
        if a[mid] = key then
          exit(true);
        if a[mid] < key then
          lo := mid + 1
        else  // a[mid] > key
          hi := mid - 1;
      end;
    exit(false);
  end;

(You may wish to review the discussion of this function’s loop invariant from the lecture 3 notes.)

Let n = length(a). For each loop iteration, let s be the number of elements still under consideration, i.e. the number of values j for which lo ≤ j ≤ hi. Then initially s = n, and s effectively halves with each iteration. The loop ends when there are no more values to consider, i.e. when s < 1. So there will be at most log₂(n) iterations, each of which takes constant time. The loop runs in time O(log₂ n) = O(log n).

Let’s rewrite this function recursively:

function binary_search(const a: array of integer; key: integer): boolean;

var 
  mid: integer;

begin
  if length(a) = 1 then exit(a[0] = key);

  mid := length(a) div 2;
  if a[mid] = key then exit(true);

  if a[mid] < key then
    exit(binary_search(a[mid + 1 .. high(a)], key))
  else
    exit(binary_search(a[0 .. mid - 1], key));
end;

We can describe the recursive function’s worst-case running time using a recurrence:

T(n) = T(n / 2) + O(1)

This implies that T(n) = O(log n), just as with the iterative version.