Some of this week's topics are covered in Introduction to Algorithms:
8. Sorting in Linear Time
8.1 Lower bounds for sorting
8.2 Counting sort
8.3 Radix sort
Here are some additional notes.
We can solve many problems by using recursion to exhaustively explore an exponential space. Sometimes this is called a combinatorial search.
As a first example of this technique, let's write a function that takes an integer n and returns a list of all strings of length n containing only characters from the set {'a', 'b', 'c'}. For example, if n = 2 then the list will contain these strings:
aa ab ac ba bb bc ca cb cc
If n = 3, the list will begin like this:
aaa aab aac aba abb abc aca acb acc baa bab bac …
How many strings will there be in the output list? There are 3 possible choices for the first letter, 3 for the second letter, and so on for each of the n letters. Thus the total number of possible strings is 3n.
Any algorithm to generate these strings will surely take exponential time, since the length of the output is exponential in n. This is a sign that we will probably need to use recursion to solve this problem, since a set of loops without recursion will generally run in polynomial, not exponential, time.
Our first step in solving a problem like this is to notice its recursive structure. Specifically, consider the list of output strings above for n = 3. If we look at only the first two characters of each string, we see the strings aa, ab, ac, ba and so on, which is the list of strings for n = 2. In other words, each string for n = 3 consists of some string generated by the same function with n = 2, followed by an 'a', 'b', or 'c' character. This observation will allow us to write this function recursively.
Let's choose a base case that is as small as possible, i.e. n = 0. In this case should the function return an empty list, or a list with a single element? For a clue to the answer, recall that there should be 3n output strings. Of course 30 = 1, so this suggests that we should return a list of length 1. Indeed, there is exactly one string of length 0 containing only characters from the set {'a', 'b', 'c'}, namely the empty string.
Here is a recursive function that solves this problem:
def strings(n):
if n == 0:
return ['']
l = []
for s in strings(n - 1):
for c in 'abc':
l.append(s + c)
return lAlternatively, we may use a list comprehension to achieve the same result:
def strings(n):
if n == 0:
return ['']
return [s + c for s in strings(n - 1) for c in 'abc']Suppose that we wish to print all the strings returned by one of these functions. We may write
for s in strings(n): print(s)
However it's a bit unfortunate that with either function above all the strings will be in memory at the same time, even if our goal is only to print each one in turn. That's because each of the functions above generates an entire list before returning it.
In Programming 1, we recently learned about generator functions and generator comprehensions, which give us an elegant way to avoid this problem. We can rewrite the strings() function using a generator function:
def strings(n):
if n == 0:
yield ''
else:
for s in strings(n - 1):
for c in 'abc':
yield s + cOr using a generator comprehension (which looks a lot like a list comprehension):
def strings(n):
if n == 0:
return ['']
return (s + c for s in strings(n - 1) for c in 'abc')
With either of these implementations, this for loop to
print the strings will use only a constant amount of memory:
for s in strings(n): print(s)
As we will see, we can use recursion to solve many similar combinatorial problems, such as finding all permutations or combinations of a list, all subsets of a set, and so on. For any such problem, we can use any of these four techniques in Python for writing our recursive function:
appending each value to an array
a list comprehension
a generator function
a generator comprehension
As a second example of a combinatorial search, let's write a function that generates a list of all permutations of a string.
For example, if the input string is 'abc', the output will include these values:
abc acb bac bca cab cba
If the input string is 'abcd', the output will begin with these values:
abcd abdc acbd acdb adbc adcb bacd badc bcad …
If the length of the input string is N, then there will be N! output values, since there are N! permutations of a list of length N.
As with the previous problem, we need to find the recursive structure of the problem. Look at the permutations of 'abcd' above. Notice that the first 6 permutations consist of the character 'a' plus some permutation of 'bcd'. The next 6 permutations will consist of the character 'b' plus some permutation of 'acd', and so on. That suggests a recursive solution to the problem. To find all permutations of a string s, we must consider each character of s in turn. For each character c, we will build the string t, which we get by deleting c from s, and then generate output strings consisting of c prepended to each permutation of t. Of course, we can compute the permutations of t recursively, since t is shorter than s.
We are now in a position
to write the recursive function permutations.
Our base case is as small as possible, namely when s equals the empty
string. In that case, N = len(s) = 0, so there should be 0! = 1
element in the output list. And, indeed, there is exactly one
permutation of the empty string: it is, of course, the empty string
itself.
Here is the function:
def permutations(s):
if s == '':
return ['']
l = []
for i in range(len(s)):
t = s[:i] + s[(i + 1):] # delete character s[i]
for u in permutations(t):
l.append(s[i] + u)
return lAs we discussed in the previous section, we could alternatively write this function using a list or generator comprehension, or a generator function.