Introduction to Algorithms, 2020-1
Week 11: Notes

Some of this week's topics are covered in Problem Solving with Algorithms:

heapsort

We’ve already seen several sorting algorithms in this course: bubble sort, insertion sort, merge sort and quicksort. We can use a binary heap to implement another sorting algorithm: heapsort. The essence of heapsort is simple: given a array of numbers to sort, we put them into a heap, and then we pull out the numbers one by one in sorted order.

Let’s examine these steps in a bit more detail. For heapsort, we want a max-heap, which lets us remove the largest element by calling a method removeLargest. Given an array of N values, we first need to heapify it, i.e. turn it into a heap. We could simply perform N insert operations at a total cost of O(N log N), but actually there is a way to heapify N values in time O(N); see the section Binary Heap implementation in the Problem Solving text for details.

Once we have heapified the array, sorting it is easy: we repeatedly call removeLargest to remove the largest element, which we place at the end of the array:

def heapsort(a):
    heap = BinaryHeap()
    heap.buildHeap(a)

    for i in range(len(a) – 1, 0, -1):
        a[i] = heap.removeLargest()

The total time for heapsort to sort N values is the time to heapify the array plus the time to remove each of the N values from the heap in turn. This is

O(N) + O(N log N) = O(N log N)

Thus heapsort has the same asymptotic running time as merge sort and quicksort. Some advantages of heapsort are that it runs on an array in place (unlike merge sort) and that it is guaranteed to run in time O(N log N) for any input (unlike quicksort).

representing graphs

A graph consists of a set of vertices (sometimes called nodes in computer science) and edges. Each edge connects two vertices. In a undirected graph, edges have no direction: two vertices are either connected by an edge or they are not. In a directed graph, each edge has a direction: it points from one vertex to another.

Graphs are fundamental in computer science. Many problems can be expressed in terms of graphs, and we can answer lots of interesting questions using various graph algorithms.

We can represent graphs in a computer program in either of two ways. Suppose that a graph has N vertices numbered 0 through (N - 1). We can represent the graph using adjacency-matrix representation as a two-dimensional array A of booleans, with dimensions N x N. In this representation, A[i][j] is true if and only there is an edge from vertex i to vertex j. (If the graph is undirected, then A[i][j] = A[j][i] for all i and j, and so we may optionally save space by storing only the matrix elements above the main diagonal, i.e. elements for which i < j.)

In Python, we may store a graph in adjacency-matrix representation using a list of lists of booleans, which is essentially a two-dimensional array.

Alternatively, we can use adjacency-list representation, in which for each vertex u we store a list of its adjacent vertices.

If vertices are numbered 0 through (N - 1), then in Python we can store a graph in adjacency-list representation using a list of lists of integers. For a graph g, g[i] is a list of the integer ids of its adjacent vertices. Or, if vertices have identifiers such as strings, then we can store a graph in adjacency-list representation using a dictionary that maps each vertex id to a list of adjacent vertex ids.

When we store an undirected graph in adjacency-list representation, then if there is an edge from u to v we store u in v’s adjacency list and also store v in u’s adjacency list.

Adjacency-matrix representation is more compact if a graph is dense. It allows us to immediately tell whether two vertices are connected, but it may take O(V) time to enumerate a given vertex’s edges. On the other hand, adjacency-list representation is more compact for sparse graphs. With this representation, if a vertex has e edges then we can enumerate them in time O(e), but determining whether two vertices are connected may take time O(e), where e is the number of edges of either vertex. Thus, the running time of some algorithms may differ depending on which representation we use.

In this course we will usually represent graphs using adjacency-list representation.