Some topics from this lecture are also covered in the Introduction to Algorithms textbook:
22.1 Representations of graphs
22.2 Breadth-first search
22.3 Depth-first search
24.3 Dijkstra’s algorithm
A graph consists of a set of vertices (often called nodes in computer science) and edges. Each edge connects two vertices. In a undirected graph, edges have no direction: two vertices are either connected by an edge or they are not. In a directed graph, each edge has a direction: it points from one vertex to another.
Graphs are fundamental in computer science. Many problems can be expressed in terms of graphs, and we can answer lots of interesting questions using various graph algorithms.
We can represent graphs in a computer program in either of two ways. Suppose that a graph has V vertices numbered 0 through V - 1. We can represent the graph using adjacency-matrix representation as a two-dimensional matrix A of booleans, with dimensions V x V. In this matrix, A[i, j] is true if and only there is an edge from vertex i to vertex j. If the graph is undirected, then A[i, j] = A[j, i], and so we may optionally save space by storing only the matrix elements above the main diagonal, i.e. elements for which i < j.
Here is a Pascal data type for storing a graph in adjacency-matrix representation, where each vertex is named with a string:
type
graph = record
name: array of string;
edges: array of array of boolean;
end;Or we can use adjacency-list representation, in which for each vertex u we store a list of its adjacent vertices, i.e. all vertices v such that there is an edge from u to v. Here is a Pascal type for a graph in adjacency-list representation:
node = record
name: string;
edges: array of integer;
end;
graph = array of node;When we store an undirected graph in adjacency-list representation, then if there is an edge from u to v we store u in v’s adjacency list and also store v in u’s adjacency list.
Adjacency-matrix representation is more compact if a graph is dense. It allows us to immediately tell whether two vertices are connected, but it may take O(V) time to enumerate a given vertex’s edges. On the other hand, adjacency-list representation is more compact for sparse graphs. With this representation, if a vertex has e edges then we can enumerate them in time O(e), but determining whether two vertices are connected may take time O(e), where e is the number of edges of either vertex. Thus, the asymptotic running time of some algorithms may differ depending on the representation we use.
In this course we will generally represent graphs using adjacency-list representation. Each vertex will have a unique integer ID.
The unit below can read a graph from a text file. The code is straightforward.
{$mode delphi}
unit graphs;
interface
type
// A graph using adjacency-list representation
node = record
name: string;
edges: array of integer;
end;
graph = array of node;
// Read an undirected graph from a text file.
// Each line of the file contains a set of words, e.g.
// dog cat horse cow
// The node named by the first word ("dog") will be connected to all the others:
// dog -- cat, dog -- horse, dog -- cow.
function readGraph(filename: string): graph;
implementation
uses character;
function readWord(var input: text): string;
var
s: string = '';
c: char;
begin
while not eoln(input) do
begin
read(input, c);
if not isletter(c) then break;
s := s + c;
end;
readWord := s;
end;
// Find a node by name, or create it if it doesn't already exist
function findNode(var g: graph; name: string): integer;
var
i: integer;
begin
for i := 0 to high(g) do
if g[i].name = name then exit(i);
setLength(g, length(g) + 1);
i := length(g) - 1;
g[i].name := name;
findNode := i;
end;
// Create an directed edge between two graph nodes
procedure connect(var g: graph; i, j: integer);
begin
setLength(g[i].edges, length(g[i].edges) + 1);
g[i].edges[length(g[i].edges) - 1] := j;
end;
function readGraph(filename: string): graph;
var
input: text;
g: graph;
s: string;
i, j: integer;
begin
assign(input, filename);
reset(input);
while not eof(input) do
begin
s := readWord(input);
i := findNode(g, s);
while not eoln(input) do
begin
s := readWord(input);
j := findNode(g, s);
connect(g, i, j);
connect(g, j, i);
end;
readln(input);
end;
close(input);
readGraph := g;
end;
end.Here is an undirected graph where each node is a country in the European Union. There is an edge between every two neighboring countries, and also between any two countries connected by a car ferry (in this case appearing as a dashed line). Thus, two countries are connected by an edge if it is possible to drive from one to the other. We will use this graph to illustrate various algorithms in the following sections.
In this course we will study several algorithms that can search a graph, i.e. explore it by visiting its nodes in some order.
All of these algorithms will have some common elements. As they run, at each moment in time the graph’s vertices fall into three sets: explored vertices, vertices on the frontier, and undiscovered vertices. At the beginning all vertices are undiscovered. A vertex V joins the frontier when the algorithm first sees it. After the algorithm has followed all of V’s edges, V becomes explored. By convention, when we draw pictures illustrating graph algorithms we draw vertices as either black (explored), gray (on the frontier) or white (undiscovered).
Note that these algorithms will generally not explicitly mark vertices as belonging to one of these three states. Instead, these states are concepts that help us understand how these algorithms work.
A depth-first search is like exploring a maze by walking through it. Each time we hit a dead end, we walk back to the previous intersection, and try the next unexplored path from that intersection. If we have already explored all paths from the intersection, we walk back to the insersection before that, and so on.
Alternatively we can define a depth-first search using recursive terminology. To visit a node N, we enumerate the nodes adjacent to N and visit each of them (if there any) in turn, recursively.
It is easy to implement a depth-first search using a recursive function. In fact we have already done so in this course. For example, we wrote a function to add up all values in a binary tree, like this:
function sum(tree: ^node): integer; begin sum := sum(tree^.left) + tree^.i + sum(tree^.right); end;
This function visits all tree nodes using a depth-first tree search.
For a depth-first search on arbitrary graphs which may not be trees, we must avoid walking through a graph cycle in an infinite loop. We can do this by remembering which nodes we have already visited. When we first discover a vertex, we mark it as visited. Whenever we follow an edge, if it leads to a visited vertex then we ignore it. Both the black (explored) and gray (frontier) vertices are in the visited set.
Here is a picture of a depth-first search in progress on our Europe graph, starting from Austria:
As
the depth-first search progresses it generates a depth-first tree
that spans the original graph. If we like, we may store this tree in
memory as an product of the depth-first search. Here is a depth-first
tree for the Europe graph:
Here
is Pascal code that implements a depth-first search on an arbitrary
graph. It uses a nested procedure, a feature of Pascal that we
have not yet seen. A nested procedure or function may access local
variables in the containing procedure or function.
uses graphs, queue_array;
// depth-first search
procedure dfs(const g: graph; i: integer);
var
visited: array of boolean;
procedure visit(i: integer);
var
j: integer;
begin
visited[i] := true;
for j in g[i].edges do
if not visited[j] then
begin
writeln(g[i].name, ' -> ', g[j].name);
visit(j);
end;
end;
begin
setLength(visited, length(g));
visit(i);
end;When we implement depth-first search using a recursive procedure, at any moment all nodes on the call stack are gray. When the recursive call for particular node completes, that node becomes black.
A depth-first search has many uses. It will visit all reachable nodes in a graph, so we can easily determine whether a graph is connected using a depth-first search: we simply check whether a depth-first search has visited all of its nodes.
A depth-first search can also determine whether a graph is cyclic, i.e. contains any cycles. A graph is cyclic if and only if a depth-first search discovers any back edges, which are edges that lead to gray (frontier) nodes at the time they are discovered. (See Introduction to Algorithms, Lemma 22.11 for a proof of this fact.)
Depth-first search is also a useful building block for other graph algorithms: topological sorting, discovering strongly connnected components and so on. We may see some of these in Programming II.
A depth-first search does a constant amount of work for each vertex (making a recursive call) and for each edge (following the edge and checking whether the vertex it points to has been visited). So it runs in time O(V + E), where V and E are the numbers of vertices and edges in the graph.
Starting from some node N, a breadth-first search first visits nodes adjacent to N, i.e. nodes of distance 1 from N. It then visits nodes of distance 2, and so on.
We can easily implement a breadth-first search using a queue. Just like with depth-first graph search, we must remember all nodes that we have visited. We begin by adding the start node to the queue and marking it as visited. In a loop, we repeatedly remove nodes from the queue. Each time we remove an node, we mark all of its adjacent unvisited nodes as visited and add them to the queue. The algorithm terminates once the queue is empty, at which point we will have visited all reachable nodes.
The queue represents the frontier: all nodes in the queue are gray. When we remove a node from the queue, it turns black. Just like with depth-first graph search, the visited nodes are those which are black or gray.
As the algorithm runs, all nodes in the queue are at approximately the same distance from the start node. To be more precise, at any moment in time there is some value d such that the queue consists of some number of nodes at distance d followed by some number (possibly zero) of nodes at distance d + 1. As the algorithm dequeues each node of distance d, it adds all of its undiscovered adjacent nodes, which have distance d + 1. Once it has dequeued all nodes of distance d, it will begin dequeueing nodes of distance d + 1 and enqueueing nodes of distance d + 2. In this way it visits nodes in increasing order of distance from the start node.
Here is a breadth-first search in progress on our Europe graph, starting from Austria:
Just
like depth-first search, breadth-first search generates a tree which
spans the original graph. A breadth-first tree has a useful property:
it indicates the shortest path from the start node to every other
node in the graph. Here is a breadth-first tree for the Europe graph:
We
often choose to store a breadth-first tree as a directed graph in
memory with the arrows pointing in the other direction, i.e. toward
the start node. This is sometimes called an in-tree. In this
representation each node points toward its predecessor in the
breadth-first graph. Here is the previous breadth-first tree as an
in-tree:
Here
is a Pascal procedure that implements a breadth-first search on an
arbitrary graph:
// breadth-first search
procedure bfs(const g: graph; start: integer);
var
visited: array of boolean;
q: queue;
i, j: integer;
begin
setLength(visited, length(g));
init(q);
enqueue(q, start);
visited[start] := true;
while not isEmpty(q) do
begin
i := dequeue(q);
for j in g[i].edges do
if not visited[j] then
begin
writeln(g[j].name, ' -> ', g[i].name);
enqueue(q, j);
visited[j] := true;
end;
end;
end;Like a depth-first search, a breadth-first search does a constant amount of work for each vertex and edge, so it also runs in time O(V + E).
If we replace the queue in the preceding implementation with a stack then the procedure will actually perform a depth-first search instead!
We would often like to work with weighted graphs, in which each edge has a numeric weight. We can extend our adjacency-list representation to include edge weights as follows:
type
edge = record
target: integer;
weight: integer;
end;
node = record
name: string;
edges: array of edge;
end;
graph = array of node;Here is a Pascal unit that can read a weighted graph from a text file. The implementation is straightforward.
{$mode delphi}
unit weighted_graph;
interface
type
// A weighted graph with adjacency-list representation
edge = record
target: integer;
weight: integer;
end;
node = record
name: string;
edges: array of edge;
end;
graph = array of node;
// Read an undirected weighted graph from a text file.
// Each line of the file contains two node names followed by an integer weight, e.g.
// dog cat 7
function readGraph(filename: string): graph;
implementation
uses character;
function readWord(var input: text): string;
var
s: string = '';
c: char;
begin
while not eoln(input) do
begin
read(input, c);
if not isletter(c) then break;
s := s + c;
end;
readWord := s;
end;
// Find a node by name, or create it if it doesn't already exist
function findNode(var g: graph; name: string): integer;
var
i: integer;
begin
for i := 0 to high(g) do
if g[i].name = name then exit(i);
setLength(g, length(g) + 1);
i := length(g) - 1;
g[i].name := name;
findNode := i;
end;
// Create an directed edge between two graph nodes
procedure connect(var g: graph; i, j: integer; weight: integer);
var
n: integer;
begin
n := length(g[i].edges);
setLength(g[i].edges, n + 1);
g[i].edges[n].target := j;
g[i].edges[n].weight := weight;
end;
function readGraph(filename: string): graph;
var
input: text;
g: graph;
s: string;
i, j, weight: integer;
begin
assign(input, filename);
reset(input);
while not eof(input) do
begin
s := readWord(input);
i := findNode(g, s);
s := readWord(input);
j := findNode(g, s);
readln(input, weight);
connect(g, i, j, weight);
connect(g, j, i, weight);
end;
close(input);
readGraph := g;
end;
end.Here is a weighted graph in which nodes are capital cities of countries in the European Union. The edge weights in this graph represent driving distances in hours between these capital cities (possibly including ferry transport). We will use this graph as an example in the following section.
In a weighted graph we may define the length of a path as the sum of the weights of all edges along the path. With this definition in mind, we may wish to find the shortest path between two vertices. Dijkstra’s algorithm can achieve this as long as all edge weights are non-negative.
Dijkstra’s algorithm is somewhat similar to breadth-first
search. Like breadth-first search, Dijkstra’s algorithm starts at a
source node and moves outward, keeping a frontier of nodes that are
at approximately the same distance from the start node. But the
algorithms differ in the order in which they remove nodes from the
frontier. Breadth-first search uses a simple queue, removing nodes in
the same order in which they were added. Dijkstra’s algorithm, by
contrast, keeps an estimate of the shortest-path distance to
each node. At each step, it removes the node with the lowest
shortest-path estimate. Typically Dijkstra’s algorithm is
implemented using a priority queue, which has a removeSmallest
operation that is appropriate here.
Here is how the algorithm works in more detail. We begin by adding the start node to the priority queue and setting its distance estimate (i.e. priority) to zero. We now remove nodes from the queue in a loop. On each iteration we remove the node u with the lowest distance estimate d. We now consider each of u’s adjacent nodes v. Let w be the weight of the edge leading from u to v. If v is black (already explored), we ignore it. If v is white (undiscovered), we add it to the queue, setting its estimate to d + w. If v is gray (on the frontier, i.e. in the queue) and its estimate is higher than d + w, we lower the estimate to d + w. u has now been explored and turns black.
In a typical implementation of Dijkstra’s algorithm, we use a priority queue that holds the frontier nodes and also all the undiscovered nodes, which we give a distance estimate of infinity. This simplifies the implementation because we can treat frontier and undiscovered nodes in the same way: undiscovered simply have an infinite estimated cost since we don’t yet know any path to them. To see whether a node has already been explored, we simply check whether it is in still in the priority queue.
Each time Dijkstra’s algorithm removes a node u from the frontier with distance estimate d, the shortest path from the start node to u in fact has distance d. We can see this with an inductive argument. When the start node is removed from the frontier, it has distance estimate 0 which is trivially the shortest distance to the start node. Now suppose that all nodes already removed from the frontier (i.e. explored nodes) have correct shortest distances. Each node v currently on the frontier has an estimate e which is the minimum of all values d + w, where d is the distance estimate to some explored node u and w is the weight of an edge pointing from u to v. By the inductive hypothesis the distances d are actual shortest distances. So if a node v has estimate e, then e is the length of the shortest path from the start node to v that goes from an explored node directly to v, i.e. that does not pass through any other frontier nodes. Now let v1 be the frontier node with the smallest estimate e1. Any path from the start node to v1 that passes through any other frontier node must be longer than e1, because it will be at least as large as that node’s estimate. So the shortest path to v1 must pass from an explored node directly to v1, and e1 is the shortest such path. Thus the estimate e1 is optimal, completing the inductive step of the proof.
Here is a picture of Dijkstra’s algorithm in action on the European cities graph, with Athens as the start node:
Like
depth-first and breadth-first search, Dijkstra’s algorithm
constructs a spanning tree of the graph. The tree generates by
Dijkstra’s algorithm indicates the shortest path from the start
node to every other node in the graph. Here is the tree generated for
the European cities graph, again starting from Athens:
To
implement Dijkstra’s algorithm we need a priority queue. In the
last lecture we saw the following interface for a priority queue of
integers, and an efficient binary heap implementation of this
interface:
procedure init(var q: priority_queue); function isEmpty(q: priority_queue); procedure insert(var q: priority_queue; i: integer); function removeLargest(var q: priority_queue): integer;
For our implementation of Dijkstra’s algorithm we will need to modify this interface in three ways.
First, instead of removeLargest we will need a function removeSmallest, since we will want to find the frontier node with the smallest estimate.
Second, each element in the priority queue will now need to consist of both an integer value (which will be a graph node id) and an integer priority (which will be the distance estimate to that node). When we remove the element with the smallest priority, we need to know its value as well.
Third, we will need several new priority queue operations:
inQueue,
which tells us whether a given value is in the queue
getPriority,
which returns a given value’s current priority
decrease,
which decreases a value’s priorty. We will use this to update
distance estimates.
The enhanced priority queue interface looks like this:
// Initialize a queue, which will initally be empty. procedure init(var q: pqueue; n: integer); // Return true if the queue is empty. function isEmpty(q: pqueue): boolean; // Insert a value, which must not already be present in the queue. procedure insert(var q: pqueue; i: integer; priority: integer); // Remove the value with the smallest priority. function removeSmallest(var q: pqueue; out priority: integer): integer; // Return true if i is in the queue. function inQueue(var q: pqueue; i: integer): boolean; // Return i's priority. function getPriority(var q: pqueue; i: integer): integer; // Decrease i's priority to the given value. procedure decrease(var q: pqueue; i: integer; priority: integer);
Assuming that we have an implementation of this priority queue interface, we can implement Dijkstra’s algorithm in Pascal as follows:
{$mode delphi}
uses weighted_graph;
// Dijkstra's algorithm
procedure dijkstra(const g: graph; start: integer);
var
queue: pqueue;
pred: array of integer;
i, distance: integer;
e: edge;
begin
init(queue, length(g));
for i := 0 to high(g) do
insert(queue, i, MaxInt);
setLength(pred, length(g));
decrease(queue, start, 0);
while not isEmpty(queue) do
begin
i := removeSmallest(queue, { out } distance);
if i <> start then
writeln(g[pred[i]].name, ' -> ', g[i].name, ': ', distance);
for e in g[i].edges do
if inQueue(queue, e.target) and
(distance + e.weight < getPriority(queue, e.target)) then
begin
decrease(queue, e.target, distance + e.weight);
pred[e.target] := i;
end;
end;
end;One way to implement the extended priority queue described above is to use an array with N elements, where integer values stored in the queue can range from 0 to N – 1. The array maps each value to its priority. When we need to remove the value with smallest priority, we scan the entire array.
The implementation is straightforward and appears below. All
operations run in constant time except init and
removeSmallest, which take O(N).
How long will Dijkstra’s algorithm take to run with this
priority queue implementation? For each of V vertices we must call
removeSmallest, which takes O(V). And for each edge we must
potentially perform several priority queue operations (inQueue,
getPriority, decrease), all of which run in
constant time. The total time is O(V2 + E) = O(V2).
{$mode delphi}
{$r+}
unit priority_queue_array;
interface
const
None = -MaxInt;
// A priority queue that holds unique values from 0 to N - 1, each with
// an integer priority.
type
pqueue = record
a: array of integer; // each value's priority, or None if absent
count: integer;
end;
procedure init(var q: pqueue; n: integer);
function isEmpty(q: pqueue): boolean;
// Insert a value, which must not already be present in the queue.
procedure insert(var q: pqueue; i: integer; priority: integer);
// Remove the value with the smallest priority.
function removeSmallest(var q: pqueue; out priority: integer): integer;
// Return true if i is in the queue.
function inQueue(var q: pqueue; i: integer): boolean;
// Return i's priority.
function getPriority(var q: pqueue; i: integer): integer;
// Decrease i's priority to the given value.
procedure decrease(var q: pqueue; i: integer; priority: integer);
implementation
procedure error(s: string);
begin
writeln(s);
halt;
end;
procedure init(var q: pqueue; n: integer);
var
i: integer;
begin
setLength(q.a, n);
for i := 0 to n - 1 do
q.a[i] := None;
q.count := 0;
end;
function isEmpty(q: pqueue): boolean;
begin
exit(q.count = 0);
end;
procedure insert(var q: pqueue; i: integer; priority: integer);
begin
if q.a[i] <> None then error('value is already in queue');
q.a[i] := priority;
q.count := q.count + 1;
end;
function inQueue(var q: pqueue; i: integer): boolean;
begin
exit(q.a[i] <> None);
end;
function getPriority(var q: pqueue; i: integer): integer;
begin
exit(q.a[i]);
end;
procedure decrease(var q: pqueue; i: integer; priority: integer);
begin
if q.a[i] = None then error('value is not in queue');
if q.a[i] < priority then error('existing priority is lower');
q.a[i] := priority;
end;
function removeSmallest(var q: pqueue; out priority: integer): integer;
var
i: integer;
min: integer = MaxInt;
minIndex: integer = -1;
begin
for i := 0 to high(q.a) do
if (q.a[i] <> None) and (q.a[i] <= min) then
begin
minIndex := i;
min := q.a[i];
end;
removeSmallest := minIndex;
priority := q.a[minIndex];
q.a[minIndex] := None;
q.count := q.count - 1;
end;
end.It is not difficult to modify our existing binary heap
implementation to use removeSmallest instead of
removeLargest and to store both values and priorities.
Unfortunately supporting the additional operations is not as trivial.
We must keep an additional array that remembers the heap position of
each element. When we swap two heap elements in an up heap or down
heap operation, we must also swap the corresponding elements in the
array of heap positions. An implementation appears below. init,
isEmpty, inQueue and getPriority
will all run in constant time. removeSmallest and
decrease run in time O(log N), where there are N nodes
in the heap. insert will run in O(log N) on average
(individual calls may take longer if they need to expand the heap
array).
With this priority queue implementation, for each of V vertices
Dijkstra’s algorithm will call removeSmallest, which
takes O(log V) time. And for each edge it may call decrease,
which also takes O(log V). The total time is O((V + E) log V), which
is O(E log V) if at least a constant fraction of vertices are
reachable. Note that this may actually be asymptotically slower than
using the array-based priority queue if the graph is dense (i.e. E =
O(V2)).
{$mode delphi}
{$r+}
unit priority_queue_heap;
interface
// A priority queue that holds unique values from 0 to N - 1, each with
// an integer priority.
type
pqueue = record
count: integer;
heap: array of integer; // binary heap holding values minimized by priority
priority: array of integer; // each value's priority, or None if absent
position: array of integer; // each value's position in the heap array, or None if absent
end;
procedure init(var q: pqueue; n: integer);
function isEmpty(q: pqueue): boolean;
// Insert a value, which must not already be present in the queue.
procedure insert(var q: pqueue; i: integer; priority: integer);
// Remove the value with the smallest priority.
function removeSmallest(var q: pqueue; out priority: integer): integer;
// Return true if i is in the queue.
function inQueue(var q: pqueue; i: integer): boolean;
// Return i's priority.
function getPriority(var q: pqueue; i: integer): integer;
// Decrease i's priority to the given value.
procedure decrease(var q: pqueue; i: integer; priority: integer);
implementation
const
None = -MaxInt;
procedure error(s: string);
begin
writeln(s);
halt;
end;
procedure swap(var i, j: integer);
var
k: integer;
begin
k := i;
i := j;
j := k;
end;
function parent(i: integer): integer;
begin
parent := (i - 1) div 2;
end;
function leftChild(i: integer): integer;
begin
leftChild := 2 * i + 1;
end;
function rightChild(i: integer): integer;
begin
rightChild := 2 * i + 2;
end;
procedure init(var q: pqueue; n: integer);
var
i: integer;
begin
q.count := 0;
setLength(q.heap, 1);
setLength(q.priority, n);
setLength(q.position, n);
for i := 0 to n - 1 do
begin
q.priority[i] := None;
q.position[i] := None;
end;
end;
function isEmpty(q: pqueue): boolean;
begin
exit(q.count = 0);
end;
function inQueue(var q: pqueue; i: integer): boolean;
begin
exit(q.priority[i] <> None);
end;
function getPriority(var q: pqueue; i: integer): integer;
begin
exit(q.priority[i]);
end;
// Move the value at index i downward until its children are larger than it
procedure down(var q: pqueue; i: integer);
var
l, r, smallest: integer;
begin
l := leftChild(i);
r := rightChild(i);
smallest := i;
if (l < q.count) and (q.priority[q.heap[l]] < q.priority[q.heap[i]]) then
smallest := l;
if (r < q.count) and (q.priority[q.heap[r]] < q.priority[q.heap[smallest]]) then
smallest := r;
if smallest <> i then // some child is smaller
begin
swap(q.heap[i], q.heap[smallest]);
swap(q.position[q.heap[i]], q.position[q.heap[smallest]]);
down(q, smallest);
end;
end;
// Move the value at index i upward until its parent is smaller than it
procedure up(var q: pqueue; i: integer);
var
p: integer;
begin
if i = 0 then exit;
p := parent(i);
if q.priority[q.heap[i]] < q.priority[q.heap[p]] then // smaller than parent
begin
swap(q.heap[i], q.heap[p]);
swap(q.position[q.heap[i]], q.position[q.heap[p]]);
up(q, p);
end;
end;
procedure insert(var q: pqueue; i: integer; priority: integer);
begin
if q.priority[i] <> None then error('value is already in queue');
if q.count >= length(q.heap) then // array is full
setLength(q.heap, length(q.heap) * 2);
q.count := q.count + 1;
q.heap[q.count - 1] := i;
q.position[i] := q.count - 1;
q.priority[i] := priority;
up(q, q.count - 1);
end;
procedure decrease(var q: pqueue; i: integer; priority: integer);
begin
if q.priority[i] = None then error('value is not in queue');
if q.priority[i] < priority then error('existing priority is lower');
q.priority[i] := priority;
up(q, q.position[i]);
end;
function removeSmallest(var q: pqueue; out priority: integer): integer;
var
smallest: integer;
begin
smallest := q.heap[0];
removeSmallest := smallest;
priority := q.priority[smallest];
q.priority[smallest] := None;
q.position[smallest] := None;
q.heap[0] := q.heap[q.count - 1];
q.position[q.heap[0]] := 0;
q.count := q.count - 1;
down(q, 0);
end;
end.Above, we implemented several graph search algorithms (depth-first search, breadth-first search, Dijkstra’s algorithm) to run on concrete graphs stored as a set of nodes and edges in memory. We can use these same algorithms to search abstract graphs that form the space of potential solutions to problems we are trying to solve. This is a powerful and general technique. We illustrate this with a couple of examples below.
The 8 queens problem is a classic puzzle. Recall that in chess, a queen can move any number of squares horizontally, vertically or diagonally. Can we place 8 queens on an 8 x 8 chessboard such that none of them attack any other? More generally, what about N queens on an N x N chessboard?
This a constraint satisfaction problem: we want to find a variable assignment (in this case, the positions of the 8 queens) that satisfies a certain set of constraints (in this case, no queen may attack any other). This may not look like a graph problem, but we can transform it to one by considering an incremental formulation of the problem. Certainly no two queens can occupy the same column. We create a root node, representing an empty chessboard. We add 8 nodes at level 1, one for each possible position (r, 1) of a queen in column 1. Now, for each node (r, 1) we create edges to new nodes at level 2 for each possible position (s, 2) where a queen can be placed if there is already a queen at (r, 1). And so on:
Now
each node in this tree corresponds to a unique way of placing queens
on a chessboard such that none of them attack each other. If we can
place all 8 queens in this way, then there must be some node at level
8 in the graph. And by searching the graph we can look for it.
We can do this by performing a recursive depth-first search, just as with the concrete graph we saw in an earlier section. This abstract graph is a tree, so we need not check for visited nodes; we can simply write a recursive function that explores all these possible positions. A Pascal implementation appears below.
This is really not so different from recursive functions that we’ve written earlier in this course, such as when we printed all permutations of a string or found all the sets of integers that multiplied to a particular number. Here, however, we are searching for a solution in a problem space where it may not be obvious how many solutions exist or whether there even is a solution. So the running time of a program like this may be unknown until we try it. The number of ways to place N queens on a board (even if they may attack each other) certainly increases exponentially as a function of N. Fortunately, however, the program below can quickly solve the N queens problem for modest values of N such as N = 8 or even N = 20.
uses strutils;
function attack(r1, c1, r2, c2: integer): boolean;
begin
attack := (r1 = r2) or (r2 - r1 = c2 - c1) or (r2 - r1 = c1 - c2);
end;
var
size: integer;
row: array of integer;
// Return true if any of the queens in columns 1 .. n attack (r, c).
function anyAttack(n, r, c: integer): boolean;
var
j: integer;
begin
for j := 1 to n do
if attack(row[j], j, r, c) then exit(true);
exit(false);
end;
function place(n: integer): boolean;
var
i: integer;
begin
for i := 1 to size do
if not anyAttack(n - 1, i, n) then
begin
row[n] := i;
if (n = size) or place(n + 1) then exit(true);
end;
exit(false);
end;
var
i, j: integer;
begin
write('size? ');
readln(size);
setLength(row, size + 1);
if place(1) then
for i := 1 to size do
begin
for j := 1 to size do
write(ifThen(row[j] = i, 'Q', '.'), ' ');
writeln;
end
else writeln('no solution found');
end.Suppose that we want to find the shortest path (if one exists) through a maze. This is a classic application of breadth-first search.
The program below constructs a random maze in a 30 x 30 grid by creating walls in 30% of squares at random. We’d like to solve this maze by finding the shortest path from the upper left corner to the lower right corner. We can use a breadth-first search, though there is no need to create concrete graph node objects for each of the 900 squares in the maze. Instead, we can consider each square to be a node in an abstract graph. We modify our breadth-first search function to compute the positions of any square’s neighbors and to add any neighbor to the queue that is unoccupied by a wall and has not already been visited.
To make this work, it is convenient to assign a unique integer id to each square in the maze (and, by extension, the corresponding abstract graph node). We number the rows and columns of the maze from 0 to 29. Then for each row r and column c we assign the id
s = 30 * r + c
This yields a value from 0 to 899. We can easily map any id back to a row and column number:
r = s div 30
c = s mod 30
We already have 2 different implementations of a queue of integers: one using an array, and another using a linked list. Since we have unique integer ids for nodes, we can use either of those implementations to store a queue of nodes for a breadth-first search in this program.
As we perform the breadth-first search we need to store the
breadth-first graph that it generates so that we can find the
shortest path from the goal back to the start. To do this we have an
array pred that points from each graph node to its
predecessor node. There is no need to keep a separate array
identifying which nodes have been visited, since pred[n] >= 0 for
any visited node n and pred[n] = -1 for unvisited nodes.
Here is the implementation:
{$mode delphi}
{$r+}
uses strutils, queue_array;
const
Size = 30;
type
square = (Empty, Wall, Path);
var
maze: array[0 .. Size - 1, 0 .. Size - 1] of square;
pred: array[0 .. Size * Size - 1] of integer;
procedure bfs;
var
i, j, s: integer;
q: queue;
procedure add(i, j: integer);
var
s1: integer;
begin
if (i < 0) or (i >= Size) or (j < 0) or (j >= Size) then exit;
s1 := i * Size + j;
if (maze[i, j] = Empty) and (pred[s1] = -1) then
begin
pred[s1] := s;
enqueue(q, s1);
end;
end;
begin
init(q);
for i := 0 to high(pred) do
pred[i] := -1; // unvisited
enqueue(q, 0);
pred[0] := 0;
while not isEmpty(q) do
begin
s := dequeue(q);
i := s div Size;
j := s mod Size;
add(i - 1, j);
add(i + 1, j);
add(i, j - 1);
add(i, j + 1);
end;
end;
var
i, j, s: integer;
begin
randomize;
for i := 0 to Size - 1 do
for j := 0 to Size - 1 do
if random(100) < 30 then
maze[i, j] := Wall
else maze[i, j] := Empty;
maze[0, 0] := Empty;
maze[Size - 1, Size - 1] := Empty;
bfs;
s := Size * Size - 1;
if pred[s] <> -1 then
while true do
begin
maze[s div Size, s mod Size] := Path;
if s = 0 then break;
s := pred[s];
end;
for i := 0 to Size - 1 do
begin
for j := 0 to Size - 1 do
case maze[i, j] of
Empty: write(' ');
Wall: write('[]');
Path: write('<>');
end;
writeln;
end;
end.Here is a shortest path this program found through a random maze:
<> [] [][][][] [] [] [] [][]
<> [] [] [] [] [][] [][] [][]
<>[] [] [] [][] [] [] []
<>[] [] [] [][] [][] [] [][]
<> [] [] [] [][] [] [] [] []
<> [][] [] [][] [][] [] []
<> [] [] [][] [] [] [] []
<><><><><><><><><> [][] [] [] [] [][]
[] [][] []<><><><><><><> [] [] [] [] []
[] [] [][]<><> []
[] [] [] [] []<><><>[][] [][] [] []
[] [][] []<> [] [] [][]
[][] [] [] [][] [] [][] <> [] []
[] [] [][] [] [] <>[] [][]
[][] [][] [][] <><> []
[] [][] [][][][][] [] []<><><>
[] [] [] [] [] [] []<><><><><><><>[]
[][] [] [][][] [] [] []<>[][]
[] [] [] [] [][] [] [][] [] [] []<><>
[] [] [] [][][] [] [] [][]<>
[] [][] [] [] []<>
[][][] [] [] [][] [] [][][] [] <><>
[] [] [] [] [] [] [] []<>[]
[] [] [] [][] []<><>
[] [][] [][] [] [] [] [] []<>
[] [] [][] [][] [] [][] [] [] <>
[] [][][] [][] [] [][] [][] <>
[][][] [] [] [][] [] <>
[][] [] [][] [] [] [][] [] [][] []<>
[] [][][] [] [] [] []<>