Introduction to Algorithms, 2020-1
Week 8: Notes

Some of today's topics are covered in Problem Solving with Algorithms:

• 4. Basic Data Structures
  • 4.3 What is a Stack?
  • 4.4 The Stack Abstract Data Type
  • 4.5 Implementing a Stack in Python
  • 4.10 What Is a Queue?
  • 4.11 The Queue Abstract Data Type
  • 4.12 Implementing a Queue in Python
• 7. Trees and Tree Algorithms
  • 7.3 Vocabulary and Definitions
  • 7.5 Nodes and References

And in Introduction to Algorithms:

• 10.1 Stacks and Queues

Here are some more notes.

Stacks

A stack is an abstract data type that provides two operations called push and pop. push(x) pushes a value x onto a stack, and pop() removes the value that was most recently pushed. This is like a stack of sheets of paper on a desk, where sheets can be added or removed at the top.

In other words, a stack is a last in first out (LIFO) data structure: the last element that was added is the first to be removed.

Often a stack will provide an additional operation isEmpty that returns true if it contains no elements.

Before we implement a stack, let's look at how one can be used:

s = Stack();
s.push(4);
s.push(8);
for i in range(1, 6) do
  s.push(i);
  
while not s.isEmpty() do
  print(pop(s), end = ' ')
print()

This code will write

5 4 3 2 1 8 4

Implementing a stack with an array

A straightforward way to implement a stack in Python is using a list, i.e. an array:

class Stack:
  def __init__(self):
    self.vals = []

  def isEmpty(self):
    return len(self.vals) == 0
  
  def push(self, x):
    self.vals.append(x)
  
  def pop(self):
    assert not self.isEmpty(), 'stack is empty'
    return self.vals.pop()

The implementation is straightforward. The list contains all stack elements, with the top of the stack (i.e. the most recently pushed element) at the end of the list.

With this implementation, push will run in O(1) on average, since that is the running time of append(). pop will always run in O(1).

implementing a stack using a linked list

Alternatively we can implement a stack using a linked list. To accomplish this, we will create a class LinkedStack with an attribute head that points to the head of the list. Now our stack operations are quite straightforward:

class LinkedStack:
  def __init__(self):
    self.head = None
    
  def isEmpty(self):
    return self.head == None
  
  def push(self, x):
    self.head = Node(x, self.head)
  
  def pop(self):
    assert self.head != None, 'stack is empty'
    x = self.head.val
    self.head = self.head.next
    return x

Different implementations of an abstract type may have different performance characteristics. If we implement a stack using an array (i.e. a Python list), the push operation will take O(N) in the worst case, where N is the current stack size. Our linked list-based implementation performs differently: push always runs in O(1).

Queues

Queues are another important abstract data type. A queue provides two operations called enqueue and dequeue. enqueue(x) adds an element to the tail of a queue, and dequeue() removes the element at the head. A queue is something like people waiting in a line: you must join at the back of the line, and the person at the front of the line is served next.

In other words, queues are a first in first out (FIFO) data structure: the first value added to a queue will be the first one to be removed.

A queue can be used like this:

q = Queue()

q.enqueue(4)
q.enqueue(77)
q.enqueue(12)

print(q.dequeue())  # writes 4
print(q.dequeue())  # writes 77

implementing a queue using a linked list

We can implement a queue efficiently using a linked list. To do so, we keep pointers to both the head (first node) and tail (last node) in the list. This allows us to enqueue or dequeue elements in O(1). Here is an implementation:

class LinkedQueue:
  def __init__(self):
    self.head = self.tail = None
  
  def isEmpty(self):
    return self.head == None
  
  def enqueue(self, x):
    n = Node(x, None)
    if self.head == None:
      self.head = self.tail = n
    else:
      self.tail.next = n
      self.tail = n
      
  def dequeue(self):
    assert self.head != None, 'queue is empty'
    x = self.head.val
    self.head = self.head.next
    return x

binary trees

A binary tree consists of a set of nodes. Each node contains a single value and may have 0, 1, or 2 children.

Here is a picture of a binary tree of integers. (Note that this is not a binary search tree, which is a special kind of binary tree that we will discuss later.)

In this tree, node 10 is the root node. 14 is the parent of 12 and 6. 12 is the left child of 14, and 6 is the right child of 14. 14 is an ancestor of 22, and 22 is a descendant of 14.

A node may have 0, 1, or 2 children. In this tree, node 15 has a right child but no left child.

The subtree rooted at 14 is the left subtree of node 10. The subtree rooted at 1 is the right subtree of node 10.

The nodes 12, 5, 22, 4, and 3 are leaves: they have no children. Nodes 10, 14, 1, 6, and 15 are internal nodes, which are nodes that have at least one child.

The depth of a node is its distance from the root. The root has depth 0. In this tree, node 15 has depth 2 and node 4 has depth 3. The height of a tree is the greatest depth of any node. This tree has height 3.

The tree with no nodes is called the empty tree.

Note that a binary tree may be asymmetric: the right side might not look at all like the left. In fact a binary tree can have any structure at all, as long as each node has 0, 1, or 2 children.

A binary tree is complete iff every internal node has 2 children and all leaves have the same height. Here is a complete binary tree of height 3:

A complete binary tree of height h has 2^h leaves, and has 2⁰ + 2¹ + … + 2^h-1 = 2^h – 1 interior nodes. So it has a total of 2^h + 2^h – 1 = 2^h + 1 – 1 nodes. In this tree there are 2³ = 8 leaves and 2³ – 1 = 7 interior nodes, a total of 2⁴ – 1 = 15 nodes.

Conversely if a complete binary tree has N nodes, then N = 2^{h + 1} – 1, where h is the height of the tree. And so h = log₂(N + 1) – 1 ≈ log₂(N) – 1 = O(log N).

binary trees in Python

We can represent a binary tree in Python using node objects, similarly to how we represent linked lists. Here is a node type for a binary tree of integers:

class Node:
  def __init__(self, val, left, right):
    self.val = val
    self.left = left      # left child, or None if absent
    self.right = right    # right child, or None if absent
  

We will generally refer to a tree using a reference to its root. We use None to represent the empty tree, just as we used None for the empty linked list. In all leaf nodes, left and right will be None.

Here is a small binary tree with just 3 nodes:

We can build this in Python as follows:

q = Node(7, None, None)
r = Node(5, None, None)
p = Node(4, q, r)

To build larger trees, we will write functions that use loops or recursion.

Here is a function that computes the sum of all values in a binary tree:

def treeSum(node):
  if node == None:
    return 0
  return node.val + treeSum(node.left) + treeSum(node.right)

It is much easier to write this function recursively than iteratively. Recursion is a natural fit for trees, since the pattern of recursive calls in a function like this one can mirror the tree structure.