1.0 - Minimum Spanning Trees

Consider laying cable (e.g. for the NBN)
- Create a connected network of houses that uses the least amount of cable.
- Assume that the speed of the cable is fast, i.e. the distance from house-to-house is irrelevant (we aren’t looking for the shortest paths
We are given an undirected, weighted graph $G=(V, E)$ with weights $w$ such that:
- Vertices $V$ represent houses to be connected to the NBN network
- For each edge $(u, v) \in E$ , the weight $w(u,v)$ is the cost of laying the cable from house $u$ to $v$ .

We want to find an acyclic subset $T\subseteq E$ that:
- Connects all of the vertices in $G$ such that the total weight $T=\sum_{(u,v)\in T} w(u,v)$ is minimised.

Inputs

$G$ a connected, undirected, weighted graph $G=(V, E)$
$w$ Weights $w$ , where $w(u,v)$ is the weight of the edge from $u$ to $v$

Output

$T$ a subset of $E$ , that forms a spanning tree (connected acyclic subgraph of $G$ ) that contains all vertices of $G$ (spanning) and is of minimal weight:

$\text{weight(T)}=\sum_{(u,v)\in T} w(u,v)$
The smallest connected, undirected, weighted graph with more than one minimum spanning tree are three connected vertices, with equally weighted edges.

1.1 - Generic Construction of a MST

💡 Incrementally construct T which is a set of edges, and which will eventually become a MST of G

genericMST(G, w):
  T = Graph() # Empty
  while T is not a spanning tree
		// invariant: T is a subset of some MST of G
    find an edge (u, v) that is safe for T // new edge added forms a new T that
                                           // is also a subset of a MST for the graph
    T = T ∪ {(u, v)}
  return T

We want to figure out how to “greedily” choose edges to add to T - choose the edges in an efficient way.

1.1 - Prim’s Algorithm

🌱 Guaranteed to form a MST at the end of the algorithm if we choose least-weighted edges connected to our graph.

T is always a tree (a connected acyclic sub-graph of G):

Initially, T is chosen to contain any one vertex from G.V
At each step, the least-weight edge leaving T is added
The algorithm stops when T is spanning.
How do we (efficiently) find the least-weight edge leaving T?
Maintain a priority queue, Q containing vertices $V-T$ (all vertices that are not in tree)
For each $v\in V-T$ :
- $\text{v.key}$ the least weight of an edge connecting v to T
- $v.\pi$ the vertex adjacent to v on that least-weight edge (parent)
Represent

$T=\{(v, v.\pi)\ :\ v\in V-\{r\}-Q\}$
where $r$ is the first vertex chosen for $T$

1.1.1 - Priority Queues

A priority queue Q maintains a set S of elements, each associated with a key, denoting its priority

In a min-priority queue, an element with the smallest key has the highest priority

Operations are available to:

insert(Q, x) Insert an element $\text x$ , with key $\text{x.key}$ into Q
extract_min(Q) Removes and returns the element of Q with the smallest key
decrease_key(Q,x,k) decreases the key of x in Q to the value k

We have the possible implementations of priority queues, and their associated time complexities:

Unsorted List PQ

Insert $\Theta(1)$
extract_min(Q) $\Theta(n)$
decrease_min(Q, x, k) $\Theta(1)$ given you have a reference to the object

Sorted List PQ

Insert $\Theta(n)$
extract_min(Q) $\Theta(1)$ Remove element at start of list
decrease_key(Q, x, k) $\Theta(n)$ Decrease key, re-sort list.

Heap (Complete Tree)

Insert $O(\log n)$
extract_min(Q) $O(\log n)$
decrease_key(Q, x, k) $O(\log n)$

As it is a complete tree, the height of the tree $\Theta(\log n)$

1.1.2 - Implementation of Prim’s Algorithm Using PQ

Operates on a graph G, with weights w at a start vertex r

MST_Prim(G, w, r)
	for each v ∈ G.V
			// v.key: least edge weight connecting v to T
			v.key = infinity 
			//vertex adjacent to v in T on least edge
      v.pi = NULL
	r.key = 0 // Initially set to highest priority in PQ
  while Q ≠ ∅
			// invariant: T is a subset of some MST of G
			//                where T = {(v, v.pi) : v \in V-{r}-Q
      u = extract_min(Q)
			// Need to re-compute the priority of edges in sub-MST
			// as we have changed the minimum spanning tree -priorities
			// might have changed
      for each v ∈ G.Adj[i] // "Edge relaxation process"
          if v ∈ Q and w(u, v) < v.key
							v.key = w(u, v) // Decrease the key
              v.pi = u

1.1.3 - Prim’s Algorithm in Practice

Firstly begin with all vertices red (in the priority queue) with infinite weights, except for our initial vertex, which we assign a priority of 0.

We first remove the vertex with the highest priority
Also, need to update the priority of the adjacent vertices using edge relaxation; improving weights

Choose the next vertex with highest priority
Update the weights of the vertices using the edge relaxation process

Choose the next vertex with highest priority.
Update weights - notice 12→6, 10→8 - even though relaxed once, can be relaxed further

Choose next vertex with highest priority
Update weights of adjacent vertices.

1.1.4 - Performance of Prim’s Algorithm

Q <- V 
key[v] <- ∞ for all v ∈ V
key[s] <- 0 for some arbitrary s ∈ V
while Q ≠ ∅:
	do u <- extract_min(Q)
		for each v ∈ Adj[u]
			do if v ∈ Q and w(u, v) < key[v]
				then key[v] <- w(u, v)
					π[v] <- u

We know that the initialisation process (lines 1-3) take $\Theta(V)$ time:
- Insertion of all of the vertices in the priority queue can be completed in $\Theta(V)$ time if inserted intelligently - all of our start vertices have a priority of $\infty$ except for our start vertex so inserting the start vertex intelligently is all it takes to achieve $\Theta(V)$ time.
- Additionally, the two for loops can similarly be completed in $\Theta(V)$ time.
The outermost while loop is executed $|V|$ times, as each vertex is removed from the queue once.
- This also means that we perform $|V|$ extract_min(Q) operations (in $\Theta(\log n)$ time)
- The for-loop iterates at most $\text{degree}(u)$ times, which is the amount of vertices adjacent to it.
  - We perform some constant-time operations (i.e. checks in if statement) and may optionally perform a decrease-key operation of time $\Theta(\log n)$
  - Through the handshaking lemma, we know that we’re implicitly performing $\Theta(E)$ decrease-key operations

\Theta(V)\times T_{\text{extract-min}} +\Theta(E)\times T_{\text{decrease-key}}

Ultimately, the performance of the algorithm depends on the implementation of the priority queue.

PQ Implementation	$T_{\text{extract-min}}$	$T_\text{decrease-key}$	Total
unsorted array	$O(V)$	$O(1)$	$O(V^2)$
binary heap	$O(\lg V)$	$O(\lg V)$	$O(E \lg V)$
fibonacci heap	$O(\lg V)$ amortised	$O(1) $ amortised	$O(E+V \lg V)$ worst case

$\text{Unsorted Array}: \Theta(V)\times O(V)+\Theta(E)\times O(1)=\Theta(V^2)+\Theta(V^2)\times O(1)$

as $\Theta(E)=V^2$

$\text{Binary Heap}:\Theta(V)\times O(\lg V)+\Theta(E)\times O(\lg V)=O(V \lg V + E \lg V)$

Since $E>V$ as the graph is connected, we can write $O(E \lg V)$

Fibonacci heaps are very complicated data structures to implement, and have large associated constant factors such that in practice it isn’t really used.