Binary heap operations



This article discusses common binary heap operations:

Get top (min / max)
This is the simplest operation of a binary heap, which does not involve any structural changes to the heap. To get the minimum node in a min-heap, or the maximum node in the max-heap, we just need to return the root node.

If viewed from the array perspective, it means just to return the index 0.

Hence the time complexity is O(1).

Updating a node
Updating a node is divided into two steps:
 * 1) Update the value of the node.
 * 2) Percolate up / down the updated node to make the heap satisfy heap property again.

Percolate
Percolating up / down means to swap a node with its parent / children to make the heap satisfy min-heap or max-heap property.

Here are the rules of percolating actions for a min-heap. Percolating actions on a max-heap follow the opposite rules.


 * Min-heap
 * Percolate up
 * If a node's value < its parent's value, swap this node with its parent.
 * Percolate down
 * If one child's value < the node's value < other child's value, swap the node with the smaller children.
 * Or, if the node's value > both children's value, swap the node with the smaller children.

Since a complete binary tree with n nodes have the height of $$\lfloor\log_{2}{n}\rfloor$$, the time complexity for percolating up / down is O(log n).

Hence, updating a node, inserting a node, and deleting a node all have time complexity O(log n).

Example: update in a min-heap

 * Original min-heap
 * Change 1 to 10, and percolate the nodes to make it satisfy the min-heap properties.


 * Step 1
 * Change 1 to 10.


 * Step 2
 * Percolate down.


 * The resulting min-heap

Inserting a node
Inserting a node takes two steps:
 * 1) Append the node at the end.
 * 2) Percolate up the inserted node if necessary.

The time complexity for inserting a node is O(log n), i.e. O(height of tree).

Example: insert into a min-heap

 * Original min-heap
 * Insert 2 into this min-heap.


 * Step 1
 * Put 2 at the end.


 * Step 2
 * Percolate up.


 * The resulting min-heap

Deleting the root node
Deleting the root node takes three steps:
 * 1) Update the root node with the tail node's value.
 * 2) Delete the tail node (i.e. the original root node).
 * 3) Percolate down the new root node if necessary.

Example: delete the root

 * Original min-heap
 * Delete the root (1) from this min-heap.


 * Step 1
 * Remove 1, and put the tail's value (i.e. 8) at the root.


 * Step 2
 * Percolate down.


 * The resulting min-heap

Heapify
Heapify is reordering an unsorted array, so that it follows either the min-heap or max-heap property.

This is the most important method to deal with arbitrary unsorted arrays.

Approach 1: Percolate up
To achieve in-place heapifying for the input array, we use one trick: treat the left part as processed, and right part as unprocessed. From left to right, after each time we process an index, all nodes up to that index is considered "processed", while the remaining nodes are considered "unprocessed". Repeat until all nodes are processed.

For each index i the processing steps (percolating up) are as follows:
 * 1) Find the parent index of i:.
 * 2) If , swap the two indices.
 * 3) Set.
 * 4) Repeat the steps above until i = 0. This means that the percolating process is complete for indices 0 through i in the original array.

Then after all elements are processed, the original array is heapified in place.

Example: heapify an unsorted array (percolate up)

 * Input
 * Input array: [6, 2, 8, 9, 1, 4, 3, 5, 7]
 * Binary tree:


 * Procedure
 * Each item number represents the index to be processed.


 * 1) Start with index 0 (value = 6). No change.
 * 2) Percolate up index 1 (value = 2).
 * 3) Percolate up index 2 (value = 8). No change.
 * 4) Percolate up index 3 (value = 9). No change.
 * 5) Percolate up index 4 (value = 1). 1 moves up to the top (height change = 2).
 * 6) Percolate up index 5 (value = 4). 4 moves up by one level.
 * 7) Percolate up index 6 (value = 3). 3 moves up by one level.
 * 8) Percolate up index 7 (value = 5). 5 moves up by one level.
 * 9) Percolate up index 8 (value = 7). no change.


 * Final output
 * Output array: [1, 2, 3, 5, 6, 8, 4, 9, 7]

Time complexity analysis
Note that in heapifying an array, as we process each node, we only keep percolating up necessary nodes. So for each node the time complexity for percolating up is O(tree of height), where the tree of height are as follows:

Total time complexity = $$T(n)_1 = 1 \cdot 2^1 + 2 \cdot 2^2 + 3 \cdot 2^3 + \dots + \lfloor \log_{2}(n+1) \rfloor \cdot 2^{\lfloor \log_{2}(n+1) \rfloor}$$

$$2 \cdot T(n) = 1 \cdot 2^2 + 2 \cdot 2^3 + 3 \cdot 2^4 + \dots + \lfloor \log_{2}(n+1) \rfloor \cdot 2^{\lfloor \log_{2}(n+1) \rfloor + 1}$$

So $$2 \cdot T(n) - T(n) = -2^1 - 2^2 - 2^3 - \ldots - 2^{\lfloor \log_{2}(n+1) \rfloor} + \lfloor \log_{2}(n+1) \rfloor \cdot (n+1) \cdot 2$$$$= \lfloor \log_{2}(n+1) \rfloor \cdot (n+1) \cdot 2 - 2(1 + 2 + 2^2 + 2^3 + \dots + 2^{\lfloor \log_{2}(n+1) \rfloor})$$$$= 2(n+1) \lfloor \log_{2}(n+1) \rfloor - 2 \cdot \frac{1 - 2^{\lfloor \log_{2}(n+1) \rfloor}}{1-2}$$$$= 2(n+1) \lfloor \log_{2}(n+1) \rfloor + 2(1 - 2^{\lfloor \log_{2}(n+1) \rfloor})$$$$= O(n \log n)$$.

But this is not the optimal approach. So here is another approach.

Approach 2: Percolate down
Instead by trying to insert each element at the end and percolate up, we can do the following: for each node that is not a leaf node (i.e. each node that has at least one child), we percolate down that node if necessary. Repeatedly perform the percolate-down action from the deepest level to the root.

Example: Reversing a max-heap to a min-heap (percolate down)

 * Input
 * Input array: [9, 8, 7, 6, 5, 4, 3, 2, 1]
 * Binary tree representation:


 * Procedure
 * The number before each step indicates the index to be processed.
 * In this example, we start from index 3, which is the last index that is not a leaf node.


 * Final output
 * Output array: [1, 2, 3, 6, 5, 4, 7, 8, 9]
 * Binary tree representation:

Time complexity analysis
We first note that a binary heap with n nodes has the height $$\lfloor\log_{2}(n+1)\rfloor$$.

Then we need to find the correspondence between each node index k and the level of that node:

So for a binary heap with n total nodes, for each index k, the number of levels to percolate down, in the worst case, is $$d(k) = \lfloor\log_{2}n\rfloor - \lfloor\log_{2}(k+1)\rfloor \le \log_{2}\frac{n}{k+1} + 1$$

In terms of the maximum number of levels to percolate down (d(k)), we have

Total time complexity to process n nodes = $$T(n)_2 =$$$$(\lfloor \log_{2}n \rfloor - 0) \cdot 2^0 + (\lfloor \log_{2}n \rfloor - 1) \cdot 2^1 + (\lfloor \log_{2}n \rfloor - 2) \cdot 2^2 + \dots + 0 \cdot 2^{\lfloor \log_{2}n \rfloor}$$ $$= \lfloor \log_{2}n \rfloor \cdot (2^0 + 2^1 + 2^2 + \dots + 2^{\lfloor \log_{2}n \rfloor}) - (0 \cdot 2^0 + 1 \cdot 2^1 + 2 \cdot 2^2 + \cdots$$$$+ \lfloor \log_{2}n \rfloor \cdot 2^{\lfloor \log_{2}n \rfloor})$$ $$= \lfloor \log_{2}n \rfloor \cdot \frac{2(1-2^{\lfloor \log_{2}n \rfloor})}{1-2} - 2n \lfloor \log_{2}n \rfloor - 2(1 - 2^{\lfloor \log_{2}n \rfloor})$$$$= \lfloor \log_{2}n \rfloor \cdot 2(2^{\lfloor \log_{2}n \rfloor} - 1)- 2n \lfloor \log_{2}n \rfloor - 2(1 - 2^{\lfloor \log_{2}n \rfloor})$$ (from what we had for $$T(n)_1$$)

To simplify the approximation, let n be some power of 2. Then $$T(n)_2 = \log_{2}n \cdot 2(n-1) - 2n \log_{2}n - 2 + 2n$$$$= 2n \log_{2}n - 2 \log_{2}n - 2n \log_{2}n - 2 + 2n$$$$= -2 \log_{2}n + 2n - 2 = O(n)$$.

Hence the percolate-down approach gives the total time complexity of O(n).