In my last post I introduced quadratic assignment problems. As I explained, the problem is to assign an equal number of facilities to locations, minimizing the transportation cost. A compact way to write the problem is to store the “flows” in a matrix A, the “distances” in matrix B, and represent assignments as permutation matrices. If you throw in a linear term C, then the problem can be written as:

Branch-and-bound algorithms are the most common technique for finding optimal solutions for QAP. The next few posts will focus on describing how branch-and-bound works and will result in a complete implementation of a QAP solver in C#. Branch-and-bound is a “divide and conquer” solution technique for combinatorial problems. Branch-and-bound repeatedly divides a problem into simpler subproblems of the same type, which are then solved or further subdivided. The search space of a problem is partitioned into search spaces for its subproblems. The algorithm generates a search tree with the root corresponding to the original problem to be solved. At each node in the search tree, a relaxed version of the subproblem is solved, typically by loosening the constraints of the subproblem. The solution to the relaxation is a lower bound to the original subproblem. If a lower bound at a given node exceeds the value of a previously known solution, then continued searching on this path cannot lead to an improved solution, and there’s no need to continue searching on that node’s subtree (this is called “fathoming”).

A little less formally, we take the search space and divide it into subregions. Then for each piece, we solve a “relaxed” version of the problem to see if we can rule out the solution lying in that subregion. If we can’t rule it out, we divide the subregion into pieces and repeat. The “relaxation” part is bounding, the dividing is called branching. The effectiveness of a branch-and-bound algorithm depends on 1) the strength of the bound, 2) how fast bounds can be computed, and 3) the branching decisions. For example, if the bound is terrible we’ll never fathom subproblems and we’ll end up searching the entire tree.

So how do we apply branch-and-bound to QAP? The solution to a QAP is an assignment of facilities to locations. Therefore the subproblems are obtained by assigning some of the facilities to locations. To branch, I can pick a facility and try assigning it to each of the available locations (or vice-versa). As we will see, the logic for selecting which facility/location to branch on is extremely important.

Let’s start putting together some of the pieces of our branch-and-bound solver for QAP. First, let’s define a QAP, which as I said before consists of the flow and distance matrices A and B, and a linear term C.

// A quadratic assignment problem. public class Qap { // A (flow matrix). public double[][] A { get; set; } // B (distance matrix). public double[][] B { get; set; } // C (linear costs). public double[][] C { get; set; } // Constant term. public double Shift { get; set; } // Size of (A, B, C) matrices. public int Size { get { return A == null ? 0 : A.Length; } } // Compute tr AXBX' + CX', where the permutation matrix X is determined by p. public double Evaluate(int[] p) { double obj = this.Shift; for (int i = 0; i < A.Length; i++) { obj += this.C[i][p[i]]; for (int j = 0; j < A[i].Length; j++) { obj += this.A[i][j] * this.B[p[i]][p[j]]; } } return obj; } }

To make the code shorter I am using C# 3.0 automatic properties. In addition to the matrices I have also added a constant term “Shift”, whose purpose will become clear in a moment. I’ve also provided a method called Evaluate() that computes the objective given a permutation. Let’s think about how branching works. If we go back to the mathematical formulation of QAP at the start of this post, assigning a facility to a location means setting X_ij = 1. If we plug X_ij into the formula above we see that we end up with a QAP of size n – 1 with:

A′ = A(ii)

B′ = B(jj)

C′ = C(ij) + 2 a_i b_j ,

Shift’ = A[i][i] B[j][j] + C[i][j] + Shift

Where A(ij) denotes matrix A with row i and column j removed, a_i are the elements in row i of A, excluding A[i][i], and let b_j are the elements in column j of B, excluding b[j][j]. Here’s the code:

private static Qap Reduce(Qap qap, int i, int j) { Qap r = new Qap(); r.A = Reduce(qap.A, i, i); r.B = Reduce(qap.B, j, j); r.C = Reduce(qap.C, i, j); // Tack on: 2 a'_i b'_j (where a'_i is row i of A, with element i removed) for (int ii = 0; ii < r.Size; ii++) { for (int jj = 0; jj < r.Size; jj++) { r.C[ii][jj] += 2 * qap.A[i][ii < i ? ii : ii + 1] * qap.B[jj < j ? jj : jj + 1][j]; } } r.Shift = qap.Shift + qap.A[i][i] * qap.B[j][j] + qap.C[i][j]; return r; } private static double[][] Reduce(double[][] M, int i, int j) { double[][] R = LinearAlgebra.NewMatrix(M.Length - 1); for (int ii = 0; ii < R.Length; ii++) { for (int jj = 0; jj < R[ii].Length; jj++) { R[ii][jj] = M[ii < i ? ii : ii + 1][jj < j ? jj : jj + 1]; } } return R; }

LinearAlgebra.NewMatrix() is a helper function that initializes the matrix. So now we have a data structure for QAP, and we now how to obtain subproblems. Next time we’ll look at how to compute lower bounds.

**Update 6/9/2010: **Here is an additional method to read a problem in QAPLIB format. It does not attempt to read the “C” matrix, and it is not totally robust. But it should work.

public static Qap Read(StreamReader reader) { string content = reader.ReadToEnd(); var tokens = content.Split(new char[] { ' ', '\n', '\r'}, StringSplitOptions.RemoveEmptyEntries); int k = 0; int size; if (Int32.TryParse(tokens[k++], NumberStyles.Integer, NumberFormatInfo.InvariantInfo, out size)) { Qap qap = new Qap(); qap.A = ReadMatrix(size, size, tokens, ref k); qap.B = ReadMatrix(size, size, tokens, ref k); return qap; } throw new InvalidDataException("Invalid QAP size."); } private static double[][] ReadMatrix(int m, int n, string[] tokens, ref int k) { if (k + m * n > tokens.Length) { throw new InvalidDataException("Not enough matrix data."); } double[][] matrix = LinearAlgebra.NewMatrix(m, n); for (int i = 0; i < matrix.Length; i++) { for (int j = 0; j < matrix[i].Length; j++) { if (!Double.TryParse(tokens[k++], NumberStyles.Float, NumberFormatInfo.InvariantInfo, out matrix[i][j])) { throw new InvalidDataException("Invalid matrix entry."); } } } return matrix; }