How the Gram-Schmidt Process Creates an Orthonormal Basis

This article provides a complete overview of how the Gram-Schmidt process transforms a set of linearly independent vectors into an orthonormal basis. It defines the essential concepts of orthogonality and normalization, breaks down the mathematical algorithm into a sequential step-by-step guide, and walks through a practical two-dimensional numerical example to demonstrate how vector projections are subtracted to achieve perpendicularity before scaling the vectors to a unit length of one.

Understanding the Goal

To understand the Gram-Schmidt process, you must first understand the starting material and the desired end result:

  • Linearly Independent Vectors (Input): A set of vectors where no vector in the set can be written as a linear combination of the others. Simply put, none of the vectors point in the exact same (or opposite) direction, and none are redundant.
  • Orthogonal Basis (Intermediate Step): A set of vectors that are all perpendicular to one another. Mathematically, the dot product of any two different vectors in this set is zero.
  • Orthonormal Basis (Output): An orthogonal basis where every vector has been scaled (normalized) so that its length (magnitude) is exactly 1.

The Gram-Schmidt process takes the messy, angled vectors of a linearly independent set and straightens them out into right angles, then trims or stretches them to a uniform length of 1, all while spanning the exact same geometric space.

The Core Mechanism: Vector Projection

The secret to the Gram-Schmidt process is vector projection. If you have two vectors, v₁ and v₂, that are not perpendicular, it means v₂ leans slightly in the direction of v₁.

You can think of the projection as the "shadow" that v₂ casts onto v₁. The formula for projecting vector v onto vector u is:

proj_u(v) = [(v · u) / (u · u)] u

(Note: v · u represents the dot product of the two vectors).

If you take the original vector v₂ and subtract the shadow it casts on v₁, the remaining part of v₂ has no component left in the direction of v₁. Therefore, the resulting vector is completely perpendicular (orthogonal) to v₁.

The Step-by-Step Algorithm

Suppose you have a set of three linearly independent vectors: {v₁, v₂, v₃}. We will first create an orthogonal set {u₁, u₂, u₃}, and then normalize them to create the final orthonormal basis {e₁, e₂, e₃}.

Phase 1: Orthogonalization

Step 1: Set the first orthogonal vector equal to the first original vector. It serves as your starting anchor. u₁ = v₁

Step 2: Take the second vector (v₂) and subtract its projection onto the first orthogonal vector (u₁). u₂ = v₂ − proj_u₁(v₂)

Step 3: Take the third vector (v₃) and subtract its projections onto both of the previously found orthogonal vectors (u₁ and u₂). u₃ = v₃ − proj_u₁(v₃) − proj_u₂(v₃)

(If you have more vectors, you continue this pattern: take the next vector vₖ and subtract its projections onto all previously calculated u vectors).

Phase 2: Normalization

Now that {u₁, u₂, u₃} are all mutually perpendicular, you must make their lengths equal to 1. You do this by dividing each vector by its own magnitude (length), denoted as ||u||. The magnitude is found by taking the square root of the vector's dot product with itself: ||u|| = √(u · u).

Step 4: Divide each orthogonal vector by its magnitude.

  • e₁ = u₁ / ||u₁||
  • e₂ = u₂ / ||u₂||
  • e₃ = u₃ / ||u₃||

The resulting set {e₁, e₂, e₃} is your orthonormal basis.

A Practical Example

Let's convert the following two linearly independent 2D vectors into an orthonormal basis: v₁ = (3, 0) v₂ = (1, 2)

1. Find the first orthogonal vector (u₁): We simply keep the first vector as it is. u₁ = (3, 0)

2. Find the second orthogonal vector (u₂): We subtract the projection of v₂ onto u₁ from v₂. First, calculate the dot products needed for the projection formula:

  • v₂ · u₁ = (1)(3) + (2)(0) = 3
  • u₁ · u₁ = (3)(3) + (0)(0) = 9

Now, calculate the projection: proj_u₁(v₂) = (3 / 9) * (3, 0) = (1/3) * (3, 0) = (1, 0)

Subtract this projection from v₂: u₂ = (1, 2) − (1, 0) = (0, 2)

We now have our orthogonal basis: u₁ = (3, 0) and u₂ = (0, 2). You can verify they are perpendicular because their dot product is (3)(0) + (0)(2) = 0.

3. Normalize the orthogonal vectors (e₁ and e₂): Find the magnitude of u₁: ||u₁|| = √(3² + 0²) = √9 = 3 Divide u₁ by its magnitude: e₁ = (3, 0) / 3 = (1, 0)

Find the magnitude of u₂: ||u₂|| = √(0² + 2²) = √4 = 2 Divide u₂ by its magnitude: e₂ = (0, 2) / 2 = (0, 1)

Final Result: The orthonormal basis is {(1, 0), (0, 1)}. Both vectors are perpendicular to each other, and both have a length of exactly 1.