How Tensors Generalize Scalars, Vectors, and Matrices

This article explores how tensors serve as the foundational generalization of scalars, vectors, and matrices in advanced mathematics. By examining the concepts of tensor rank, index notation, coordinate invariance, and multilinear mappings, this guide breaks down how these mathematical objects transition from simple single numbers into complex, multidimensional arrays capable of describing intricate geometric and physical relationships.

The Hierarchy of Tensor Rank

To understand how tensors generalize simpler mathematical objects, it is best to view them through the concept of rank (or order). The rank of a tensor dictates the number of indices required to uniquely identify a specific component within the array, as well as the number of directions the object encompasses in a given space.

  • Scalars (Rank-0 Tensors): A scalar is a single number representing magnitude only, such as temperature or mass. It has no directional components and requires zero indices to specify its value (\(x\)).
  • Vectors (Rank-1 Tensors): A vector is a one-dimensional array of numbers representing both magnitude and a single direction, such as velocity. In an \(N\)-dimensional space, a vector has \(N^1\) components. It requires one index to identify a specific component (\(v_i\)).
  • Matrices (Rank-2 Tensors): A matrix is a two-dimensional grid of numbers arranged in rows and columns. In an \(N\)-dimensional space, it contains \(N^2\) components. It requires two indices to pinpoint a specific value (\(M_{ij}\)).
  • Higher-Order Tensors (Rank-3 and beyond): When you stack matrices together into a three-dimensional cube of numbers, you create a Rank-3 tensor, which requires three indices (\(T_{ijk}\)) and contains \(N^3\) components. Tensors generalize this pattern infinitely. A Rank-\(R\) tensor in an \(N\)-dimensional space is a multidimensional array containing \(N^R\) components, requiring \(R\) indices to specify an element.

Multilinear Mappings

In advanced mathematics, viewing a tensor merely as a multidimensional grid of numbers is incomplete. More fundamentally, a tensor is a multilinear map.

A function is linear if scaling the input scales the output proportionally, and adding inputs adds the outputs. A matrix (Rank-2 tensor) represents a linear transformation: it takes one vector as an input and outputs another vector, or it takes two vectors and outputs a scalar (via the dot product).

Tensors generalize this into multilinearity. A multilinear map is a function that takes multiple vectors (and dual vectors, known as covectors) as inputs and produces a single scalar output, with the strict requirement that the function remains linear in each of its arguments independently. If you hold all inputs constant except for one, the tensor acts as a simple linear operator on that single varying input.

Construction via the Tensor Product

Higher-rank tensors are formally constructed from lower-rank tensors using an operation called the tensor product, denoted by the symbol \(\otimes\).

While the dot product of two vectors reduces their rank to yield a scalar, the tensor product of two vectors expands their rank to create a Rank-2 tensor (a matrix). Specifically, if you take two vectors \(u\) and \(v\), their tensor product \(u \otimes v\) produces a grid consisting of all possible products of their individual components:

\[(u \otimes v)_{ij} = u_i v_j\]

By repeatedly applying the tensor product to vectors and covectors, mathematicians construct spaces of multilinear arrays of any arbitrary rank.

Coordinate Invariance and Transformation Laws

The ultimate distinction between a standard multidimensional array of numbers (like a spreadsheet) and a true mathematical tensor lies in how the object behaves when you change your perspective.

Values in a vector or matrix depend on the chosen coordinate system (the basis). If you rotate or stretch your coordinate system, the individual numbers inside the vector or matrix must change to compensate, ensuring the actual geometric object they represent remains unchanged.

Tensors generalize this property of coordinate invariance. A multidimensional array is only a tensor if its components transform according to specific, predictable mathematical rules when the basis changes. These rules split tensor indices into two categories:

  1. Contravariant components (written with upper indices, \(T^i\)): These components transform inversely to the change of basis. Standard vectors are contravariant; if you double the size of your coordinate grid units, the numerical values of the vector's components halve so the vector's physical length stays the same.
  2. Covariant components (written with lower indices, \(T_j\)): These components transform in the exact same way as the change of basis. Covectors (linear functionals) are covariant.

A general tensor can possess both types of indices simultaneously, written as \(T^{ij...}_{kl...}\). By combining contravariant and covariant properties into a single multilinear array, tensors provide a universal language for mathematics and physics, allowing equations to hold true regardless of the coordinate system used to measure them.