How iOS 18 Math Notes Interprets Handwritten Symbols

Math Notes in iOS 18 uses advanced on-device machine learning, stroke-tracking technology, and contextual mathematical parsing to interpret complex handwritten symbols like pi (\(\pi\)) and square root radicals (\(\sqrt{}\)). By analyzing how a stroke is drawn in real time rather than simply scanning the final static image, the system reliably distinguishes between visually similar characters, binds operators to their corresponding variables, and instantly evaluates expressions the moment an equals sign is drawn.

Stroke-Level Vector Tracking

Instead of treating handwriting as a flat grid of pixels, iPadOS and iOS capture strokes as temporal vector data. The Neural Engine records the sequence, direction, velocity, and curvature of the Apple Pencil or finger input:

  • Pi (\(\pi\)): To prevent confusion with the letter "n," the number "11," or the letters "tt," the model looks for the distinct stroke order: typically a horizontal top bar followed by two downward vertical or slightly curved strokes. The temporal sequence helps the system determine that the strokes form a single mathematical character.
  • Square Root (\(\sqrt{}\)): A radical symbol is identified by its specific continuous trajectory—a small downward notch, a sharp drop, an upward diagonal slash, and an extending horizontal bar (the vinculum).

Spatial Parsing and the Vinculum

Interpreting a symbol like a square root requires structural comprehension, not just character recognition. Math Notes analyzes the spatial relationship between the radical symbol and the numbers nearby:

  • Scope and Grouping: The horizontal bar (vinculum) acts as a grouping operator. The system calculates the bounding box beneath the bar to determine exactly which values or expressions fall inside the radical.
  • Dynamic Expansion: If you write additional numbers underneath an existing radical bar, the system updates its spatial model to include those values in the radicand.

Contextual Language Models

Handwriting varies drastically between individuals, making isolated character recognition prone to errors. iOS 18 overcomes this by passing recognized strokes through a specialized mathematical language model:

  • Semantic Disambiguation: If a drawn symbol resembles both the letter "x" and a multiplication cross, or the letter "t" and a plus sign, the model evaluates the surrounding characters. If placed between two numbers, the system weights the symbol as an operator.
  • Constant Recognition: When \(\pi\) appears adjacent to a number (e.g., \(2\pi r\)), the parser recognizes it as a mathematical constant rather than a stray set of lines, automatically assigning it the value of approximately 3.14159.

Real-Time Evaluation Pipeline

Once symbols are classified and spatially mapped into an abstract syntax tree (AST), Math Notes evaluates the problem locally using the device’s hardware. Writing an equals sign (\(=\)) acts as the execution trigger. The system processes the syntax tree, computes the result—factoring in constants like pi and functions like square roots—and renders the solution inline, matching the user's personal handwriting style via Apple's Smart Script synthesis.