Fast-Fourier Transform (FTT) and Number-Theoretic Transform (NTT) - FFT, NTT and LBC

#cryptography #fft-ntt-and-lbc

Table of Contents

The need to make systems and algorithms faster so as to make them more practical has been ever constant in the world of software engineering and cryptography at large, and lattice-based cryptography is not left out.

Number-theoretic Transform (NTT) was used in ML-KEM (Module-Lattice-Based Key-Encapsulation Mechanism), a lattice-based cryptographic algorithm used in establishing a shared secret key between two parties over a public channel. NTT is an analogue to the beautiful yet powerful Fast-fourier Transform (FTT).

This article is part one of two-part series titled Fast-fourier Transform (FTT), Number-theoretic Transform (NTT) and Lattice-based Cryptography. The goal of this post is to take a deep dive into FFT and NTT while part two will explore how NTT is used in lattice-based algorithms like ML-KEM and ML-DSA.

As always, we are going to take a discovery approach, building up slowly from the familiar to the unfamiliar.

Polynomial Multiplication

Given two polynomials AA and BB of degree 2 :

  • A=34x+x2A = 3 - 4x + x^2
  • B=65x+x2B = 6 - 5x + x^2

We are asked to multiply them. Easy, right?

  • C=AB=(34x+x2)(65x+x2)=3(65x+x2)4x(65x+x2)+x2(65x+x2)=1839x+29x29x3+x4C = A\ast B = (3 - 4x + x^2)(6 - 5x + x^2) = 3 * (6 - 5x + x^2) - 4x * (6 - 5x + x^2) + x^2 * (6 - 5x + x^2) = 18 - 39x + 29x^2 - 9x^3 + x^4

Below is an implementation of this in Python:

A = [3, -4, 1] # coefficients of A

B = [6, -5, 1] # coefficients of B

d = 2 # degree

C = [0 for _ in range((2 * d) + 1)]

for i in range(d + 1):
	for j in range(d + 1):
		C[i + j] += A[i] * B[j]

This algorithm takes O(d2)O(d^2) time. Is there a faster way to perform polynomial multiplication?

From the implementation above, you can tell that the polynomial is represented as a list of its coefficients. There are other representations.

According to the uniqueness theorem, a polynomial of degree at most dd is uniquely determined by its values at d+1d + 1 distinct points. That is, a polynomial f(x)f(x) of degree dd can be uniquely constructed by the points: (x0,f(x0)),(x1,f(x1)),,(xd,f(xd))(x_0, f(x_0)), (x_1, f(x_1)),…,(x_d, f(x_d)). Let’s call this representation point representation.

A more general version of the uniqueness theorem states that a polynomial of degree at most dd is uniquely determined by its values at at least d+1d + 1 distinct points. That means we could uniquely represent our polynomial using more than d+1d + 1 points.

Now, let’s represent our polynomials using the point representation, using the values x=5,4,3,2,1x = -5, -4, -3, -2, -1. Therefore, our polynomial can be represented as follows:

xxA(x)A(x)B(x)B(x)(x,A(x))(x, A(x))(x,B(x))(x, B(x))
5-548485656(5,48)(-5, 48)(5,56)(-5, 56)
4-435354242(4,35)(-4, 35)(4,42)(-4, 42)
3-324243030(3,24)(-3, 24)(3,30)(-3, 30)
2-215152020(2,15)(-2, 15)(2,20)(-2, 20)
1-1881212(1,8)(-1, 8)(1,12)(-1, 12)

Now, on to the interesting part. It turns out that C(xi)=A(xi)B(xi)C(x_i) =A(x_i) \ast B(x_i). That is, if you multiply the evaluation A(xi)A(x_i) by the evaluation B(xi)B(x_i), you get the evaluation C(xi)C(x_i). See the table below:

xxA(x)A(x)B(x)B(x)(x,A(x))(x, A(x))(x,B(x))(x, B(x))C(x)=A(x)B(x)C(x) = A(x) * B(x)(x,C(x))(x, C(x))
5-548485656(5,48)(-5, 48)(5,56)(-5, 56)26882688(5,2688)(-5, 2688)
4-435354242(4,35)(-4, 35)(4,42)(-4, 42)14701470(4,1470)(-4, 1470)
3-324243030(3,24)(-3, 24)(3,30)(-3, 30)720720(3,720)(-3, 720)
2-215152020(2,15)(-2, 15)(2,20)(-2, 20)300300(2,300)(-2, 300)
1-1881212(1,8)(-1, 8)(1,12)(-1, 12)9696(1,96)(-1, 96)

This means that the points (x,C(x))(x, C(x)) uniquely represents the polynomial C=18+39x+29x29x3+x4C = 18 + 39x + 29x^2 - 9x^3 + x^4.

What does this all mean? We just achieved an O(d)O(d) time polynomial multiplication by using the point representation form of the polynomials.

Does that mean we just achieved O(d)O(d) for polynomial multiplication? Not really.

Given two polynomials f(x)f(x) and g(x)g(x) of the form: a0+a1x+a2x2++adxda_0 + a_1x + a_2x^2 + \cdots + a_{d}x^{d}, here’s the full process:

  1. Convert from coefficient to point representation: we pick a set of xx values and evaluate the polynomials at those points thereby converting the polynomials to the point representation.

    That is, we pick xi={x0,x1,,xd}x_i = \lbrace x_0, x_1, … ,x_d \rbrace. Then, we compute f(xi)={f(x0),f(x1),,f(xd)}f(x_i) = \lbrace f(x_0), f(x_1),…,f(x_d) \rbrace and g(xi)={g(x0),g(x1),,g(xd)}g(x_i) = \lbrace g(x_0), g(x_1),…,g(x_d) \rbrace.

    There’s a way of expressing this in a matrix form:

    V=[x00x01x02x0nx10x11x12x1nx20x21x22x2nxn0xn1xn2xnn]=[1x01x02x0n1x11x12x1n1x21x22x2n1xn1xn2xnn] \begin{aligned} V &= \begin{bmatrix}x_0^0 & x_0^1 & x_0^2 & \cdots & x_0^{n} \\ x_1^0 & x_1^1 & x_1^2 & \cdots & x_1^{n} \\ x_2^0 & x_2^1 & x_2^2 & \cdots & x_2^{n} \\[0.3em]\vdots & \vdots & \vdots & \ddots & \vdots \\[0.3em]x_n^0 & x^1_{n} & x_{n}^2 & \cdots & x_{n}^{n}\end{bmatrix} \\[2pt] &= \begin{bmatrix}1 & x_0^1 & x_0^2 & \cdots & x_0^{n} \\ 1 & x_1^1 & x_1^2 & \cdots & x_1^{n} \\ 1 & x_2^1 & x_2^2 & \cdots & x_2^{n} \\[0.3em]\vdots & \vdots & \vdots & \ddots & \vdots \\[0.3em] 1 & x^1_{n} & x_{n}^2 & \cdots & x_{n}^{n}\end{bmatrix} \end{aligned}

    a=[a0a1a2an]a = \begin{bmatrix}a_0 \\ a_1 \\ a_2 \\ \vdots \\ a_n\end{bmatrix}

    Va=[a0x00+a1x01+a2x02++anx0na0x10+a1x11+a2x12++anx1na0x20+a1x21+a2x22++anx2na0xn0+a1xn1+a2xn2++anxnn]=[f(x0)f(x1)f(x2)f(xn)]\begin{aligned} Va &= \begin{bmatrix}a_0x_0^0 + a_1x_0^1 + a_2x_0^2 + \cdots + a_nx_0^n \\ a_0x_1^0 + a_1x_1^1 + a_2x_1^2 + \cdots + a_nx_1^n \\ a_0x_2^0 + a_1x_2^1 + a_2x_2^2 + \cdots + a_nx_2^n \\ \vdots \\ a_0x_n^0 + a_1x_n^1 + a_2x_n^2 + \cdots + a_nx_n^n \end{bmatrix} \\[2pt] &= \begin{bmatrix} f(x_0) \\ f(x_1) \\ f(x_2) \\ \vdots \\ f(x_n) \end{bmatrix} \end{aligned}

    Va=f(xi) Va = f(x_i)

    where nn is at least dd, VV is the matrix of xix_i and their powers, aa is a vector of coefficients and f(xi)f(x_i) are the evaluations. VV is called the vandermonde matrix.

    This shows that the process of evaluating the polynomial f(xi)f(x_i) is a matrix-vector multiplication which is an O(n2)O(n^2) operation. f(xi)f(x_i) can also be written as follows: f(xi)=j=0najxijf(x_i) = \sum^{n}_{j=0}a_jx^j_i

    Keep this in mind, as this formula for expressing f(xi)f(x_i) is important in understanding FFT later.

    Using A(x)A(x) as an example, choosing n=4n = 4 and x={x0,x1,x2,x3,x4}={5,4,3,2,1}x =\lbrace x_0, x_1, x_2, x_3, x_4 \rbrace= \lbrace-5, -4, -3, -2, -1 \rbrace, we have the following:

    V=[1x0x02x03x041x1x12x13x141x2x22x23x241x3x32x33x341x4x42x43x44]=[1552535414424344133233341222232411121314]=[1525125625141664256139278112481611111]\begin{aligned} V &= \begin{bmatrix}1 & x_0 & x_0^2 & x_0^3 & x_0^4 \\ 1 & x_1 & x_1^2 & x_1^3 & x_1^4 \\ 1 & x_2 & x_2^2 & x_2^3 & x_2^4 \\ 1 & x_3 & x_3^2 & x_3^3 & x_3^4 \\ 1 & x_4 & x_4^2 & x_4^3 & x_4^4 \end{bmatrix} \\[2pt] &= \begin{bmatrix}1 & -5 & -5^2 & -5^3 & -5^4 \\ 1 & -4 & -4^2 & -4^3 & -4^4 \\ 1 & -3 & -3^2 & -3^3 & -3^4 \\ 1 & -2 & -2^2 & -2^3 & -2^4 \\ 1 & -1 & -1^2 & -1^3 & -1^4 \end{bmatrix} \\[2pt] &= \begin{bmatrix}1 & -5 & 25 & -125 & 625 \\ 1 & -4 & 16 & -64 & 256 \\1 & -3 & 9 & -27 & 81 \\ 1 & -2 & 4 & -8 & 16 \\ 1 & -1 & 1 & -1 & 1\end{bmatrix} \end{aligned}

    a=[a0a1a2a3a4]=[34100]a = \begin{bmatrix}a_0 \\ a_1 \\ a_2 \\ a_3 \\ a_4\end{bmatrix} = \begin{bmatrix}3 \\ -4 \\ 1 \\ 0 \\ 0\end{bmatrix}

    Va=A(xi)=[A(x0)A(x1)A(x2)A(x3)A(x4)]=[483524158]Va = A(x_i) = \begin{bmatrix}A(x_0) \\ A(x_1) \\ A(x_2) \\ A(x_3) \\ A(x_4)\end{bmatrix} = \begin{bmatrix}48 \\ 35 \\ 24 \\ 15 \\ 8 \end{bmatrix}

  2. Multiply pairwise (i.e C(x)=A(x)B(x)C(x) = A(x) \ast B(x)). This runs in O(n)O(n) time.

  3. Convert back from the point represention to the coefficient representation: This is the reverse of step one. We want to find aa, given VV and f(xi)f(x_i). To compute this, we find the inverse of VV denoted by V1V^{-1} and multiply by f(xi)f(x_i). That is:

    Va=f(xi)Va = f(x_i)

    V1Va=V1f(xi)V^{-1}Va = V^{-1}f(x_i)

    a=V1f(xi)a = V^{-1}f(x_i)

    Using A(x)A(x) as an example:

    V1=[151010525/1261/639/2107/677/1235/2441/649/459/671/245/1211/6313/67/121/241/61/41/61/24]V^{-1} = \begin{bmatrix}1 & -5 & 10 & -10 & 5 \\ 25/12 & -61/6 & 39/2 & -107/6 & 77/12 \\ 35/24 & -41/6 & 49/4 & -59/6 & 71/24 \\ 5/12 & -11/6 & 3 & -13/6 & 7/12 \\ 1/24 & -1/6 & 1/4 & -1/6 & 1/24 \end{bmatrix}

    A(xi)=[483524158]A(x_i) = \begin{bmatrix}48 \\ 35 \\ 24 \\ 15 \\ 8 \end{bmatrix}

    a=V1A(xi)=[151010525/1261/639/2107/677/1235/2441/649/459/671/245/1211/6313/67/121/241/61/41/61/24][483524158]=[34100]a = V^{-1}A(x_i) = \begin{bmatrix}1 & -5 & 10 & -10 & 5 \\ 25/12 & -61/6 & 39/2 & -107/6 & 77/12 \\ 35/24 & -41/6 & 49/4 & -59/6 & 71/24 \\ 5/12 & -11/6 & 3 & -13/6 & 7/12 \\ 1/24 & -1/6 & 1/4 & -1/6 & 1/24 \end{bmatrix} \begin{bmatrix}48 \\ 35 \\ 24 \\ 15 \\ 8 \end{bmatrix} = \begin{bmatrix}3 \\ -4 \\ 1 \\ 0 \\ 0\end{bmatrix}

    The best version of this algorithm runs in O(n2)O(n^2) time.

In total, this process still takes O(n2)O(n^2) time.

But, what if there was actually a faster way? A way that allows us to do polynomial multiplication in less than O(n2)O(n^2).

Fast-fourier Transform (FFT) gives us that. FFT allows us to do step one and step three in O(nlogn)O(n\log n) time; making the whole process O(nlogn)O(n \log n) time.

To understand FFT, we need to understand the concept of complex numbers and roots of unity

Complex Numbers and Roots of Unity

The magic of FFT is based on the beautiful concept of roots of unity but that in turn is only made possible by the unique properties of complex numbers. So, here’s a deep dive on complex numbers and roots of unity.

Given the polynomial P(x)=1+x+x2P(x) = 1 + x + x^2, find the roots(i.e, the values of xx for which P(x)=0P(x) = 0) of the polynomial. The simple answer is the roots does not exist.

Using the quadratic formula x=b±b24ac2ax = \dfrac{-b \pm \sqrt{b^2 - 4ac}}{2a}, let’s solve anyways:

  • a=1a = 1, b=1b = 1 and c=1c = 1
  • x=1±124(1)(1)2(1)=1±32=12±32x = \dfrac{-1 \pm \sqrt{1^2 - 4(1)(1)}}{2(1)} = \dfrac{-1 \pm \sqrt{-3}}{2} = \dfrac{-1}{2} \pm \dfrac{\sqrt{-3}}{2}
    Therefore, the roots of P(x)P(x) are 12+32\dfrac{-1}{2} + \dfrac{\sqrt{-3}}{2} and 1232\dfrac{-1}{2} - \dfrac{\sqrt{-3}}{2}.

But, we previously said the polynomial didn’t have a root. How come we got two roots? The answer lies in the fact that from the beginning we have been dealing with real numbers R\mathbb{R} and the roots 12+32\dfrac{-1}{2} + \dfrac{\sqrt{-3}}{2} and 1232\dfrac{-1}{2} - \dfrac{\sqrt{-3}}{2} are not real numbers. Therefore, the roots does not exist in real space.

If the roots are not real numbers, then what are they? They are complex numbers!

Let’s further simplify the roots. 3\sqrt{-3} can be written 31\sqrt{3} * \sqrt{-1}. But, we know that 1\sqrt{-1} is impossible to find. It has a special name, the imaginary unit denoted as i\mathrm{i}.

We can rewrite our roots as follows: 12+3i2\dfrac{-1}{2} + \dfrac{\sqrt{3}\mathrm{i}}{2} and 123i2\dfrac{-1}{2} - \dfrac{\sqrt{3}\mathrm{i}}{2}. Therefore, a complex number C\mathbb{C} is a number of the form: a+bia + b\mathrm{i} where aa and bb are real numbers and i\mathrm{i} is the imaginary unit. aa is called the real part and bib\mathrm{i} is called the imaginary part. This form of representing a complex number is called the rectangular form. You’ll see why soon.

The set of real numbers R\mathbb{R} is a proper subset of the complex numbers C\mathbb{C} (i.e, RC\mathbb{R} \subset \mathbb{C}). Every real number can be written as a complex number where b=0b = 0 (e.g 5=5+0i5 = 5 + 0\mathrm{i}).

One way to describe a real number is as follows: a real number is any number that can be located on the infinite number line. That means, real numbers are represented on a number line as shown below.

png

How do we represent a complex number? The number line would have been a good option but it would only be able to take the real part. We need diagram that allows us to show both the real part and the imaginary part. That’s where the Argand diagram comes in!

The Argand diagram is two dimensional plane with the real part on the horinzontal line(x-axis) and the imaginary part on the vertical line (y-axis).

For example, the complex number 2+3i2 + 3\mathrm{i} is represented as follows:

png

Notice how the dotted lines from 22 and 33 forms a rectangle. That’s why the form a+bia + b\mathrm{i} is called the rectangular form.

Imagine we were given the diagram with just the line from the origin as shown below. Would we be able to identify the complex number?

png

Notice how the angle forms a right angle triangle. The diagram below should help.

png

What do we know about right angle triangles?

According to the Pythagoras Theorem, in a right angled triangle, the square of the length of the hypotenuse(the side opposite the right angle) is equal to the sum of the squares of the lengths of the other two sides(opposite and adjacent). Mathematically, if a right-angled triangle has sides aa and bb, and hypotenuse cc, then: c2=a2+b2c^2 = a^2 + b^2.

Having known this; let’s adjust out diagram.

png

From the diagram, we can see that we have two unknowns, the hypotenuse which will be denoted as rr and the angle θ\theta.

From pythagoras theorem, we know r2=a2+b2r^2 = a^2 + b^2. Therefore:r2=22+32=4+9=13r^2 = 2^2 + 3^2 = 4 + 9 = 13 r=13r = \sqrt{13}

Now we know rr, how do we find θ\theta? Do you remember the acronym SOHCAHTOA from high school? This implies: sinθ=oppositehypotenuse=br,b=rsinθ\sin\theta = \dfrac{\mathrm{opposite}}{\mathrm{hypotenuse}} = \dfrac{b}{r},\qquad b = r\sin\theta cosθ=adjacenthypotenuse=ar,a=rcosθ\cos\theta = \dfrac{\mathrm{adjacent}}{\mathrm{hypotenuse}} = \dfrac{a}{r},\qquad a = r\cos\theta tanθ=oppositeadjacent=ba,θ=tan1(ba)\tan\theta = \dfrac{\mathrm{opposite}}{\mathrm{adjacent}} = \dfrac{b}{a}, \qquad \theta = \tan^{-1}(\dfrac{b}{a})

From this, we can deduce that: a+bi=rcosθ+irsinθ=r(cosθ+isinθ)a + b\mathrm{i} = r\cos\theta + \mathrm{i}r\sin\theta = r(\cos\theta + \mathrm{i}\sin\theta). This form of representing a complex number is called the polar form.

Now we find θ\theta using tan1(ba)\tan^{-1}(\dfrac{b}{a}), θ=tan1(32)=0.983\quad \theta = \tan^{-1}(\dfrac{3}{2}) = 0.983 radians.

We have that r=13r = \sqrt{13} and θ=0.983\theta = 0.983. Given these two values, we represent the complex number 2+3i2 + 3\mathrm{i} as 13(cos(0.983)+isin(0.983))\sqrt{13}(\cos(0.983) + \mathrm{i}\sin(0.983)). 0.9830.983 radians is 56.356.3^\circ in degrees and it’s better for visualization as shown below.

The polar representation might not look like the simplest way to express a complex number but it’s going to help us understand roots of unity.

Formally, rr is the distance from the point of the complex number to the origin and it is called the modulus or the absolute value. θ\theta is called the argument of the complex number and it is the angle that the modulus rr makes from the positive real axis.

png

Let’s look at another representation of complex numbers.

Have you ever wondered how calculators computed trignometric functions like sin(x)\sin(x), cos(x)\cos(x) and the exponential function exe^x? The answer lies in approximation using power series.

cos(x)\cos(x) and sin(x)\sin(x) are approximated using the power series 1 x22!+x44!x66!+x88!+1\ -\frac{x^{2}}{2!} + \frac{x^{4}}{4!} - \frac{x^{6}}{6!} + \frac{x^{8}}{8!} + \cdots and x x33!+x55!x77!+x99!+x\ -\frac{x^{3}}{3!} + \frac{x^{5}}{5!} - \frac{x^{7}}{7!} + \frac{x^{9}}{9!} + \cdots respectively. I urge you to verify this using a graph(i.e compare their graphs against each other while extending the series).

Let’s substitute θ\theta for xx:

cos(θ)=1 θ22!+θ44!θ66!+θ88!+\cos(\theta) = 1\ - \frac{\theta^{2}}{2!} + \frac{\theta^{4}}{4!} - \frac{\theta^{6}}{6!} + \frac{\theta^{8}}{8!} + \cdots

sin(θ)=θ θ33!+θ55!θ77!+θ99!+\sin(\theta) = \theta\ - \frac{\theta^{3}}{3!} + \frac{\theta^{5}}{5!} - \frac{\theta^{7}}{7!} + \frac{\theta^{9}}{9!} + \cdots

Now, let’s subsitute this approximation into the polar form:

cosθ+isinθ=(1 θ22!+θ44!θ66!+θ88!+)+i(θ θ33!+θ55!θ77!+θ99!+)=(1 θ22!+θ44!θ66!+θ88!+)+(iθ iθ33!+iθ55!iθ77!+iθ99!+)=1 +iθθ22!iθ33!+θ44!+iθ55!θ66!iθ77!+θ88!+iθ99!+\begin{aligned} \cos\theta + \mathrm{i}\sin\theta &= (1\ - \frac{\theta^{2}}{2!} + \frac{\theta^{4}}{4!} - \frac{\theta^{6}}{6!} + \frac{\theta^{8}}{8!} + \cdots) + \mathrm{i}(\theta\ - \frac{\theta^{3}}{3!} + \frac{\theta^{5}}{5!} - \frac{\theta^{7}}{7!} + \frac{\theta^{9}}{9!} + \cdots) \\[6pt]&= (1\ - \frac{\theta^{2}}{2!} + \frac{\theta^{4}}{4!} - \frac{\theta^{6}}{6!} + \frac{\theta^{8}}{8!} + \cdots) + (\mathrm{i}\theta\ - \mathrm{i}\frac{\theta^{3}}{3!} + \mathrm{i}\frac{\theta^{5}}{5!} - \mathrm{i}\frac{\theta^{7}}{7!} + \mathrm{i}\frac{\theta^{9}}{9!} + \cdots) \\[6pt] &= 1\ + \mathrm{i}\theta - \frac{\theta^{2}}{2!} - \mathrm{i}\frac{\theta^{3}}{3!} + \frac{\theta^{4}}{4!} + \mathrm{i}\frac{\theta^{5}}{5!} - \frac{\theta^{6}}{6!} - \mathrm{i}\frac{\theta^{7}}{7!} + \frac{\theta^{8}}{8!} + \mathrm{i}\frac{\theta^{9}}{9!} + \cdots \end{aligned}

Let’s look at that of the exponential function exe^x. The power series for exe^x as follows: 1+x+x22!+x33!+x44!+x55!+x66!+x77!+x88!+x99!1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \frac{x^4}{4!} + \frac{x^5}{5!} + \frac{x^6}{6!} + \frac{x^7}{7!} + \frac{x^8}{8!} + \frac{x^9}{9!} \cdots

Let’s substitute iθ\mathrm{i}\theta for xx: eiθ=1+iθ+(iθ)22!+(iθ)33!+(iθ)44!+(iθ)55!+(iθ)66!+(iθ)77!+(iθ)88!+(iθ)99!+ e^{\mathrm{i}\theta} = 1 + \mathrm{i}\theta + \frac{(\mathrm{i}\theta)^2}{2!} + \frac{(\mathrm{i}\theta)^3}{3!} + \frac{(\mathrm{i}\theta)^4}{4!} + \frac{(\mathrm{i}\theta)^5}{5!} + \frac{(\mathrm{i}\theta)^6}{6!} + \frac{(\mathrm{i}\theta)^7}{7!} + \frac{(\mathrm{i}\theta)^8}{8!} + \frac{(\mathrm{i}\theta)^9}{9!} + \cdots

We will compute the powers of (iθ)(\mathrm{i}\theta). From indices we know that (iθ)n=θnin(\mathrm{i}\theta)^n = \theta^n\mathrm{i}^n. Let’s focus on in\mathrm{i}^n:

  • i1=i\mathrm{i}^1 = \mathrm{i}
  • i2=11=1\mathrm{i}^2 = \sqrt{-1} \ast \sqrt{-1} = -1
  • i3=111=11=i\mathrm{i}^3 = \sqrt{-1} \ast \sqrt{-1} \ast \sqrt{-1} = -1 \ast \sqrt{-1} = -\mathrm{i}
  • i4=1111=11=1\mathrm{i}^4 = \sqrt{-1} \ast \sqrt{-1} \ast \sqrt{-1} \ast \sqrt{-1} = -1 \ast -1 = 1
  • i5=i\mathrm{i}^5 = \mathrm{i}
  • i6=1\mathrm{i}^6 = -1
  • i7=i\mathrm{i}^7 = -\mathrm{i}
  • i8=1\mathrm{i}^8 = 1
  • i9=i\mathrm{i}^9 = \mathrm{i}

Do you see the pattern? The values are cyclic in nature; i\mathrm{i} to 11, it repeats and continues like that. We will come back to this but for now let’s go back to computing eiθe^{\mathrm{i}\theta}. It now becomes: eiθ=1+iθθ22!iθ33!+θ44!+iθ55!θ66!iθ77!+θ88!+iθ99!+e^{\mathrm{i}\theta} = 1 + \mathrm{i}\theta - \frac{\theta^2}{2!} - \mathrm{i}\frac{\theta^3}{3!} + \frac{\theta^4}{4!} + \mathrm{i}\frac{\theta^5}{5!} - \frac{\theta^6}{6!} - \mathrm{i}\frac{\theta^7}{7!} + \frac{\theta^8}{8!} + \mathrm{i}\frac{\theta^9}{9!} + \cdots

From this we can see that eiθ=cos(θ)+isin(θ)e^{\mathrm{i}\theta} = \cos(\theta) + \mathrm{i}\sin(\theta). Therefore, we have our third representation, the exponential form: reiθre^{\mathrm{i}\theta}.

In summary, we have:

  • rectangular form: a+bia + b\mathrm{i}
  • polar form: r(cosθ+isinθ)r(\cos\theta + \mathrm{i}\sin\theta)
  • exponential form: reiθre^{\mathrm{i}\theta}

Soon, you will see why exponential and polar form are best suited for understanding and working with roots of unity.

Now, back to where this it all started: finding the roots of the polynomial P(x)=1+x+x2P(x) = 1 + x + x^2. We saw that the roots are 12+3i2\dfrac{-1}{2} + \dfrac{\sqrt{3}\mathrm{i}}{2} and 123i2\dfrac{-1}{2} - \dfrac{\sqrt{3}\mathrm{i}}{2}.

Let’s represent them using the argand diagram.

png

Let’s find the modulus rr and the argument θ\theta of these roots:

  • For 12+3i2\dfrac{-1}{2} + \dfrac{\sqrt{3}\mathrm{i}}{2}:
    • finding rr: r2=a2+b2r^2 = a^2 + b^2, a=12\quad a = \dfrac{-1}{2}, b=32\quad b = \dfrac{\sqrt{3}}{2}, r2=(12)2+(32)2=14+34=44 =1\quad r^2 = (\dfrac{-1}{2})^2 + (\dfrac{\sqrt{3}}{2})^2 = \dfrac{1}{4} + \dfrac{3}{4} = \dfrac{4}{4} = 1, r=1\quad r = 1
    • finding θ\theta: θ=tan1(ba)=tan1(3212)=tan1(31)=tan1(3)=60\theta = \tan^{-1}(\dfrac{b}{a}) = \tan^{-1}(\dfrac{\dfrac{\sqrt{3}}{2}}{\dfrac{-1}{2}}) = \tan^{-1}(\dfrac{\sqrt{3}}{-1}) = \tan^{-1}(-\sqrt{3}) = -60^\circ or π3-\dfrac{\pi}{3}
  • For 123i2\dfrac{-1}{2} - \dfrac{\sqrt{3}\mathrm{i}}{2}:
    • finding rr: r2=a2+b2r^2 = a^2 + b^2, a=12\quad a = \dfrac{-1}{2}, b=32\quad b = \dfrac{-\sqrt{3}}{2}, r2=(12)2+(32)2=14+34=44 =1\quad r^2 = (\dfrac{-1}{2})^2 + (\dfrac{-\sqrt{3}}{2})^2 = \dfrac{1}{4} + \dfrac{3}{4} = \dfrac{4}{4} = 1, r=1\quad r = 1
    • finding θ\theta: θ=tan1(ba)=tan1(3212)=tan1(31)=tan1(3)=60\theta = \tan^{-1}(\dfrac{b}{a}) = \tan^{-1}(\dfrac{\dfrac{-\sqrt{3}}{2}}{\dfrac{-1}{2}}) = \tan^{-1}(\dfrac{-\sqrt{3}}{-1}) = \tan^{-1}(\sqrt{3}) = 60^\circ or π3\dfrac{\pi}{3}

Let’s show our rr and θ\theta values to our diagram.

png

This is our argand diagram represented in respect to rr and θ\theta. But, it’s not exactly correct.

Recall that the argument θ\theta is “the angle that the modulus makes from the positive real axis”. The key phrase is from the positive real axis.

Let’s show this:

png

As you can see, θ\theta for 12+3i2\dfrac{-1}{2} + \dfrac{\sqrt{3}\mathrm{i}}{2} becomes 120120^\circ or 2π3\dfrac{2\pi}{3} and θ\theta for 123i2\dfrac{-1}{2} - \dfrac{\sqrt{3}\mathrm{i}}{2} becomes 240240^\circ or 4π3\dfrac{4\pi}{3}.

In exponential form, this will be written as e2πi/3e^{2\pi\mathrm{i}/3} and e4πi/3e^{4\pi\mathrm{i}/3} respectively as r=1r = 1.

This also helps us see the argument θ\theta not just as angle but a rotation from the positive real axis and this is the mental model we need to understand roots of unity.

Let’s do a quick experiment: we are going take the roots individually and compute their powers up to the third power. Then, we are going to see what happens in the argand diagram.

Starting with e2πi/3e^{2\pi\mathrm{i}/3}:

  • We compute the zero power: (e2πi/3)0=e0=1(e^{2\pi\mathrm{i}/3})^0 = e^{0} = 1

png

  • The first power: (e2πi/3)1=e2πi/3(e^{2\pi\mathrm{i}/3})^1 = e^{2\pi\mathrm{i}/3}

png

  • The second power (e2πi/3)2=e4πi/3(e^{2\pi\mathrm{i}/3})^2 = e^{4\pi\mathrm{i}/3}

png

  • The third power (e2πi/3)3=e2πi(e^{2\pi\mathrm{i}/3})^3 = e^{2\pi\mathrm{i}}

png


Then, for e4πi/3e^{4\pi\mathrm{i}/3}:

  • We compute the zero power: (e4πi/3)0=e0=1(e^{4\pi\mathrm{i}/3})^0 = e^0 = 1

png

  • The first power: (e4πi/3)1=e4πi/3(e^{4\pi\mathrm{i}/3})^1 = e^{4\pi\mathrm{i}/3}

png

  • The second power: (e4πi/3)2=e8πi/3(e^{4\pi\mathrm{i}/3})^2 = e^{8\pi\mathrm{i}/3}

png

  • The third power: (e4πi/3)3=e4πi(e^{4\pi\mathrm{i}/3})^3 = e^{4\pi\mathrm{i}}

png


What do you notice?

The powers cycles back to 11 at the third power. If we continued this cycle, you would notice that it doesn’t just cycle back to 11 at the third power but every power that’s a multiple of 33. I urge you to try this out.

Let’s try this again for another polynomial.

Given the polynomial G(x)=x41G(x) = x^4 - 1, let’s the find the roots.

We get that:

  • G(x)=x41=(x21)(x2+1)=(x1)(x+1)(x2+1)G(x) = x^4 - 1 = (x^2 - 1)(x^2 + 1) = (x - 1)(x + 1)(x^2 + 1)

Setting them to zero, we have that:

  • (x1)=0(x - 1) = 0, x=1x = 1
  • (x+1)=0(x + 1) = 0, x=1x = -1
  • (x2+1)=0(x^2 + 1) = 0, x2=1x^2 = -1, x=±1=±ix = \pm\sqrt{-1} = \pm\mathrm{i}

Therefore, the roots of G(x)G(x) are 11, 1-1, i-\mathrm{i} and i\mathrm{i}

Let’s show them on the argand diagram.

png


Let’s compute the modulus rr and argument θ\theta for each of them.

  • For 11:
    • finding rr: r2=a2+b2r^2 = a^2 + b^2, a=1\quad a = 1, b=0\quad b = 0, r2=12+02=1\quad r^2 = 1^2 + 0^2 = 1, r=1\quad r = 1
    • finding θ\theta: θ=tan1(ba)=tan1(01)=tan1(0)=0=0\theta = \tan^{-1}(\dfrac{b}{a}) = \tan^{-1}(\dfrac{0}{1}) = \tan^{-1}(0) = 0 = 0^\circ or 0π0\pi
  • For 1-1:
    • finding rr: r2=a2+b2r^2 = a^2 + b^2, a=1\quad a = -1, b=0\quad b = 0, r2=12+02=1\quad r^2 = -1^2 + 0^2 = 1, r=1\quad r = 1
    • finding θ\theta: θ=tan1(ba)=tan1(01)=tan1(0)=0=0\theta = \tan^{-1}(\dfrac{b}{a}) = \tan^{-1}(\dfrac{0}{-1}) = \tan^{-1}(0) = 0 = 0^\circ. 00^\circ from the positive x-axis is 180180^\circ so θ=180\theta = 180^\circ or π\pi
  • For i\mathrm{i}:
    • finding rr: r2=a2+b2r^2 = a^2 + b^2, a=0\quad a = 0, b=1\quad b = 1, r2=02+12=1\quad r^2 = 0^2 + 1^2 = 1, r=1\quad r = 1
    • finding θ\theta: θ=tan1(ba)=tan1(10)=90\theta = \tan^{-1}(\dfrac{b}{a}) = \tan^{-1}(\dfrac{1}{0}) = 90^\circ or π2\dfrac{\pi}{2}. This looks like we are dividing 1 by 0 but trust me that’s not the case. I will leave you to verify this yourself.
  • For i-\mathrm{i}:
    • finding rr: r2=a2+b2r^2 = a^2 + b^2, a=0\quad a = 0, b=1\quad b = -1, r2=02+(1)2=1\quad r^2 = 0^2 + (-1)^2 = 1, r=1\quad r = 1
    • finding θ\theta: θ=tan1(ba)=tan1(10)=tan1(0)=90\theta = \tan^{-1}(\dfrac{b}{a}) = \tan^{-1}(\dfrac{-1}{0}) = \tan^{-1}(0) = -90^\circ. 90-90^\circ from the positive x-axis is 270270 so θ=270\theta = 270^\circ or 3π2\dfrac{3\pi}{2}

Converting them to exponential form, being that r=1r = 1 for all roots, we have that:

  • 1=e0i=e01 = e^{0\mathrm{i}} = e^0
  • 1=eπi-1 = e^{\pi\mathrm{i}}
  • i=eπi/2\mathrm{i} = e^{\pi\mathrm{i}/{2}}
  • i=e3πi/2-\mathrm{i} = e^{3\pi\mathrm{i}/{2}}

Now, let’s compute the powers of each root up to the fourth power.

Starting with e0e^0:

  • We compute the zero power (e0)0=e0(e^0)^0 = e^0. Notice that all the powers are equal as (e0)0=(e0)1=(e0)2=(e0)3=(e0)4=e0(e^0)^0 = (e^0)^1 = (e^0)^2 = (e^0)^3 = (e^0)^4 = e^0 so we have just one argand diagram showing the point without rotation.

png


For eπie^{\pi\mathrm{i}}:

  • We compute the zero power: (eπi)0=e0(e^{\pi\mathrm{i}})^0 = e^0

png

  • The first power: (eπi)1=eπi(e^{\pi\mathrm{i}})^1 = e^{\pi\mathrm{i}}

png

  • The second power: (eπi)2=e2πi(e^{\pi\mathrm{i}})^2 = e^{2\pi\mathrm{i}}

png

  • The third power: (eπi)3=e3πi(e^{\pi\mathrm{i}})^3 = e^{3\pi\mathrm{i}}

png

  • The fourth power: (eπi)4=e4πi(e^{\pi\mathrm{i}})^4 = e^{4\pi\mathrm{i}}

png


For eπi/2e^{\pi\mathrm{i}/{2}}:

  • We compute the zero power: (eπi/2)0=e0(e^{\pi\mathrm{i}/{2}})^0 = e^0

png

  • The first power: (eπi/2)1=eπi/2(e^{\pi\mathrm{i}/{2}})^1 = e^{\pi\mathrm{i}/{2}}

png

  • The second power: (eπi/2)2=eπi(e^{\pi\mathrm{i}/{2}})^2 = e^{\pi\mathrm{i}}

png

  • The third power: (eπi/2)3=e3πi/2(e^{\pi\mathrm{i}/{2}})^3 = e^{3\pi\mathrm{i}/2}

png

  • The fourth power: (eπi/2)4=e2πi(e^{\pi\mathrm{i}/{2}})^4 = e^{2\pi\mathrm{i}}

png


For e3πi/2e^{3\pi\mathrm{i}/{2}}:

  • We compute the zero power: (e3πi/2)0=e0(e^{3\pi\mathrm{i}/{2}})^0 = e^0

png

  • The first power: (e3πi/2)1=e3πi/2(e^{3\pi\mathrm{i}/{2}})^1 = e^{3\pi\mathrm{i}/{2}}

png

  • The second power: (e3πi/2)2=e3πi(e^{3\pi\mathrm{i}/{2}})^2 = e^{3\pi\mathrm{i}}

png

  • The third power: (e3πi/2)3=e9πi/2(e^{3\pi\mathrm{i}/{2}})^3 = e^{9\pi\mathrm{i}/2}

png

  • The fourth power: (e3πi/2)4=e6πi(e^{3\pi\mathrm{i}/{2}})^4 = e^{6\pi\mathrm{i}}

png


What do we notice?

  • For e0e^0:
    • They are no rotations. It just remains at 11
  • For eπie^{\pi\mathrm{i}}:
    • It rotates between 11 and 1-1 and it’s at point 11 at the powers divisible by 22(22 and 44).
  • For eπi/2e^{\pi\mathrm{i}/{2}}:
    • It rotates across all roots and meets 11 at the fourth power.
  • For e3πi/2e^{3\pi\mathrm{i}/{2}}:
    • Same as eπi/2e^{\pi\mathrm{i}/{2}}

Finally, what are roots of unity and why does all this matter?

The nnth roots are unity are simply values of xx that satisfy the function xn=1x^n = 1 for an arbitrary nn value.

Having known this, notice how the roots 11, 1-1, i\mathrm{i} and i-\mathrm{i} all satisfy function x4=1x^4 = 1. That is:

  • 14=11^4 = 1 and (e0)4=1(e^0)^4 = 1
  • 14=1-1^4 = 1 and (eπi)4=1(e^{\pi\mathrm{i}})^4 = 1
  • i4=1\mathrm{i}^4 = 1 and (eπi/2)4=e2πi=1(e^{\pi\mathrm{i}/{2}})^4 = e^{2\pi\mathrm{i}} = 1
  • (i)4=1(-\mathrm{i})^4 = 1 and (e3πi/2)4=e6πi=1(e^{3\pi\mathrm{i}/{2}})^4 = e^{6\pi\mathrm{i}} = 1

And, how x41=0x^4 - 1 = 0 is equals x4=1x^4 = 1.

These are 44th roots of unity.

But then, what about the polynomial P(x)=1+x+x2P(x) = 1 + x + x^2, does that hold for it too? Let’s find out.

Given the polynomial x31x^3 - 1, let’s find the roots.

  • (x31)=(x1)(1+x+x2)=(x1)P(x)(x^3 - 1) = (x - 1)(1 + x + x^2) = (x - 1)P(x)

Setting them to zero, we have that:

  • x1=0x - 1 = 0, x=1x = 1
  • P(x)=0P(x) = 0, we already know the roots are 12+3i2\dfrac{-1}{2} + \dfrac{\sqrt{3}\mathrm{i}}{2} and 123i2\dfrac{-1}{2} - \dfrac{\sqrt{3}\mathrm{i}}{2}.

Therefore, 11, 12+3i2\dfrac{-1}{2} + \dfrac{\sqrt{3}\mathrm{i}}{2} and 123i2\dfrac{-1}{2} - \dfrac{\sqrt{3}\mathrm{i}}{2} are the 33rd roots of unity. That is:

  • 13=11^3 = 1 and (e0)3=1(e^0)^3 = 1
  • (12+3i2)3=1(\dfrac{-1}{2} + \dfrac{\sqrt{3}\mathrm{i}}{2})^3 = 1 and (e2πi/3)3=e2πi=1(e^{2\pi\mathrm{i}/3})^3 = e^{2\pi\mathrm{i}} = 1
  • (123i2)3=1(\dfrac{-1}{2} - \dfrac{\sqrt{3}\mathrm{i}}{2})^3 = 1 and (e4πi/3)3=e4πi=1(e^{4\pi\mathrm{i}/3})^3 = e^{4\pi\mathrm{i}} = 1

Let’s extract more insights from our argand diagrams.

Notice how in the 33rd roots of unity, e2πi/3e^{2\pi\mathrm{i}/3} and e4πi/3e^{4\pi\mathrm{i}/3} generates every other root in the set and in the 44th roots of unity eπi/2e^{\pi\mathrm{i}/{2}} and e3πi/2e^{3\pi\mathrm{i}/{2}} also generates every other other roots in the set. For example,

  • For e2πi/3e^{2\pi\mathrm{i}/3} in the 33rd roots of unity:
    • (e2πi/3)0=1(e^{2\pi\mathrm{i}/3})^ 0 = 1
    • (e2πi/3)1=e2πi/3(e^{2\pi\mathrm{i}/3})^ 1 = e^{2\pi\mathrm{i}/3}
    • (e2πi/3)2=e4πi/3(e^{2\pi\mathrm{i}/3})^ 2 = e^{4\pi\mathrm{i}/3}
  • For e3πi/2e^{3\pi\mathrm{i}/{2}} in the 44th roots of unity:
    • (e3πi/2)0=1(e^{3\pi\mathrm{i}/{2}})^ 0 = 1
    • (e3πi/2)1=e3πi/2(e^{3\pi\mathrm{i}/{2}})^ 1 = e^{3\pi\mathrm{i}/{2}}
    • (e3πi/2)2=e3πi=eπi(e^{3\pi\mathrm{i}/{2}})^ 2 = e^{3\pi\mathrm{i}} = e^{\pi\mathrm{i}}
    • (e3πi/2)3=e9πi/2=eπi/2(e^{3\pi\mathrm{i}/{2}})^ 3 = e^{9\pi\mathrm{i}/{2}} = e^{\pi\mathrm{i}/{2}}

When a root generates all other roots in the set, it is called a primitive root of unity. This is evident in the diagrams as eπi/2e^{\pi\mathrm{i}/{2}} and e3πi/2e^{3\pi\mathrm{i}/{2}} hits every root of unity as it rotates around the circle.

There’s something that has been recurring ever since we started talking about roots of polynomials: the modulus rr has always been 11. This is not a concidence.

If you interpret rr as the radius of a circle being the unit circle(i.e a circle of radius 11) then every root of unity is a point on the unit circle hence the name roots of unity.

And, there’s a general formular for representing every nnth root of unity: e2πik/ne^{{2\pi\mathrm{i} \ast k}/n} where nn is an arbitrary value and k=0,1,,n1k = 0, 1, \cdots, n - 1.

For example:

  • The 55th roots of unity are {e2πi0/5,e2πi1/5,e2πi2/5,e2πi3/5,e2πi4/5}={e0,e2πi/5,e4πi/5,e6πi/5,e8πi/5}\lbrace e^{{2\pi\mathrm{i} \ast 0}/5}, e^{{2\pi\mathrm{i}\ast 1}/5}, e^{{2\pi\mathrm{i} \ast 2}/5}, e^{{2\pi\mathrm{i} \ast 3}/5}, e^{{2\pi\mathrm{i} \ast 4}/5} \rbrace = \lbrace e^0, e^{{2\pi\mathrm{i}}/5}, e^{{4\pi\mathrm{i}}/5}, e^{{6\pi\mathrm{i}}/5}, e^{{8\pi\mathrm{i}}/5} \rbrace

png

  • The 88th roots of unity are {e2πi0/8,e2πi1/8,e2πi2/8,e2πi3/8,e2πi4/8,e2πi5/8,e2πi6/8,e2πi7/8}={e0,eπi/4,eπi/2,e3πi/4,eπi,e5πi/4,e3πi/2,e7πi/4}\lbrace e^{{2\pi\mathrm{i} \ast 0}/8}, e^{{2\pi\mathrm{i} \ast 1}/8}, e^{{2\pi\mathrm{i} \ast 2}/8}, e^{{2\pi\mathrm{i} \ast 3}/8}, e^{{2\pi\mathrm{i} \ast 4}/8}, e^{{2\pi\mathrm{i} \ast 5}/8}, e^{{2\pi\mathrm{i} \ast 6}/8}, e^{{2\pi\mathrm{i} \ast 7}/8} \rbrace = \lbrace e^{0}, e^{{\pi\mathrm{i}}/4}, e^{{\pi\mathrm{i}}/2}, e^{{3\pi\mathrm{i}}/4}, e^{\pi\mathrm{i}}, e^{{5\pi\mathrm{i}}/4}, e^{{3\pi\mathrm{i}}/2}, e^{{7\pi\mathrm{i}}/4} \rbrace

png


A common way of representing roots of unity is in terms of the primitive root of unity. Recall that a primitive root of unity generates all other roots of unity.

Given a primitive root of unity ωn\omega_n, the roots of unity are {ωn0,ωn1,,ωnn1}\lbrace \omega_n^{0}, \omega_n^{1}, … , \omega_n^{n - 1} \rbrace where nn is an arbitrary positive number.

For example, from the 88th roots of unity, we pick the primitive root ω8=eπi/4\omega_8 = e^{{\pi\mathrm{i}}/4}. That 88th roots of unity are as follows: ω8k={ω80,ω81,ω82,ω83,ω84,ω85,ω86,ω87}\omega^k_8 = \lbrace \omega_8^0, \omega_8^1, \omega_8^2, \omega_8^3, \omega_8^4, \omega_8^5, \omega_8^6, \omega_8^7 \rbrace


Another important property of roots of unity is that every root has a complex conjugate and that complex conjugate is the point symmetric to it on the unit circle.

The complex conjugate of a complex number a+bia + b\mathrm{i} is abia - b\mathrm{i} and that of abia - b\mathrm{i} is a+bia + b\mathrm{i}. But then, when these are on the unit circle, the complex conjugate of any root is simply it’s vertical reflexion.

For example, from the 88th roots of unity above, Here are the complex conjugates:

  • e3πi/4e^{3\pi\mathrm{i}/4} is e5πi/4e^{5\pi\mathrm{i}/4}
  • eπi/4e^{\pi\mathrm{i}/4} is e7πi/4e^{7\pi\mathrm{i}/4}
  • eπi/2e^{\pi\mathrm{i}/2} is e3πi/2e^{3\pi\mathrm{i}/2}

Here are the 88th roots of unity in rectangular form for clarification: ω8k={e0,eπi/4,eπi/2,e3πi/4,eπi,e5πi/4,e3πi/2,e7πi/4}={1,22+i22,i,22+i22,1,22i22,i,22i22}\begin{aligned} \omega_8^k &= \lbrace e^{0}, e^{{\pi\mathrm{i}}/4}, e^{{\pi\mathrm{i}}/2}, e^{{3\pi\mathrm{i}}/4}, e^{\pi\mathrm{i}}, e^{{5\pi\mathrm{i}}/4}, e^{{3\pi\mathrm{i}}/2}, e^{{7\pi\mathrm{i}}/4} \rbrace \\[3pt] &= \lbrace 1, \dfrac{\sqrt{2}}{2} + \mathrm{i}\dfrac{\sqrt{2}}{2}, \mathrm{i}, -\dfrac{\sqrt{2}}{2} + \mathrm{i}\dfrac{\sqrt{2}}{2}, -1, -\dfrac{\sqrt{2}}{2} - \mathrm{i}\dfrac{\sqrt{2}}{2}, - \mathrm{i}, \dfrac{\sqrt{2}}{2} - \mathrm{i}\dfrac{\sqrt{2}}{2} \rbrace \end{aligned}

Interestingly, the complex conjugate of any root eaie^{a\mathrm{i}} is eaie^{-a\mathrm{i}}. That is:

  • e5πi/4=e5πi/4=e3πi/4e^{5\pi\mathrm{i}/4} = e^{-5\pi\mathrm{i}/4} = e^{3\pi\mathrm{i}/4}
  • e7πi/4=e7πi/4=eπi/4e^{7\pi\mathrm{i}/4} = e^{-7\pi\mathrm{i}/4} = e^{\pi\mathrm{i}/4}
  • e3πi/2=e3πi/2=eπi/2e^{3\pi\mathrm{i}/2} = e^{-3\pi\mathrm{i}/2} = e^{\pi\mathrm{i}/2}

A good mental model for complex conjugates is anticlockwise vs clockwise rotations. For example e3πi/4e^{3\pi\mathrm{i}/4} is the angle 3π/43\pi/4 in the anticlockwise direction while e5πi/4e^{5\pi\mathrm{i}/4} is the same angle in clockwise direction.

An even better way to interprete this is performing a rotation and undoing that rotation.

For example, in the diagram below, we have a value 22.

png

Let’s multiply 2 by e3πi/4e^{{3\pi\mathrm{i}}/4}. We have 2e3πi/42e^{{3\pi\mathrm{i}}/4}.

png

Now, let’s multiply 2e3πi/42e^{{3\pi\mathrm{i}}/4} by the complex conjugate of e3πi/4e^{{3\pi\mathrm{i}}/4}, e5πi/4e^{5\pi\mathrm{i}/4}. That is 2e3πi/4e5πi/4=2e2πi2e^{{3\pi\mathrm{i}}/4} * e^{5\pi\mathrm{i}/4} = 2e^{{2\pi\mathrm{i}}}.

png

As you can see in the diagrams e3πi/4e^{{3\pi\mathrm{i}}/4} rotated 22 by 3π/4{3\pi}/4 and e5πi/4e^{{5\pi\mathrm{i}}/4} undid that rotation taking 2e3πi/42e^{{3\pi\mathrm{i}}/4} back to 22.

This is a key concept in next our topic: Discrete Fourier Transform as we get closer to understanding FFT.

Discrete Fourier Transform (DFT)

Let’s go back to where it all started: Polynomial multiplication.

Recall, we chose a set of xx values, xi={x0,x1,,xn}x_i = \lbrace x_0, x_1,…, x_n \rbrace and evaluated f(xi)f(x_i) to get the set of points (x0,f(x0)),(x1,f(x1)),,(xd,f(xd))(x_0, f(x_0)), (x_1, f(x_1)),…,(x_d, f(x_d)) and we called it the point representation.

Now, let’s use roots of unit as those set xx values; our evaluation points. Given a primitive root of unity ωn\omega_n, we have our roots of unit wnk={ωn0,ωn1,ωn2,,ωnn1}w_n^k = \lbrace \omega_n^0, \omega_n^1, \omega_n^2, …, \omega_n^{n - 1} \rbrace.

With this, we have the following:

V=[ωn00ωn01ωn02ωn0(n1)ωn10ωn11ωn12ωn1(n1)ωn20ωn21ωn22ωn2(n1)ωn(n1)0ωn(n1)1ωn(n1)2ωn(n1)(n1)]=[ωn0ωn0ωn0ωn0ωn0ωn1ωn2ωnn1ωn0ωn2ωn4ωn2n2ωn0ωnn1ωn2n2ωnn22n+1]\begin{aligned} V &= \begin{bmatrix} \omega_n^{0 * 0} & \omega_n^{0 * 1} & \omega_n^{0 * 2} & \cdots & \omega_n^{0 * (n - 1)} \\[7pt] \omega_n^{1 * 0} & \omega_n^{1 * 1} & \omega_n^{1 * 2} & \cdots & \omega_n^{1 * (n - 1)} \\[7pt] \omega_n^{2 * 0} & \omega_n^{2 * 1} & \omega_n^{2 * 2} & \cdots & \omega_n^{2 * (n - 1)} \\[7pt] \vdots & \vdots & \vdots & \ddots & \vdots \\[7pt] \omega_n^{(n - 1)*0} & \omega_n^{(n - 1) * 1} & \omega_n^{(n - 1) * 2} & \cdots & \omega_n^{(n - 1)(n - 1)} \end{bmatrix} \\[7pt] &= \begin{bmatrix} \omega_n^0 & \omega_n^0 & \omega_n^0 & \cdots & \omega_n^0 \\[7pt] \omega_n^0 & \omega_n^1 & \omega_n^2 & \cdots & \omega_n^{n - 1} \\[7pt] \omega_n^0 & \omega_n^2 & \omega_n^4 & \cdots & \omega_n^{2n - 2} \\[7pt] \vdots & \vdots & \vdots & \ddots & \vdots \\[7pt] \omega_n^0 & \omega_n^{n - 1} & \omega_n^{2n - 2} & \cdots & \omega_n^{n^2 - 2n + 1} \end{bmatrix} \end{aligned}

a=[a0a1a2an1]a = \begin{bmatrix}a_0 \\[7pt] a_1 \\[7pt] a_2 \\[7pt] \vdots \\[7pt] a_{n - 1}\end{bmatrix}

Va=[a0ωn0+a1ωn0+a2ωn0++an1wn0a0ωn0+a1ωn1+a2ωn2++an1ωnn1a0ωn0+a1ωn2+a2ωn4++an1ωn2n2a0ωn0+a1ωnn1+a2ωn2n2++an1ωnn22n+1]=[f(ωn0)f(ωn1)f(ωn2)f(ωnn1)]=f(wnk)\begin{aligned} Va &= \begin{bmatrix} a_0\omega_n^0 + a_1\omega_n^0 + a_2\omega_n^0 + \cdots + a_{n - 1}w_n^0 \\[7pt] a_0\omega_n^0 + a_1\omega_n^1 + a_2\omega_n^2 + \cdots + a_{n - 1}\omega_n^{n - 1} \\[7pt] a_0\omega_n^0 + a_1 \omega_n^2 + a_2\omega_n^4 + \cdots + a_{n - 1} \omega_n^{2n - 2} \\[7pt] \vdots \\[7pt] a_0\omega_n^0 + a_1\omega_n^{n - 1} + a_2\omega_n^{2n - 2} + \cdots + a_{n - 1}\omega_n^{n^2 - 2n + 1} \end{bmatrix} \\[7pt] &= \begin{bmatrix} f(\omega_n^0) \\[7pt] f(\omega_n^1) \\[7pt] f(\omega_n^2) \\[7pt] \vdots \\[7pt] f(\omega_n^{n - 1}) \end{bmatrix} \\[7pt] &= f(w_n^k) \end{aligned}

Briefly, what is DFT? DFT is an algorithm that turns a signal from the time domain into the frequency domain. It used in Signal analysis, Image Compression and a lot more.

A good way to look at DFT is an algorithm that seeks to get more information from individual values that were created out of a combination of different measurements of values. For instance, given an chord(a chord is a set of notes played together), DFT can be used to get the frequency of the individual notes that makes up the chord.

Why do we care about DFT? The DFT algorithm is equalivent to polynomial evaluation using roots of unity as we shown above. It runs in O(n2)O(n^2) time and FFT is an optimization of DFT and it run in O(nlog(n))O(n\log(n)) time.

We are going to see how DFT is used and after that we are going to see how FFT is derived from the construction of DFT.

Let’s see an example. Using the 44th roots of unity, let’s compute C(x)=A(x)B(x)C(x) = A(x) * B(x) where A(x)=1+2xA(x) = 1 + 2x, B(x)=3+4xB(x) = 3 + 4x and C(x)=3+10x+8x2C(x) = 3 + 10x + 8x^2.

  • Step 1: Pick a primitive root of unity and define the the vandermonde matrix VV

ω4=e3πi/2\omega_4 = e^{{3\pi\mathrm{i}}/2}

ω4={ω40,ω41,ω42,ω43}={1,e3πi/2,e3πi,e9πi/2}\begin{aligned} \omega_4 &= \lbrace \omega_4^{0}, \omega_4^{1}, \omega_4^{2}, \omega_4^{3} \rbrace \\[2pt] &= \lbrace 1, e^{{3\pi\mathrm{i}}/2}, e^{3\pi\mathrm{i}}, e^{{9\pi\mathrm{i}}/2} \rbrace \end{aligned}

V=[ω400ω401ω402ω403ω410ω411ω412ω413ω420ω421ω422ω423ω430ω431ω432ω433]=[ω40ω40ω40ω40ω40ω41ω42ω43ω40ω42ω44ω46ω40ω43ω46ω49]=[11111e3πi/2e3πie9πi/21e3πie6πie9πi1e9πi/2e9πie27πi/2]\begin{aligned} V &= \begin{bmatrix} \omega_4^{0 * 0} & \omega_4^{0 * 1} & \omega_4^{0 * 2} & \omega_4^{0 * 3} \\[7pt] \omega_4^{1 * 0} & \omega_4^{1 * 1} & \omega_4^{1 * 2} & \omega_4^{1 * 3} \\[7pt] \omega_4^{2 * 0} & \omega_4^{2 * 1} & \omega_4^{2 * 2} & \omega_4^{2 * 3} \\[7pt] \omega_4^{3 * 0} & \omega_4^{3 * 1} & \omega_4^{3 * 2} & \omega_4^{3 * 3} \end{bmatrix} \\[7pt] &= \begin{bmatrix} \omega_4^{0} & \omega_4^{0} & \omega_4^{0} & \omega_4^{0} \\[7pt] \omega_4^{0} & \omega_4^{1} & \omega_4^{2} & \omega_4^{3} \\[7pt] \omega_4^{0} & \omega_4^{2} & \omega_4^{4} & \omega_4^{6} \\[7pt] \omega_4^{0} & \omega_4^{3} & \omega_4^{6} & \omega_4^{9} \end{bmatrix} \\[7pt] &= \begin{bmatrix} 1 & 1 & 1 & 1 \\[7pt] 1 & e^{{3\pi\mathrm{i}}/2} & e^{3\pi\mathrm{i}} & e^{{9\pi\mathrm{i}}/2} \\[7pt] 1 & e^{3\pi\mathrm{i}} & e^{6\pi\mathrm{i}} & e^{9\pi\mathrm{i}} \\[7pt] 1 & e^{{9\pi\mathrm{i}}/2} & e^{9\pi\mathrm{i}} & e^{{27\pi\mathrm{i}}/2} \end{bmatrix} \end{aligned}

  • Step 2: Compute the DFT of A(x)A(x) and B(x)B(x), DFT(A)\mathrm{DFT}(A) and DFT(B)\mathrm{DFT}(B) respectively.

    The formula for computing the DFT is X[k]=n=0N1x[n]ωNknX[k] = \sum_{n=0}^{N-1} x[n] \omega_N^{kn} where ωN=e2πi/N\omega_N = e^{-2\pi i / N} and NN is the NNth root of unity.

    This is the same as f(xi)=j=0najxijf(x_i) = \sum^{n}_{j=0}a_jx^j_i from earlier but updated to show that we are using the roots of unity as our evaluations points.

    DFT(A)DFT(A):

    a=[1200]a = \begin{bmatrix}1 \\ 2 \\ 0 \\ 0 \end{bmatrix}

    Va=[(13)+(14)(11)+(e3πi/22)(11)+(e3πi2)(11)+(e9πi/22)]=[71+2e3πi/21+2e3πi1+2e9πi/2]\begin{aligned} Va &= \begin{bmatrix} (1 \cdot 3) + (1 \cdot 4) \\[7pt] (1 \cdot 1) + (e^{{3\pi\mathrm{i}}/2} \cdot 2) \\[7pt] (1 \cdot 1) + (e^{3\pi\mathrm{i}} \cdot 2) \\[7pt] (1 \cdot 1) + (e^{{9\pi\mathrm{i}}/2} \cdot 2) \end{bmatrix} \\[7pt] &= \begin{bmatrix} 7 \\[7pt] 1 + 2e^{{3\pi\mathrm{i}}/2} \\[7pt] 1 + 2e^{3\pi\mathrm{i}} \\[7pt] 1 + 2e^{{9\pi\mathrm{i}}/2} \end{bmatrix} \end{aligned}

    DFT(B)DFT(B):

    a=[3400]a = \begin{bmatrix}3 \\ 4 \\ 0 \\ 0 \end{bmatrix}

    Va=[(13)+(14)(13)+(e3πi/24)(13)+(e3πi4)(13)+(e9πi/24)]=[73+4e3πi/23+4e3πi3+4e9πi/2]\begin{aligned} Va &= \begin{bmatrix} (1 \cdot 3) + (1 \cdot 4) \\[7pt] (1 \cdot 3) + (e^{{3\pi\mathrm{i}}/2} \cdot 4) \\[7pt] (1 \cdot 3) + (e^{3\pi\mathrm{i}} \cdot 4) \\[7pt] (1 \cdot 3) + (e^{{9\pi\mathrm{i}}/2} \cdot 4) \end{bmatrix} \\[7pt] &= \begin{bmatrix} 7 \\[7pt] 3 + 4e^{{3\pi\mathrm{i}}/2} \\[7pt] 3 + 4e^{3\pi\mathrm{i}} \\[7pt] 3 + 4e^{{9\pi\mathrm{i}}/2} \end{bmatrix} \end{aligned}

  • Step 3: Compute the pairwise multiplication DFT(A)DFT(B)DFT(A) \cdot DFT(B)

DFT(A)DFT(B)=[31+2e3πi/21+2e3πi1+2e9πi/2][73+4e3πi/23+4e3πi3+4e9πi/2]=[213+10e3πi/2+8e3πi3+10e3πi+8e6πi3+10e9πi/2+8e9πi]\begin{aligned} DFT(A) \cdot DFT(B) &= \begin{bmatrix} 3 \\[7pt] 1 + 2e^{{3\pi\mathrm{i}}/2} \\[7pt] 1 + 2e^{3\pi\mathrm{i}} \\[7pt] 1 + 2e^{{9\pi\mathrm{i}}/2} \end{bmatrix} \cdot \begin{bmatrix} 7 \\[7pt] 3 + 4e^{{3\pi\mathrm{i}}/2} \\[7pt] 3 + 4e^{3\pi\mathrm{i}} \\[7pt] 3 + 4e^{{9\pi\mathrm{i}}/2} \end{bmatrix} \\[7pt] &= \begin{bmatrix} 21 \\[7pt] 3 + 10e^{{3\pi\mathrm{i}}/2} + 8e^{3\pi\mathrm{i}} \\[7pt] 3 + 10e^{3\pi\mathrm{i}} + 8e^{6\pi\mathrm{i}} \\[7pt] 3 + 10e^{{9\pi\mathrm{i}}/2} + 8e^{9\pi\mathrm{i}} \end{bmatrix} \end{aligned}

  • Step 4: Compute the inverse DFT of DFT(A)DFT(B)DFT(A) \cdot DFT(B).

    This takes us back to coefficient form. The formula for this is IDFT(DFT(A)DFT(B))=x[n]=1Nk=0N1X[k]ωNknIDFT(DFT(A)\cdot DFT(B)) = x[n] = \frac{1}{N} \sum_{k=0}^{N-1} X[k] \omega_N^{-kn} This simply means we perform the same operation as before but now we pick the positive primitive root of unity and pick a=DFT(A).DFT(B)a = DFT(A).DFT(B), then we divide the resulting VaVa by N being 44 in this case.

    This formula is different from the one we used this process earlier because of the percularities of complex numbers and roots of unity but it’s requires less operations. For instance, we don’t need to find the inverse of the matrix VV.

    ω4=eπi/2\omega_4 = e^{{\pi\mathrm{i}}/2}

    ω4={ω40,ω41,ω42,ω43}={1,eπi/2,eπi,e3πi/2} \begin{aligned} \omega_4 &= \lbrace \omega_4^{0}, \omega_4^{1}, \omega_4^{2}, \omega_4^{3} \rbrace \\[2pt] &= \lbrace 1, e^{{\pi\mathrm{i}}/2}, e^{\pi\mathrm{i}}, e^{{3\pi\mathrm{i}}/2} \rbrace \end{aligned}

V=[ω40ω40ω40ω40ω40ω41ω42ω43ω40ω42ω44ω46ω40ω43ω46ω49]=[11111eπi/2eπie3πi/21eπie2πie3πi1e3πi/2e3πie9πi/2]\begin{aligned} V &= \begin{bmatrix} \omega_4^{0} & \omega_4^{0} & {\omega_4}^{0} & \omega_4^{0} \\[7pt] \omega_4^{0} & \omega_4^{1} & {\omega_4}^{2} & \omega_4^{3} \\[7pt] \omega_4^{0} & \omega_4^{2} & {\omega_4}^{4} & \omega_4^{6} \\[7pt] \omega_4^{0} & \omega_4^{3} & {\omega_4}^{6} & \omega_4^{9} \end{bmatrix} \\[7pt] &= \begin{bmatrix} 1 & 1 & 1 & 1 \\[7pt] 1 & e^{{\pi\mathrm{i}}/2} & e^{\pi\mathrm{i}} & e^{{3\pi\mathrm{i}}/2} \\[7pt] 1 & e^{\pi\mathrm{i}} & e^{2\pi\mathrm{i}} & e^{3\pi\mathrm{i}} \\[7pt] 1 & e^{{3\pi\mathrm{i}}/2} & e^{3\pi\mathrm{i}} & e^{{9\pi\mathrm{i}}/2} \end{bmatrix} \end{aligned}

a=[213+10e3πi/2+8e3πi3+10e3πi+8e6πi3+10e9πi/2+8e9πi]a = \begin{bmatrix} 21 \\[7pt] 3 + 10e^{{3\pi\mathrm{i}}/2} + 8e^{3\pi\mathrm{i}} \\[7pt] 3 + 10e^{3\pi\mathrm{i}} + 8e^{6\pi\mathrm{i}} \\[7pt] 3 + 10e^{{9\pi\mathrm{i}}/2} + 8e^{9\pi\mathrm{i}} \end{bmatrix}

Va=[11111eπi/2eπie3πi/21eπie2πie3πi1e3πi/2e3πie9πi/2][213+10e3πi/2+8e3πi3+10e3πi+8e6πi3+10e9πi/2+8e9πi]=[21+(3+10e3πi/2+8e3πi)+(3+10e3πi+8e6πi)+(3+10e9πi/2+8e9πi)21+(eπi/2(3+10e3πi/2+8e3πi))+(eπi(3+10e3πi+8e6πi))+(e3πi/2(3+10e9πi/2+8e9πi))21+(eπi(3+10e3πi/2+8e3πi))+(e2πi(3+10e3πi+8e6πi))+(e3πi(3+10e9πi/2+8e9πi))21+(e3πi/2(3+10e3πi/2+8e3πi))+(e3πi(3+10e3πi+8e6πi))+(e9πi/2(3+10e9πi/2+8e9πi))]=[1240320]\begin{aligned} Va &= \begin{bmatrix} 1 & 1 & 1 & 1 \\[7pt] 1 & e^{{\pi\mathrm{i}}/2} & e^{\pi\mathrm{i}} & e^{{3\pi\mathrm{i}}/2} \\[7pt] 1 & e^{\pi\mathrm{i}} & e^{2\pi\mathrm{i}} & e^{3\pi\mathrm{i}} \\[7pt] 1 & e^{{3\pi\mathrm{i}}/2} & e^{3\pi\mathrm{i}} & e^{{9\pi\mathrm{i}}/2} \end{bmatrix} \cdot \begin{bmatrix} 21 \\[7pt] 3 + 10e^{{3\pi\mathrm{i}}/2} + 8e^{3\pi\mathrm{i}} \\[7pt] 3 + 10e^{3\pi\mathrm{i}} + 8e^{6\pi\mathrm{i}} \\[7pt] 3 + 10e^{{9\pi\mathrm{i}}/2} + 8e^{9\pi\mathrm{i}} \end{bmatrix} \\[7pt] &= \begin{bmatrix} 21 + (3 + 10e^{{3\pi\mathrm{i}}/2} + 8e^{3\pi\mathrm{i}}) + (3 + 10e^{3\pi\mathrm{i}} + 8e^{6\pi\mathrm{i}}) + (3 + 10e^{{9\pi\mathrm{i}}/2} + 8e^{9\pi\mathrm{i}}) \\[7pt] 21 + (e^{{\pi\mathrm{i}}/2} \cdot (3 + 10e^{{3\pi\mathrm{i}}/2} + 8e^{3\pi\mathrm{i}})) + (e^{\pi\mathrm{i}} \cdot (3 + 10e^{3\pi\mathrm{i}} + 8e^{6\pi\mathrm{i}})) + (e^{{3\pi\mathrm{i}}/2} \cdot (3 + 10e^{{9\pi\mathrm{i}}/2} + 8e^{9\pi\mathrm{i}})) \\[7pt] 21 + (e^{\pi\mathrm{i}} \cdot (3 + 10e^{{3\pi\mathrm{i}}/2} + 8e^{3\pi\mathrm{i}})) + (e^{2\pi\mathrm{i}} \cdot (3 + 10e^{3\pi\mathrm{i}} + 8e^{6\pi\mathrm{i}})) + (e^{3\pi\mathrm{i}} \cdot (3 + 10e^{{9\pi\mathrm{i}}/2} + 8e^{9\pi\mathrm{i}})) \\[7pt] 21 + (e^{{3\pi\mathrm{i}}/2} \cdot (3 + 10e^{{3\pi\mathrm{i}}/2} + 8e^{3\pi\mathrm{i}})) + (e^{3\pi\mathrm{i}} \cdot (3 + 10e^{3\pi\mathrm{i}} + 8e^{6\pi\mathrm{i}})) + (e^{{9\pi\mathrm{i}}/2} \cdot (3 + 10e^{{9\pi\mathrm{i}}/2} + 8e^{9\pi\mathrm{i}})) \end{bmatrix} \\[7pt] &= \begin{bmatrix} 12 \\[7pt] 40 \\[7pt] 32 \\[7pt] 0 \end{bmatrix} \end{aligned}

IDFT(DFT(A)DFT(B))=1NVa=14Va=14[1240320]=[31080]\begin{aligned}IDFT(DFT(A)\cdot DFT(B)) &= \dfrac{1}{N}Va \\[2pt] &= \dfrac{1}{4}Va \\[2pt] &= \dfrac{1}{4} \cdot \begin{bmatrix} 12 \\[7pt] 40 \\[7pt] 32 \\[7pt] 0 \end{bmatrix} \\[7pt] &= \begin{bmatrix} 3 \\[7pt] 10 \\[7pt] 8 \\[7pt] 0 \end{bmatrix} \end{aligned}

To better understand VaVa, recall that e3πi/2e^{{3\pi\mathrm{i}}/2} is i-\mathrm{i}, e3πie^{3\pi\mathrm{i}} is 1-1 and e9πi/2e^{{9\pi\mathrm{i}}/2} is i\mathrm{i}

As you can see, we have successfully performed polynomial multiplication using DFT! Finally, let’s see how FFT works!

How FFT works

FFT is a recursive algorithm that makes DFT faster by exploiting two properties of roots of unity as follows:

  1. ωNk=ωNk+N/2-\omega_N^k = \omega_N^{k + N/2}. This is gotten from the fact that ωNN/2=1\omega_N^{N/2} = -1
  2. ωN2=ωN/2\omega_N^2 = \omega_{N/2}. For example, eπi/4e^{{\pi\mathrm{i}}/4} is a primitive root from the 88th of unity and (eπi/4)2(e^{{\pi\mathrm{i}}/4})^2 is eπi/2e^{{\pi\mathrm{i}}/2} being a primitive root from the 44th roots of unity.

As you probably guessed, these only holds when NN is even.

So far, we have explored DFT using matrices and matrix multiplication. But, to better understand FFT you need to view DFT in terms of the formulas X[k]=n=0N1x[n]ωNknX[k] = \sum_{n=0}^{N-1} x[n] \omega_N^{kn} and x[n]=1Nk=0N1X[k]ωNknx[n] = \frac{1}{N} \sum_{k=0}^{N-1} X[k] \omega_N^{-kn} so let’s do that.

Using same A(x)A(x) and B(x)B(x) from above, let’s compute DFTDFT and IDFTIDFT using the formulas

For DFT: X[k]=n=0N1x[n]ωNknX[k] = \sum_{n=0}^{N-1} x[n] \omega_N^{kn}

  • A(x)A(x):

    • X[0]=x[0]+x[1]+x[2]+x[3]=1+2+0+0=3X[0] = x[0] + x[1] + x[2] + x[3] = 1 + 2 + 0 + 0 = 3
    • X[1]=x[0]+x[1]ω4+x[2]ω42+x[3]ω43=1+2ω4X[1] = x[0] + x[1]\omega_4 + x[2]\omega_4^{2} + x[3]\omega_4^{3} = 1 + 2\omega_4
    • X[2]=x[0]+x[1]ω42+x[2]ω44+x[3]ω46=1+2ω42X[2] = x[0] + x[1]\omega_4^{2} + x[2]\omega_4^{4} + x[3]\omega_4^{6} = 1 + 2\omega_4^2
    • X[3]=x[0]+x[1]ω43+x[2]ω46+x[3]ω49=1+2ω43X[3] = x[0] + x[1]\omega_4^{3} + x[2]\omega_4^{6} + x[3]\omega_4^{9} = 1 + 2\omega_4^3
  • B(x)B(x):

    • X[0]=x[0]+x[1]+x[2]+x[3]=3+4+0+0=7X[0] = x[0] + x[1] + x[2] + x[3] = 3 + 4 + 0 + 0 = 7
    • X[1]=x[0]+x[1]ω4+x[2]ω42+x[3]ω43=3+4ω4X[1] = x[0] + x[1]\omega_4 + x[2]\omega_4^{2} + x[3]\omega_4^{3} = 3 + 4\omega_4
    • X[2]=x[0]+x[1]ω42+x[2]ω44+x[3]ω46=3+4ω42X[2] = x[0] + x[1]\omega_4^{2} + x[2]\omega_4^{4} + x[3]\omega_4^{6} = 3 + 4\omega_4^2
    • X[3]=x[0]+x[1]ω43+x[2]ω46+x[3]ω49=3+4ω43X[3] = x[0] + x[1]\omega_4^{3} + x[2]\omega_4^{6} + x[3]\omega_4^{9} = 3 + 4\omega_4^3

For IDFT: x[n]=1Nk=0N1X[k]ωNknx[n] = \frac{1}{N} \sum_{k=0}^{N-1} X[k] \omega_N^{-kn}

  • x[0]=14(X[0]+X[1]+X[2]+X[3])=3x[0] = \dfrac{1}{4}(X[0] + X[1] + X[2] + X[3]) = 3
  • x[1]=14(X[0]+X[1]ω41+X[2]ω42+X[3]ω43)=10x[1] = \dfrac{1}{4}(X[0] + X[1]\omega_4^{-1} + X[2]\omega_4^{-2} + X[3]\omega_4^{-3}) = 10
  • x[2]=14(X[0]+X[1]ω42+X[2]ω44+X[3]ω46)=8x[2] = \dfrac{1}{4}(X[0] + X[1]\omega_4^{-2} + X[2]\omega_4^{-4} + X[3]\omega_4^{-6}) = 8
  • x[3]=14(X[0]+X[1]ω43+X[2]ω46+X[3]ω49)=0x[3] = \dfrac{1}{4}(X[0] + X[1]\omega_4^{-3} + X[2]\omega_4^{-6} + X[3]\omega_4^{-9}) = 0

Having set the foundation, let’s look at the FFT algorithm step by step.

Starting from the DFT formula: X[k]=n=0N1x[n]ωNknX[k] = \sum_{n=0}^{N-1} x[n] \omega_N^{kn}

  • Step 1: We are going to split the formula into even/odd indices. Setting n=2mn = 2m and n=2m+1n = 2m + 1 so that: X[k]=m=0(N/2)1x[2m]ωNk(2m)+m=0(N/2)1x[2m+1]ωNk(2m+1)=m=0(N/2)1x[2m]ωN/2km+ωNkm=0(N/2)1x[2m+1]ωN/2km\begin{aligned} X[k] &= \sum_{m=0}^{(N/2)-1} x[2m] \omega_N^{k(2m)} + \sum_{m=0}^{(N/2)-1} x[2m + 1] \omega_N^{k(2m + 1)} \\ &= \sum_{m=0}^{(N/2)-1} x[2m] \omega_{N/2}^{km} + \omega_N^{k} \sum_{m=0}^{(N/2)-1} x[2m + 1] \omega_{N/2}^{km} \end{aligned}

    Notice the second property at play here.

  • Step 2: Let’s label the parts:E[k]=m=0(N/2)1x[2m]ωN/2kmE[k] = \sum_{m=0}^{(N/2)-1} x[2m] \omega_{N/2}^{km}O[k]=m=0(N/2)1x[2m+1]ωN/2kmO[k] = \sum_{m=0}^{(N/2)-1} x[2m + 1] \omega_{N/2}^{km} So that: X[k]=E[k]+ωNkO[k]X[k] = E[k] + \omega_N^{k}O[k]

  • Step 3: Let’s compute: X[k]=E[k]+ωNkO[k]X[k] = E[k] + \omega_N^{k}O[k] X[k+N/2]=E[k]+ωNk+N/2O[k]X[k + N/2] = E[k] + \omega_N^{k + N/2}O[k]From the first property, we can update the as follows:X[k]=E[k]+ωNkO[k]X[k] = E[k] + \omega_N^{k}O[k] X[k+N/2]=E[k]ωNkO[k]X[k + N/2] = E[k] - \omega_N^{k}O[k] So, we just have to compute E[k]E[k], O[k]O[k] and ωNk\omega_N^k and use it for both computations, flipping the sign on the second one.

  • Step 4: We perform same steps recursively for E[k]E[k] and O[k]O[k] until N=1N = 1.

Let’s go through an example using P(x)=1+2x+3x2+4x3P(x) = 1 + 2x + 3x^2 + 4x^3 using the 44th roots of unity.

xi={1,2,3,4}x_i = \lbrace 1, 2, 3, 4 \rbrace

We compute: X[0]=E[0]+ω40O[0]X[0] = E[0] + \omega_4^{0}O[0] X[2]=E[0]ω40O[0]X[2] = E[0] - \omega_4^0O[0] and X[1]=E[1]+ω4O[1]X[1] = E[1] + \omega_4O[1] X[3]=E[1]ω4O[1]X[3] = E[1] - \omega_4O[1]

To solve we just have to compute E[0]E[0], O[0]O[0], E[1]E[1] and O[1]O[1].

E[0]=m=01x[2m]ω20=m=01x[2m]=x[0]+x[2]=1+3=4\begin{aligned} E[0] &= \sum_{m=0}^{1} x[2m] \omega_{2}^{0} \\[2pt] &= \sum_{m=0}^{1} x[2m] \\[6pt] &= x[0] + x[2] \\[2pt] &= 1 + 3 \\[2pt] &= 4 \end{aligned}

O[0]=m=01x[2m]=x[1]+x[3]=2+4=6\begin{aligned} O[0] &= \sum_{m=0}^{1} x[2m] \\[2pt] &= x[1] + x[3] \\[2pt] &= 2 + 4 \\[2pt] &= 6 \end{aligned}

E[1]=m=01x[2m]ω2k=x[0]+x[2]ω2=1+(31)=2\begin{aligned} E[1] &= \sum_{m=0}^{1} x[2m]\omega_{2}^k \\[2pt] &= x[0] + x[2]\omega_{2} \\[2pt] &= 1 + (3\cdot-1) \\[2pt] &= -2 \end{aligned}

O[1]=m=01x[2m]ω2=x[1]+x[3]ω2=24=2\begin{aligned} O[1] &= \sum_{m=0}^{1} x[2m] \omega_{2} \\[2pt] &= x[1] + x[3]\omega_{2} \\[2pt] &= 2 - 4 \\[2pt] &= -2 \end{aligned}

Now, let’s use it in X[k]X[k]:

X[0]=E[0]+ω40O[0]=E[0]+O[0]=4+6=10\begin{aligned} X[0] &= E[0] + \omega_4^{0}O[0] \\[2pt] &= E[0] + O[0] \\[2pt] &= 4 + 6 \\[2pt] &= 10 \end{aligned}

X[2]=E[0]O[0]=46=2\begin{aligned} X[2] &= E[0] - O[0] \\[2pt] &= 4 - 6 \\[2pt] &= -2 \end{aligned}

X[1]=E[1]+ω4O[1]=E[1]+ω4O[1]=2+(ω42)=2+(e3πi/22)=22e3πi/2\begin{aligned} X[1] &= E[1] + \omega_4O[1] \\[2pt] &= E[1] + \omega_4O[1] \\[2pt] &= -2 + (\omega_4 \cdot -2) \\[2pt] &= -2 + (e^{{3\pi\mathrm{i}}/2} \cdot -2) \\[2pt] &= -2 - 2e^{{3\pi\mathrm{i}}/2} \end{aligned}

X[3]=E[1]ω4O[1]=2+2e3πi/2\begin{aligned} X[3] &= E[1] - \omega_4O[1] \\[2pt] &= -2 + 2e^{{3\pi\mathrm{i}}/2} \end{aligned}

In the same vein, let’s look into inverse FFT.

Starting from the inverse DFT formula: x[n]=1Nm=0N1X[m]ωNmnx[n] = \frac{1}{N} \sum_{m=0}^{N-1} X[m] \omega_N^{-mn}

x[n]=1N(m=0(N/2)1X[2m]ωNn(2m)+m=0(N/2)1X[2m+1]ωNn(2m+1))=1N(m=0(N/2)1X[2m]ωN/2nm+ωNnm=0(N/2)1X[2m+1]ωN/2nm)\begin{aligned}x[n] &= \dfrac{1}{N}(\sum_{m=0}^{(N/2)-1} X[2m] \omega_N^{-n(2m)} + \sum_{m=0}^{(N/2)-1} X[2m + 1] \omega_N^{-n(2m + 1)}) \\[2pt] &= \dfrac{1}{N}(\sum_{m=0}^{(N/2)-1} X[2m] \omega_{N/2}^{-nm} + \omega_{N}^{-n}\sum_{m=0}^{(N/2)-1} X[2m + 1] \omega_{N/2}^{-nm}) \end{aligned}

The even and odd parts:

E[n]=m=0(N/2)1x[2m]ωN/2nmE[n] = \sum_{m=0}^{(N/2)-1} x[2m] \omega_{N/2}^{-nm}

O[n]=m=0(N/2)1x[2m+1]ωN/2nmO[n] = \sum_{m=0}^{(N/2)-1} x[2m + 1] \omega_{N/2}^{-nm}

Then:

x[n]=1N(E[n]+ωNnO[n])x[n] = \dfrac{1}{N}(E[n] + \omega_N^{-n}O[n])

x[n+N/2]=1N(E[n]ωNnO[n])x[n + N/2] = \dfrac{1}{N}(E[n] - \omega_N^{-n}O[n])

Let’s continue from the last example. We got the DFT as {10,22e3πi/2,2,2+2e3πi/2}\lbrace 10, -2 - 2e^{{3\pi\mathrm{i}}/2}, -2, -2 + 2e^{{3\pi\mathrm{i}}/2} \rbrace. Let’s compute the inverse DFT using inverse FFT.

x[0]=14(E[0]+ω40O[0])x[0] = \dfrac{1}{4}(E[0] + \omega_4^{-0}O[0])

x[2]=14(E[0]ω40O[0])x[2] = \dfrac{1}{4}(E[0] - \omega_4^{-0}O[0]) and

x[1]=14(E[1]+ω41O[1])x[1] = \dfrac{1}{4}(E[1] + \omega_4^{-1}O[1])

x[3]=14(E[1]ω41O[1])x[3] = \dfrac{1}{4}(E[1] - \omega_4^{-1}O[1])

We will compute E[0]E[0], O[0]O[0], E[1]E[1] and O[1]O[1].

E[0]=m=01X[2m]=X[0]+X[2]=102=8\begin{aligned} E[0] &= \sum_{m=0}^{1} X[2m] \\[2pt] &= X[0] + X[2] \\[2pt] &= 10 - 2 \\[2pt] &= 8 \end{aligned}

O[0]=m=01X[2m]=X[1]+X[3]=22e3πi/22+2e3πi/2=4\begin{aligned} O[0] &= \sum_{m=0}^{1} X[2m] \\[2pt] &= X[1] + X[3] \\[2pt] &= -2 - 2e^{{3\pi\mathrm{i}}/2} -2 + 2e^{{3\pi\mathrm{i}}/2} \\[2pt] &= -4 \end{aligned}

E[1]=m=01X[2m]ω2m=X[0]+X[2]ω21=10+(21)=12\begin{aligned} E[1] &= \sum_{m=0}^{1} X[2m] \omega_{2}^{-m} \\[2pt] &= X[0] + X[2]\omega_{2}^{-1} \\[2pt] &= 10 + (-2\cdot-1) \\[2pt] &= 12 \end{aligned}

O[1]=m=01X[2m]ω2m=X[1]+X[3]ω21=22e3πi/2+(2+2e3πi/21)=22e3πi/2+22e3πi/2=4e3πi/2\begin{aligned} O[1] &= \sum_{m=0}^{1} X[2m] \omega_{2}^{-m} \\[2pt] &= X[1] + X[3]\omega_{2}^{-1} \\[2pt] &= -2 - 2e^{{3\pi\mathrm{i}}/2} + (-2 + 2e^{{3\pi\mathrm{i}}/2}\cdot -1) \\[2pt] &= -2 - 2e^{{3\pi\mathrm{i}}/2} + 2 - 2e^{{3\pi\mathrm{i}}/2} \\[2pt] &= -4e^{{3\pi\mathrm{i}}/2} \end{aligned}

Now, let’s use it in x[n]x[n]:

x[0]=14(E[0]+ω40O[0])=14(E[0]+O[0])=14(84)=14(4)=1\begin{aligned} x[0] &= \dfrac{1}{4}(E[0] + \omega_4^{0}O[0]) \\[2pt] &= \dfrac{1}{4}(E[0] + O[0]) \\[2pt] &= \dfrac{1}{4}(8 - 4) \\[2pt] &= \dfrac{1}{4}(4) \\[2pt] &= 1 \end{aligned}

x[2]=14(E[0]O[0])=14(8+4)=14(12)=3\begin{aligned} x[2] &= \dfrac{1}{4}(E[0] - O[0]) \\[2pt] &= \dfrac{1}{4}(8 + 4) \\[2pt] &= \dfrac{1}{4}(12) \\[2pt] &= 3 \end{aligned}

x[1]=14(E[1]+ω41O[1])=14(12+(ω414e3πi/2))=14(12+(eπi/24e3πi/2))=14(124)=14(8)=2\begin{aligned} x[1] &= \dfrac{1}{4}(E[1] + \omega_4^{-1}O[1]) \\[2pt] &= \dfrac{1}{4}(12 + (\omega_4^{-1} \cdot -4e^{{3\pi\mathrm{i}}/2})) \\[2pt] &= \dfrac{1}{4}(12 + (e^{{\pi\mathrm{i}}/2} \cdot -4e^{{3\pi\mathrm{i}}/2})) \\[2pt] &= \dfrac{1}{4}(12 - 4) \\[2pt] &= \dfrac{1}{4}(8) \\[2pt] &= 2 \end{aligned}

x[3]=14(E[1]ω41O[1])=14(12(ω414e3πi/2))=14(12(eπi/24e3πi/2))=14(12+4)=14(16)=4\begin{aligned} x[3] &= \dfrac{1}{4}(E[1] - \omega_4^{-1}O[1]) \\[2pt] &= -\dfrac{1}{4}(12 - (\omega_4^{-1} \cdot -4e^{{3\pi\mathrm{i}}/2})) \\[2pt] &= \dfrac{1}{4}(12 - (e^{{\pi\mathrm{i}}/2} \cdot -4e^{{3\pi\mathrm{i}}/2})) \\[2pt] &= \dfrac{1}{4}(12 + 4) \\[2pt] &= \dfrac{1}{4}(16) \\[2pt] &= 4 \end{aligned}

As you can see FFT and inverse FFT are the same algorithm with changes in the primitive root and division by NN at end.

Note that this only works when NN is power of 2. There are other versions of FFT that deal with other numbers but that’s out of the scope right now and this is recursive algorithm so we keep perform them on the even/odd parts until NN is 1.

Below is an implementation of both algorithms in python

def fft(x):
    N = len(x)
    if N == 1:
        return x

    wN = cmath.exp(-2j * cmath.pi / N)

    even = fft(x[0::2])
    odd  = fft(x[1::2])

    X = [0] * N
    w = 1
    for k in range(N // 2):
        t = w * odd[k]
        X[k] = even[k] + t
        X[k + N // 2] = even[k] - t
        w *= wN

    return X

def ifft(X):
    N = len(X)
    if N == 1:
        return X

    wN = cmath.exp(2j * cmath.pi / N)

    even = ifft(X[0::2])
    odd  = ifft(X[1::2])

    x = [0] * N
    w = 1
    for k in range(N // 2):
        t = w * odd[k]
        x[k] = even[k] + t
        x[k + N // 2] = even[k] - t
        w *= wN

    return [v / 2 for v in x]   # divide by 2 at each level → 1/N total

x = [1, 2, 3, 4]
X = fft(x)
xr = ifft(X)

print(X)
print(xr)

Lastly, let’s briefly look at Number Theoritic Transform and how it relates to FFT.

Number Theoritic Transform (NTT)

NTT is simply FFT over modular arithmetic (no complex numbers). We use polynomial multiplication a lot in cryptography so we need FFT and we also work with large numbers but what you notice is that the higher up we go in complex roots of unity you start dealing with small decimal numbers and decimals numbers are not suitable for cryptography. We need certainty and precision but decimal numbers don’t offer that. So, NTT is just an adaptation of FFT to modular arithmetic; finite fields to be precise.

A finite field is a field with finite elements and a field is a set where you add, subtract, multiply and divide by any non-zero element.

An example of a finite field is the set of integers mod\bmod 11 Z11={0,1,2,3,4,5,6,7,8,9,10}\mathbb{Z}_{11} = \lbrace 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 \rbrace

  • Addition: 3+25mod11=63 + 25 \bmod 11 = 6
  • Subtraction: 323=20mod11=23 - 23 = -20 \bmod 11 = 2. What we actually did here is we found the number xx such that 20+xmod11=020 + x \bmod 11 = 0 and it’s called the additive inverse.
  • Multiplication: 3×20mod11=53 \times 20 \bmod 11 = 5
  • Division: 5/2mod11=85 / 2 \bmod 11 = 8. Similar to subtraction, what we did here was find the number xx such 2×xmod11=12 \times x \bmod 11 = 1 and then we multiplied xx by 5. xx is 66 in case. xx is the multiplicative inverse of 22 and denoted at 212^{-1}. Keep this in mind.

You can look at mod\bmod as clock arithmetic. No matter how many times you go around the clock, the time would always be from 00 to 1111. That’s the same here. Every operation you perform gives you a result in the set.

Now, let’s look at how NTT works.

The formulas are as follows:

NTT=X[k]=n=0N1x[n]ωkn(modq)NTT = X[k] = \sum_{n = 0}^{N - 1}x[n]\omega^{kn} (\bmod q)

INTT=x[n]=N1k=0N1X[k]ωkn(modq)INTT = x[n] = N^{-1}\sum_{k = 0}^{N - 1}X[k]\omega^{-kn} (\bmod q)

where

  • qq is prime
  • ω\omega is the primitive root in the NN-root of unity mod\bmod qq
  • ωN1(modq)\omega^N \equiv 1 (\bmod q)
  • q1q - 1 is divisible by NN and NN is a power of 2.

Let’s take an example

  • N=4N = 4
  • q=17q = 17
  • ω=13\omega = 13
  • ωk={ω0,ω1,ω2,ω3}={130,131,132,133}=={1,13,16,4}\omega^k = \lbrace \omega^0, \omega^1, \omega^2, \omega^3 \rbrace = \lbrace 13^0, 13^1, 13^2, 13^3 \rbrace = = \lbrace 1, 13, 16, 4 \rbrace
  • P(x)=1+2x+3x2+4x3P(x) = 1 + 2x + 3x^2 + 4x^3

NTT:

  • X[0]=x[0]+x[1]+x[2]+x[3]=1+2+3+4(mod17)=10X[0] = x[0] + x[1] + x[2] + x[3] = 1 + 2 + 3 + 4 (\bmod 17) = 10
  • X[1]=x[0]+x[1]ω+x[2]ω2+x[3]ω3=(1+(213)+(316)+(44))mod17=6X[1] = x[0] + x[1]\omega + x[2]\omega^2 + x[3]\omega^3 = (1 + (2\cdot13) + (3\cdot16) + (4\cdot4)) \bmod 17 = 6
  • X[2]=x[0]+x[1]ω2+x[2]ω4+x[3]ω6=(1+(216)+(31)+(416))mod17=15X[2] = x[0] + x[1]\omega^2 + x[2]\omega^4 + x[3]\omega^6 = (1 + (2 \cdot 16) + (3 \cdot 1) + (4 \cdot 16)) \bmod 17 = 15
  • X[3]=x[0]+x[1]ω3+x[2]ω6+x[3]ω9=(1+(24)+(316)+(413))mod17=7X[3] = x[0] + x[1]\omega^3 + x[2]\omega^6 + x[3]\omega^9 = (1 + (2 \cdot 4) + (3 \cdot 16) + (4 \cdot 13)) \bmod 17 = 7

INTT:

  • x[0]=13(X[0]+X[1]+X[2]+X[3])=13(10+6+15+7)mod17=1x[0] = 13(X[0] + X[1] + X[2] + X[3]) = 13(10 + 6 + 15 + 7) \bmod 17 = 1
  • x[1]=13(X[0]+X[1]ω1+X[2]ω2+X[3]ω3)=13(10+(64)+(1516)+(713))mod17=2x[1] = 13(X[0] + X[1]\omega^{-1} + X[2]\omega^{-2} + X[3]\omega^{-3}) = 13(10 + (6 \cdot 4) + (15 \cdot 16) + (7 \cdot 13)) \bmod 17 = 2
  • x[2]=13(X[0]+X[1]ω2+X[2]ω4+X[3]ω6)=13(10+(616)+(151)+(716))mod17=3x[2] = 13(X[0] + X[1]\omega^{-2} + X[2]\omega^{-4} + X[3]\omega^{-6}) = 13(10 + (6 \cdot 16) + (15 \cdot 1) + (7 \cdot 16)) \bmod 17 = 3
  • x[3]=13(X[0]+X[1]ω3+X[2]ω6+X[3]ω9)=13(10+(613)+(1516)+(74))mod17=4x[3] = 13(X[0] + X[1]\omega^{-3} + X[2]\omega^{-6} + X[3]\omega^{-9}) = 13(10 + (6 \cdot 13) + (15 \cdot 16) + (7 \cdot 4)) \bmod 17 = 4

Notice how ωk\omega^k cycles back to 11 at k=4k = 4 and continues like that. This is not a coincidence. It’s cyclic in nature and is part of the structure that makes NTT possible. Also, this works for N=2,8,16N = 2, 8, 16 with there distinct respective primitive root ω\omega.

Lastly, our example is just DFT translated to NTT. It’s still runs in O(n2)O(n^2) time. As an exercise, apply FFT on this.


I understand this is quite a lot to take in so I advice you follow at your pace and go through as many times as you need. Feel free to ask questions in the comments too!

In part two, we will be talking more about NTT and how it’s used in lattice-based algorithms like ML-KEM and DSA. See you there!

Buy Me A Coffee