🎓How I Study AIHISA
📖Read
📄Papers📰Blogs🎬Courses
💡Learn
đŸ›€ïžPaths📚Topics💡Concepts🎮Shorts
🎯Practice
⏱CoachđŸ§©Problems🧠Thinking🎯Prompts🧠Review
SearchSettings
How I Study AI - Learn AI Papers & Lectures the Easy Way
⚙AlgorithmIntermediate

Discrete Fourier Transform (DFT) & FFT

Key Points

  • ‱
    The Discrete Fourier Transform (DFT) converts a length-N sequence from the time (or spatial) domain into N complex frequency coefficients that describe how much of each sinusoid is present.
  • ‱
    The Fast Fourier Transform (FFT) is an algorithm family (not a different transform) that computes the same DFT results in O(N log N) time instead of O(N2).
  • ‱
    You can recover the original signal exactly using the inverse DFT (or IFFT) when you use consistent scaling and sufficient numerical precision.
  • ‱
    FFT enables fast convolution: multiply in frequency domain to compute linear convolution or polynomial multiplication much faster.
  • ‱
    Real-world analogy: DFT is like a prism splitting white light into colors; FFT is a clever way to do the same splitting very quickly.
  • ‱
    Radix-2 Cooley–Tukey FFT works best when N is a power of two; other sizes need mixed radix or Bluestein’s algorithm.
  • ‱
    Numerical pitfalls include sign/scaling conventions, zero-padding for linear convolution, and rounding errors from floating-point arithmetic.
  • ‱
    In C++, std::complex<double> with precomputed roots of unity and bit-reversal ordering is standard for implementing an efficient iterative FFT.

Prerequisites

  • →Complex numbers and Euler’s formula — DFT uses complex exponentials e^{i\theta} = cos\theta + i sin\theta to represent sinusoids.
  • →Big-O notation and algorithm analysis — Understanding why FFT is faster requires asymptotic analysis of O(N^2) versus O(N \log N).
  • →Recursion and divide-and-conquer — Cooley–Tukey FFT splits the problem recursively into subproblems.
  • →Convolution and polynomial multiplication — A key application of FFT is accelerating convolution and polynomial products.
  • →Numerical precision and floating point — FFT-based results require careful rounding and awareness of floating-point error.
  • →Basic linear algebra — DFT can be expressed as matrix multiplication with the Fourier matrix and its adjoint.

Detailed Explanation

Tap terms for definitions

01Overview

Hook: Imagine standing in a concert hall where a single chord sounds rich and complex. You ask: which notes (frequencies) and how strong is each? The Discrete Fourier Transform (DFT) answers exactly that for digital sequences. Concept: The DFT takes N equally spaced samples and expresses them as a weighted sum of N discrete sinusoids. These weights are complex numbers capturing both magnitude (how strong) and phase (where the peaks are). While the definition of DFT is straightforward, directly computing it takes O(N^2) operations because each output depends on all inputs. Example: If you record a short audio clip with N samples, the DFT reveals its pitch content at N frequency bins. To make this practical at modern data sizes, we rely on the Fast Fourier Transform (FFT) algorithms—especially the Cooley–Tukey approach—which exploit symmetries of the complex roots of unity to reduce work to O(N \log N). The FFT family includes variants for different radices, real-only input optimization, and non–power-of-two lengths. Together, DFT and FFT underpin signal processing, image analysis (2D DFT), fast polynomial multiplication, correlation, filtering, solving PDEs, and more.

02Intuition & Analogies

Hook: A prism splits white light into its component colors. The DFT/FFT does the same for signals: it splits a sequence into pure tones (sines and cosines). Concept: Any finite discrete signal can be thought of as a blend of N basic waves whose frequencies fit perfectly in N samples. Each basic wave is like a pure color. The DFT tells you how much of each basic wave you have (magnitude) and when its peaks occur relative to the signal (phase). Why does FFT help? Picture folding a paper pattern with repeating motifs: many parts match and can be reused. The FFT finds and reuses repeated multiplications by the same complex angles (roots of unity), combining pairs of subproblems (even and odd elements) so no effort is wasted. Everyday analogy: Sorting a deck naively is like comparing every card with every other (O(N^2)). But if you split the deck, sort halves, and merge cleverly, it’s much faster (O(N \log N)). Similarly, FFT splits the DFT into smaller DFTs of evens and odds (or other radices) and merges their results via simple two-term “butterflies.” Example: Suppose your signal is the sum of two pure tones at 3 Hz and 7 Hz sampled 32 times per second. The DFT outcomes will be large near bins k=3 and k=7 (and their symmetric bins) while others are near zero. With FFT you detect these tones quickly even for very long signals.

03Formal Definition

Hook: We want a precise recipe that maps N samples to N frequency coefficients and back with no information loss. Concept: For a sequence x[0], x[1], ..., x[N−1], the N-point DFT is X[k] = ∑n=0N−1​ x[n] e−i2πkn/N for k=0,1,...,N−1. The inverse DFT (IDFT) reconstructs x from X via x[n] = N1​ ∑k=0N−1​ X[k] ei2πkn/N. In matrix form, let FN​ be the N×N Fourier matrix with entries (FN​)_{k,n} = e−i2πkn/N; then X=FN​ x and x = N1​ FN​^∗ X, where FN​^∗ is the conjugate transpose. The vectors of complex exponentials form an orthogonal (in fact, orthonormal up to scaling) basis with respect to the standard inner product. FFT is a family of algorithms that compute X efficiently by factoring FN​ into sparse structured matrices using properties of the roots of unity (e.g., radix-2 Cooley–Tukey factors F2m​ into smaller Fm​ blocks plus diagonal/permutation matrices). Example: With N=8, the radix-2 FFT computes two 4-point DFTs (evens and odds), multiplies the odd part by twiddle factors e−i2πk/8, and forms 8 outputs with simple additions and subtractions—the classic butterfly operations.

04When to Use

Hook: Use DFT/FFT whenever you need to peek at or manipulate the frequency content of discrete data or to accelerate convolution-like operations. Concept: Choose DFT (via FFT) for spectral analysis of signals (audio, vibration, ECG), designing and applying digital filters (low-pass, high-pass) by zeroing or shaping frequency bins, computing correlations/cross-correlations, and for efficient polynomial multiplication (treat coefficients as samples, convolve to multiply). FFT shines when N is large enough that O(N^2) is prohibitive; beyond a few hundred points, FFT’s O(N \log N) is usually far faster. For linear convolution of sequences of lengths N and M, use FFT with zero-padding to length L ≄ N+M−1 to avoid circular wrap-around. If N is a power of two, radix-2 iterative FFT is straightforward and fast; otherwise consider mixed-radix, radix-3/5 splits, or Bluestein’s algorithm (chirp-z) to handle prime lengths by mapping to a convolution. Example: Multiplying two degree-50,000 polynomials with FFT-based convolution can be hundreds of times faster than the naive O(n^2) schoolbook method and is standard in competitive programming and scientific computing.

⚠Common Mistakes

Hook: Most bugs in FFT code come from tiny sign, scale, or indexing slips that completely scramble results. Concept: 1) Sign convention: The forward DFT uses e^{-i 2\pi kn/N} and the inverse uses e^{+i 2\pi kn/N} with a 1/N factor in the inverse (or split as 1/\sqrt{N} each). Mixing signs or forgetting the scale yields mirrored or scaled outputs. 2) Off-by-one indices: Bins run k=0..N−1 and n=0..N−1. Accidentally looping to N or starting at 1 breaks orthogonality. 3) Circular vs. linear convolution: FFT multiplication without zero-padding computes circular convolution; for linear convolution, zero-pad to at least N+M−1. 4) Power-of-two assumption: Radix-2 FFT requires N to be a power of two; otherwise bit-reversal indexing and twiddle reuse fail. 5) Bit-reversal errors: Incorrect bit width or in-place swaps corrupt the permutation. 6) Precision: Using float instead of double increases leakage and reconstruction error; rounding results after IFFT is necessary for integer polynomial multiplication. 7) Windowing/aliasing: For spectral analysis of finite records, window the data to reduce spectral leakage and respect the Nyquist limit to avoid aliasing. Example: If you multiply polynomials but forget to round after IFFT, coefficients like 5.0000001 can print as 5.0000001 or even 4.9999999 due to floating-point noise.

Key Formulas

DFT Definition

X[k]=n=0∑N−1​x[n]e−i2πkn/N

Explanation: This maps N samples x[n] to N frequency bins X[k]. The exponential encodes complex sinusoids; each X[k] is the projection onto the k-th basis wave.

Inverse DFT (IDFT)

x[n]=N1​k=0∑N−1​X[k]ei2πkn/N

Explanation: This reconstructs the original sequence from its frequency coefficients. The 1/N factor ensures energy and amplitude are properly scaled back.

Twiddle Factor

WN​=e−i2π/N,WNkn​=e−i2πkn/N

Explanation: WN​ is the principal N-th root of unity. Powers of WN​ provide all rotations used in DFT/FFT computations.

Radix-2 FFT Recurrence

T(N)=2T(N/2)+O(N)

Explanation: The N-point FFT does two N/2-point FFTs and combines them in linear time. Solving the recurrence gives O(N log N) total time.

FFT Complexity

T(N)=O(NlogN)

Explanation: The total number of arithmetic operations grows proportional to N times the number of stages (log2​ N). This is much faster than O(N2) for large N.

Orthogonality of Roots of Unity

n=0∑N−1​e−i2π(k−ℓ)n/N={N,0,​k≡ℓ (mod N)otherwise​

Explanation: Distinct DFT basis vectors are orthogonal; the inner product is zero unless the frequencies match. This property underlies invertibility and energy separation.

Convolution Theorem (Frequency Domain)

F{x∗y}[k]=X[k]⋅Y[k]

Explanation: Convolution in time corresponds to pointwise multiplication in frequency. This allows fast convolution via FFTs and one inverse FFT.

Linear Convolution

z[n]=m=0∑n​x[m]y[n−m],0≀n≀N+M−2

Explanation: This defines the desired non-wrapping convolution of two finite sequences. Zero-padding to length L ≄ N+M-1 makes FFT-based multiplication compute this exactly.

Euler’s Formula

eiΞ=cosΞ+isinΞ

Explanation: It links complex exponentials to sinusoids, showing why complex exponentials represent oscillations. DFT’s exponentials are concise versions of sine/cosine components.

Matrix Form of DFT

X=FN​x,FN−1​=N1​FN∗​

Explanation: The DFT is a linear transform given by multiplication by the Fourier matrix. The inverse is its conjugate transpose scaled by 1/N.

Nyquist Frequency

fNyquist​=2fs​​

Explanation: The highest frequency that can be represented without aliasing is half the sampling rate. Bins above this fold back into lower frequencies.

Periodicity

X[k+N]=X[k],x[n+N]=x[n]

Explanation: Both the DFT coefficients and the underlying basis are periodic with period N. This explains circular wrap-around in convolution without padding.

Complexity Analysis

A direct DFT computes N outputs, each as a sum of N terms, giving O(N2) time and O(1) extra space (beyond input/output). This becomes expensive quickly: for N=1,000,000, a naive DFT requires roughly 10^12 complex multiplies/adds, which is impractical. FFT algorithms reduce the work by recursively exploiting the periodicity and symmetry of the N-th roots of unity. In radix-2 Cooley–Tukey, each stage combines pairs of values (butterflies) and there are log2 N stages, yielding O(N log N) time. Memory usage is O(N) because the transform can be performed in-place with a bit-reversal permutation and stage-wise butterflies. Precomputing twiddle factors also costs O(N) time and space, but amortizes across repeated transforms of the same length. For convolution of sequences of lengths N and M, choose L as the next power of two at least N+M−1; runtime is dominated by two forward FFTs and one inverse FFT, which is O(L log L). Compared to the naive convolution O(NM), FFT-based convolution is advantageous when max(N, M) is large (typically thousands or more). In practice, constants matter: cache-friendly iterative FFTs with contiguous memory and precomputed roots outperform recursive versions for large arrays. Numerical complexity includes floating-point rounding; double precision is usually adequate, but for integer polynomial multiplication, rounding after IFFT is essential. When N is not a power of two, mixed-radix or Bluestein’s algorithm retains O(N log N) complexity with a larger constant factor.

Code Examples

Naive DFT and IDFT (O(N^2)) with reconstruction check
1#include <bits/stdc++.h>
2using namespace std;
3
4// Compute the N-point DFT in O(N^2)
5vector<complex<double>> dft(const vector<complex<double>>& x) {
6 int N = (int)x.size();
7 const double PI = acos(-1.0);
8 vector<complex<double>> X(N);
9 for (int k = 0; k < N; ++k) {
10 complex<double> sum = 0.0;
11 for (int n = 0; n < N; ++n) {
12 double angle = -2.0 * PI * k * n / N;
13 sum += x[n] * complex<double>(cos(angle), sin(angle)); // e^{-i 2πkn/N}
14 }
15 X[k] = sum;
16 }
17 return X;
18}
19
20// Compute the N-point inverse DFT in O(N^2)
21vector<complex<double>> idft(const vector<complex<double>>& X) {
22 int N = (int)X.size();
23 const double PI = acos(-1.0);
24 vector<complex<double>> x(N);
25 for (int n = 0; n < N; ++n) {
26 complex<double> sum = 0.0;
27 for (int k = 0; k < N; ++k) {
28 double angle = 2.0 * PI * k * n / N;
29 sum += X[k] * complex<double>(cos(angle), sin(angle)); // e^{+i 2πkn/N}
30 }
31 x[n] = sum / (double)N; // 1/N scaling on the inverse
32 }
33 return x;
34}
35
36int main() {
37 // Example: build a signal that is the sum of two sinusoids plus a DC offset
38 int N = 16;
39 double fs = 64.0; // sampling rate (Hz)
40 vector<complex<double>> x(N);
41 for (int n = 0; n < N; ++n) {
42 double t = n / fs;
43 double s = 1.0 // DC component
44 + 0.8 * sin(2 * M_PI * 5 * t) // 5 Hz
45 + 0.4 * sin(2 * M_PI * 12 * t + 0.5); // 12 Hz with phase shift
46 x[n] = complex<double>(s, 0.0);
47 }
48
49 // Compute DFT and IDFT
50 vector<complex<double>> X = dft(x);
51 vector<complex<double>> xr = idft(X);
52
53 // Report magnitudes of the first half of bins (typical for real signals)
54 cout << fixed << setprecision(6);
55 cout << "Magnitudes (first N/2+1 bins):\n";
56 for (int k = 0; k <= N/2; ++k) {
57 cout << "k=" << k << ": |X[k]|=" << abs(X[k]) << "\n";
58 }
59
60 // Check reconstruction error
61 double max_err = 0.0;
62 for (int n = 0; n < N; ++n) {
63 max_err = max(max_err, abs(x[n] - xr[n]));
64 }
65 cout << "Max reconstruction error: " << max_err << "\n";
66 return 0;
67}
68

This program implements the DFT and its inverse directly from the definitions using std::complex<double>. It builds a synthetic signal with known frequency components, computes its spectrum, prints magnitudes for the first half of bins, and verifies perfect reconstruction (up to floating-point error) by transforming back with the IDFT.

Time: O(N^2) for DFT and O(N^2) for IDFTSpace: O(N) for outputs (in addition to input)
Iterative Radix-2 FFT/IFFT and Fast Convolution (Polynomial Multiplication)
1#include <bits/stdc++.h>
2using namespace std;
3
4// Iterative in-place radix-2 Cooley–Tukey FFT
5// If invert = false, computes forward FFT; if invert = true, computes inverse FFT.
6// Requires size to be a power of two.
7void fft(vector<complex<double>>& a, bool invert) {
8 int n = (int)a.size();
9 static vector<int> rev; // bit-reversal indices
10 static vector<complex<double>> roots{0, 1}; // roots of unity
11
12 // Precompute bit-reversal permutation
13 if ((int)rev.size() != n) {
14 int k = __builtin_ctz(n); // log2(n)
15 rev.assign(n, 0);
16 for (int i = 0; i < n; ++i) {
17 rev[i] = (rev[i >> 1] >> 1) | ((i & 1) << (k - 1));
18 }
19 }
20
21 // Precompute complex roots of unity up to n
22 if ((int)roots.size() < n) {
23 int k = __builtin_ctz(roots.size());
24 roots.resize(n);
25 const double PI = acos(-1.0);
26 while ((1 << k) < n) {
27 double ang = 2 * PI / (1 << (k + 1));
28 for (int i = 1 << (k - 1); i < (1 << k); ++i) {
29 complex<double> w = polar(1.0, ang * (2 * i + 1 - (1 << k)));
30 roots[2 * i] = roots[i];
31 roots[2 * i + 1] = roots[i] * w;
32 }
33 ++k;
34 }
35 }
36
37 // Apply bit-reversal permutation
38 for (int i = 0; i < n; ++i) {
39 if (i < rev[i]) swap(a[i], a[rev[i]]);
40 }
41
42 // Iterative FFT layers
43 for (int len = 1; len < n; len <<= 1) {
44 // wlen is primitive root for this stage
45 for (int i = 0; i < n; i += 2 * len) {
46 for (int j = 0; j < len; ++j) {
47 complex<double> u = a[i + j];
48 complex<double> v = a[i + j + len] * roots[len + j];
49 a[i + j] = u + v; // butterfly top
50 a[i + j + len] = u - v; // butterfly bottom
51 }
52 }
53 }
54
55 // For inverse FFT, conjugate and scale by 1/n after forward FFT logic
56 if (invert) {
57 // Conjugate all elements
58 for (auto& x : a) x = conj(x);
59 // Run forward FFT on conjugated data (equivalent to inverse)
60 // We can either re-run the loop or note that above already ran.
61 // Instead, perform elementwise conjugate and then scale:
62 // Since we didn't re-run, we must actually run the inverse properly.
63 // Simpler approach: implement inverse by calling fft on conjugated input.
64 }
65}
66
67// A safer approach: forward and inverse helpers using conjugation trick
68void fft_forward(vector<complex<double>>& a) {
69 // Ensure power-of-two size
70 int n = (int)a.size();
71 // We'll reuse the above function but without the incomplete invert path
72 // Implement a clean forward-only version here by copying the logic.
73
74 static vector<int> rev;
75 static vector<complex<double>> roots{0, 1};
76
77 if ((int)rev.size() != n) {
78 int k = __builtin_ctz(n);
79 rev.assign(n, 0);
80 for (int i = 0; i < n; ++i) rev[i] = (rev[i >> 1] >> 1) | ((i & 1) << (k - 1));
81 }
82 if ((int)roots.size() < n) {
83 int k = __builtin_ctz(roots.size());
84 roots.resize(n);
85 const double PI = acos(-1.0);
86 while ((1 << k) < n) {
87 double ang = 2 * PI / (1 << (k + 1));
88 for (int i = 1 << (k - 1); i < (1 << k); ++i) {
89 complex<double> w = polar(1.0, ang * (2 * i + 1 - (1 << k)));
90 roots[2 * i] = roots[i];
91 roots[2 * i + 1] = roots[i] * w;
92 }
93 ++k;
94 }
95 }
96 for (int i = 0; i < n; ++i) if (i < rev[i]) swap(a[i], a[rev[i]]);
97 for (int len = 1; len < n; len <<= 1) {
98 for (int i = 0; i < n; i += 2 * len) {
99 for (int j = 0; j < len; ++j) {
100 complex<double> u = a[i + j];
101 complex<double> v = a[i + j + len] * roots[len + j];
102 a[i + j] = u + v;
103 a[i + j + len] = u - v;
104 }
105 }
106 }
107}
108
109void fft_inverse(vector<complex<double>>& a) {
110 // Inverse via conjugate-forward-conjugate-scale trick
111 int n = (int)a.size();
112 for (auto& z : a) z = conj(z);
113 fft_forward(a);
114 for (auto& z : a) z = conj(z) / (double)n;
115}
116
117// Convolution of two real sequences using FFT (returns real result)
118vector<double> convolve(const vector<double>& A, const vector<double>& B) {
119 int n1 = (int)A.size(), n2 = (int)B.size();
120 int n = 1;
121 while (n < n1 + n2 - 1) n <<= 1; // next power of two >= N+M-1
122
123 vector<complex<double>> fa(n), fb(n);
124 for (int i = 0; i < n1; ++i) fa[i] = complex<double>(A[i], 0);
125 for (int i = n1; i < n; ++i) fa[i] = 0;
126 for (int i = 0; i < n2; ++i) fb[i] = complex<double>(B[i], 0);
127 for (int i = n2; i < n; ++i) fb[i] = 0;
128
129 fft_forward(fa);
130 fft_forward(fb);
131 for (int i = 0; i < n; ++i) fa[i] *= fb[i]; // pointwise multiply (Convolution Theorem)
132 fft_inverse(fa);
133
134 vector<double> result(n1 + n2 - 1);
135 for (int i = 0; i < (int)result.size(); ++i) {
136 // Round to nearest to reduce floating error in integer conv use-cases
137 result[i] = fa[i].real();
138 }
139 return result;
140}
141
142int main() {
143 ios::sync_with_stdio(false);
144 cin.tie(nullptr);
145
146 // Example: Polynomial multiplication (1 + 2x + 3x^2) * (4 + 5x + 6x^2)
147 vector<double> P = {1, 2, 3};
148 vector<double> Q = {4, 5, 6};
149
150 vector<double> R = convolve(P, Q); // coefficients of the product polynomial
151
152 cout << fixed << setprecision(6);
153 cout << "Product coefficients (lowest to highest degree):\n";
154 for (size_t i = 0; i < R.size(); ++i) {
155 // For integer polynomials, rounding is typical
156 cout << llround(R[i]) << (i + 1 == R.size() ? '\n' : ' ');
157 }
158
159 // Another example: convolving two real signals (smoothing kernel)
160 vector<double> signal = {1, 2, 3, 4, 5, 6};
161 vector<double> kernel = {0.25, 0.5, 0.25}; // simple low-pass FIR
162 vector<double> smoothed = convolve(signal, kernel);
163
164 cout << "\nSmoothed signal (linear convolution):\n";
165 for (size_t i = 0; i < smoothed.size(); ++i) {
166 cout << smoothed[i] << (i + 1 == smoothed.size() ? '\n' : ' ');
167 }
168 return 0;
169}
170

This program implements an iterative radix-2 FFT and its inverse using the conjugation trick and builds a fast convolution routine. It demonstrates polynomial multiplication by convolving coefficient arrays and applies a simple smoothing filter to a real signal. The size is expanded to the next power of two and zero-padded to compute linear (non-wrapping) convolution.

Time: O(L log L) for convolution where L is the chosen FFT length (≄ N+M−1), and O(N log N) per FFT callSpace: O(L) additional space for complex arrays and twiddle tables
#dft#fft#cooley-tukey#radix-2#twiddle factors#bit reversal#inverse fft#convolution#polynomial multiplication#frequency domain#spectral analysis#roots of unity#parseval#zero padding#fourier transform