Home / Time Series Forecasting

Formula Deep-Dive: Time Series Forecasting Methods

What is Time Series Forecasting?

Goal: Predict future values based on historical patterns in sequential data.

Examples: Stock prices, temperature, sales, website traffic

Key Assumption: Past patterns will continue into the future

ARIMA: AutoRegressive Integrated Moving Average

$$\text{ARIMA}(p, d, q)$$
Where:
  • \(p\) = order of AutoRegression (AR)
  • \(d\) = degree of differencing (I)
  • \(q\) = order of Moving Average (MA)

Component 1: AR (AutoRegressive)

Predict current value from previous values:

$$y_t = c + \phi_1 y_{t-1} + \phi_2 y_{t-2} + \cdots + \phi_p y_{t-p} + \epsilon_t$$

Example: Today's temperature depends on yesterday's and the day before.

  • \(y_t\) = value at time \(t\)
  • \(\phi_i\) = coefficients (weights)
  • \(\epsilon_t\) = error term

Component 2: I (Integrated / Differencing)

Remove trend by taking differences:

$$\Delta y_t = y_t - y_{t-1}$$

Why? Many time series have trends (upward/downward). Differencing makes them stationary (constant mean).

Example: Sales: [100, 105, 110, 115] → Differences: [5, 5, 5] (now stationary!)

Component 3: MA (Moving Average)

Predict from previous forecast errors:

$$y_t = \mu + \epsilon_t + \theta_1 \epsilon_{t-1} + \theta_2 \epsilon_{t-2} + \cdots + \theta_q \epsilon_{t-q}$$

Insight: If we overshot yesterday's forecast, adjust today's prediction.

  • \(\theta_i\) = MA coefficients
  • \(\epsilon_{t-i}\) = past forecast errors

Full ARIMA Model

Combining all three components:

$$\phi(B)(1-B)^d y_t = \theta(B)\epsilon_t$$

Where:

  • \(B\) = backshift operator (\(By_t = y_{t-1}\))
  • \(\phi(B)\) = AR polynomial
  • \(\theta(B)\) = MA polynomial
  • \((1-B)^d\) = differencing operator

Choosing Parameters (p, d, q)

  • d: Apply differencing until data is stationary (constant mean/variance)
  • p: Look at ACF (AutoCorrelation Function) plot
  • q: Look at PACF (Partial AutoCorrelation Function) plot
  • Or use AIC/BIC to compare models automatically

SARIMA: Seasonal ARIMA

$$\text{SARIMA}(p,d,q)(P,D,Q)_m$$
Where:
  • \((p,d,q)\) = non-seasonal part (same as ARIMA)
  • \((P,D,Q)\) = seasonal part
  • \(m\) = seasonal period (e.g., 12 for monthly data with yearly seasonality)

The Problem with ARIMA

ARIMA can't handle seasonality (repeating patterns):

  • Ice cream sales peak every summer
  • Website traffic drops every weekend
  • Electricity usage spikes at 6 PM daily

SARIMA adds seasonal components to capture these patterns.

Seasonal Components

Seasonal AR (P)

Value depends on same period in previous cycles:

$$y_t = \Phi_1 y_{t-m} + \Phi_2 y_{t-2m} + \cdots + \epsilon_t$$

Example: January sales depend on last January, January before that, etc.

Seasonal Differencing (D)

Remove seasonal trend:

$$\Delta_m y_t = y_t - y_{t-m}$$

Example: \(y_{\text{Jan 2026}} - y_{\text{Jan 2025}}\)

Seasonal MA (Q)

Adjust based on seasonal forecast errors:

$$y_t = \Theta_1 \epsilon_{t-m} + \Theta_2 \epsilon_{t-2m} + \cdots + \epsilon_t$$

Example: Monthly Sales

SARIMA(1,1,1)(1,1,1)12

  • (1,1,1): Short-term dependencies (yesterday affects today)
  • (1,1,1)12: Seasonal dependencies (last January affects this January)
  • m=12: Monthly data with yearly cycle

Holt-Winters (Triple Exponential Smoothing)

Philosophy: Recent observations matter more than old ones.

Three components: Level, Trend, Seasonality

The Three Equations

1. Level (Smoothed Series)

$$\ell_t = \alpha \frac{y_t}{s_{t-m}} + (1-\alpha)(\ell_{t-1} + b_{t-1})$$

Weighted average of deseasonalized observation and forecasted level.

2. Trend (Slope)

$$b_t = \beta (\ell_t - \ell_{t-1}) + (1-\beta)b_{t-1}$$

Weighted average of current trend and previous trend.

3. Seasonality

$$s_t = \gamma \frac{y_t}{\ell_t} + (1-\gamma)s_{t-m}$$

Weighted average of current seasonal index and seasonal index from same period last cycle.

Forecast (h steps ahead)

$$\hat{y}_{t+h} = (\ell_t + h \cdot b_t) \cdot s_{t+h-m}$$
Parameters:
  • \(\alpha\) = smoothing parameter for level (0 to 1)
  • \(\beta\) = smoothing parameter for trend (0 to 1)
  • \(\gamma\) = smoothing parameter for seasonality (0 to 1)
  • \(m\) = seasonal period (e.g., 12 for monthly)

✓ Holt-Winters Variants

Additive: Seasonal variations constant in size

$$\hat{y}_{t+h} = \ell_t + h \cdot b_t + s_{t+h-m}$$

Multiplicative: Seasonal variations proportional to level

$$\hat{y}_{t+h} = (\ell_t + h \cdot b_t) \times s_{t+h-m}$$

When to Use Which?

  • Additive: +/- 100 units every summer (constant amplitude)
  • Multiplicative: ×2 every summer (percentage increase)

Which Method to Choose?

Method Best For Pros Cons
ARIMA Non-seasonal data with trend Flexible, well-studied, works for many cases Can't handle seasonality, requires stationarity
SARIMA Seasonal data with trend Handles both trend and seasonality Complex parameter tuning, computationally heavy
Holt-Winters Strong trend + strong seasonality Simple, fast, intuitive Less flexible than SARIMA, assumes patterns continue

Decision Tree

  1. Is there seasonality?
    • No → ARIMA
    • Yes → Continue
  2. Is seasonality strong and regular?
    • Yes → Holt-Winters (faster)
    • Complex pattern → SARIMA (more flexible)
  3. Do you need interpretability?
    • Yes → ARIMA/SARIMA (clear parameters)
    • No → Try both, compare forecasts

Practical Implementation Tips

Common Workflow

  1. Plot your data — Look for trends, seasonality, outliers
  2. Check stationarity — Use Augmented Dickey-Fuller test
  3. Difference if needed — Apply differencing until stationary
  4. Choose parameters — ACF/PACF plots or grid search
  5. Fit model — Train on historical data
  6. Validate — Check residuals (should look like white noise)
  7. Forecast — Predict future values with confidence intervals

Python Example (statsmodels)

import pandas as pd
from statsmodels.tsa.statespace.sarimax import SARIMAX

# Fit SARIMA model
model = SARIMAX(data, 
                order=(1, 1, 1),          # (p,d,q)
                seasonal_order=(1, 1, 1, 12))  # (P,D,Q,m)
results = model.fit()

# Forecast
forecast = results.forecast(steps=12)

Python Example (Holt-Winters)

from statsmodels.tsa.holtwinters import ExponentialSmoothing

# Fit Holt-Winters
model = ExponentialSmoothing(data, 
                             seasonal='multiplicative',
                             seasonal_periods=12)
results = model.fit()

# Forecast
forecast = results.forecast(steps=12)

Key Takeaways

  • ARIMA: Foundation for time series — models trend and short-term patterns
  • SARIMA: Extends ARIMA with seasonal components — best for complex patterns
  • Holt-Winters: Simple exponential smoothing — fast and effective for clear trends + seasonality
  • Always validate with train/test split and check residuals
  • Consider modern alternatives: Prophet (Facebook), LSTM (deep learning) for very complex patterns