In our previous tutorial, we became familiar with the ARMA model. But did you know that we can expand the ARMA model to handle non-stationary data?
Well, that’s exactly what we’re going to cover in this post - the intuition behind the ARIMA model, the notation that goes with it, and how it differs from the ARMA model.
Let’s get started, shall we?
What is an ARIMA model?
As usual, we’ll start with the notation. An ARIMA model has three orders – p, d, and q (ARIMA(p,d,q)). The “p” and “q” represent the autoregressive (AR) and moving average (MA) lags just like with the ARMA models. The “d” order is the integration order. It represents the number of times we need to integrate the time series to ensure stationarity, but more on that in just a bit.
Convention dictates that we always enter the three orders in the same way – “p” first, then “d” and finally – “q” (ARIMA(p,d,q)). Of course, that’s because “p” represents the AR components, “d” the Integrated ones and “q” the MA ones.
How is ARIMA related to ARMA?
Any model of the sort ARIMA (p, 0, q) is equivalent to an ARMA (p, q) model since we are not including any degree of changes. Of course, an ARIMA (0, 0, q) and an ARIMA (p, 0, 0) would also be the same as an MA(q) and an AR(p) respectively. Now that we’re familiar with the notation and how the different types of models are connected, we can continue with the intuition.
How do ARIMA models work?
These integrated models account for the non-seasonal difference between periods to establish stationarity.
Hence, even the AR components in the model should be price differences, (ΔP) rather than prices (P). In a sense, we are “integrating” “d”-many times to construct a new time-series and then fitting said series into an ARMA (p, q) model.
What does a simple ARIMA (1,1,1) look like?
Okay, since now we know this, let’s have a look at the equation of a simple ARIMA model, with all orders equal to 1. Suppose P is the price variable we’re trying to model. Then, the simple ARIMA equation for P would look as follows:
ΔPt =c+ϕ1 ΔPt-1 + θ1 ϵt-1 +ϵt
Just like we did in the other tutorials on time series models, let’s go over all the moving parts of this equation and break it down, so we can understand it better.
For starters, Pt and Pt-1 represent the values in the current period and 1 period ago respectively.
Similarly, ϵ t and ϵ t-1 are the error terms for the same two periods. And, of course, c is just a baseline constant factor. The two parameters, ϕ 1 and θ1, express what parts of the value (Pt-1) and error (ϵ t-1) last period are relevant in estimating the current one.
Finally, we have ΔPt-1 . In math, physics, and science in general, we express the difference between two values as Δ (delta). Therefore, ΔPt-1 is the difference between prices in period “t” and prices in the preceding period (ΔPt = Pt-1-Pt). Therefore, ΔP is an entire time-series, which represents the disparity between prices of consecutive periods.
Here’s an easy way to think about ARIMA models.
Essentially, the entire ARIMA model is nothing more than an ARMA model for a newly generated time-series, which is stationary.
How do we determine the orders of an ARIMA model?
We saw that the ARMA doesn’t have any functions like the ACF or PACF which suggest what the optimal order for the different components is. We can say the same about the ARIMA. After all, it’s a more complex model based on ARMA. So, our best bet is to start simple, check if integrating once grants stationarity. If so, we can fit a simple ARIMA model and examine the ACF of the residual values to get a better feel about what orders to use.
Peculiarities of integrated models
It’s important to note that we lose d-many observations when we deal with integrated values. This comes from the fact that there is no “previous” period, where we wish to integrate the very first day of the dataset. Simply put, we can’t find the difference between the first element and the one preceding it, because it doesn’t exist. Similarly, if we integrate two times, we lose two observations, one for each integration. Even though we’d have an integrated difference in prices for the second day of the dataset (ΔP2 = P 1 - P2), wouldn’t have one for the first (ΔP 1= P0- P1), to compare it with. Therefore, we’d also have a missing value for the second day of the time-series, after integrating twice (Δ2P2= ΔP1- ΔP2).
Simply put, for any integration we lose a single observation, so we should be aware of this when making our analysis. This is important because having empty values prevents the certain Python functions from compiling.
If you want to learn more about implementing ARIMA models in Python, or how the model selection process works, make sure to check out our step-by-step Python tutorials and enroll in our Time Series Analysis with Python course.
If you’re new to Python, and you’re enthusiastic to learn more, this super comprehensive article on learning Python programming will guide you all the way from the installation, through Python IDEs, Libraries, and frameworks, to the best Python career paths and job outlook.
Ready to take the next step towards a career in data science?
Check out the complete Data Science Program today. Start with the fundamentals with our Statistics, Maths, and Excel courses. Build up a step-by-step experience with SQL, Python, R, Power BI, and Tableau. And upgrade your skillset with Machine Learning, Deep Learning, Credit Risk Modeling, Time Series Analysis, and Customer Analytics in Python. Still not sure you want to turn your interest in data science into a career? You can explore the curriculum or sign up for 15 hours of beginner to advanced video content for free by clicking on the button below.
