Polynomial Regression

In the previous chapter, we explored quadratic regression, which has the graph of a parabola. In the same way, we could add the x³ to the equation to get the Cubic Regression that has a more complex graph. We could also add x⁴ and so on.

A Degree of a Polynomial Regression

In general, it is called the polynomial equation and is the equation of Polynomial Regression. The highest power of x defines a degree of a Polynomial Regression in the equation. Here is an example

N-Degree Polynomial Regression

Considering n to be a whole number greater than two, we can write down the equation of an n-degree Polynomial Regression.

Where:

• $\beta_0, \beta_1, \beta_2, \dots, \beta_n$ – are the model's parameters;
• $y_{\text{pred}}$ – is the prediction of a target;
• $x$ – is the feature value;
• $n$ – is the Polynomial Regression's degree.

Normal Equation

And as always, parameters are found using the Normal Equation:

Where:

• $\beta_0, \beta_1, \dots, \beta_n$ – are the model's parameters;

• $X$ – is an array of feature values from the training set;
• $X^k$ – is the element-wise power of $k$ of the $X$ array;
• $y_{\text{true}}$ – is an array of target values from the training set.

Polynomial Regression with Multiple Features

To create even more complex shapes, you can use the Polynomial Regression with more than one feature. But even with two features, 2-degree Polynomial Regression has quite a long equation.

Most of the time, you won't need that complex model. Simpler models (like Multiple Linear Regression) usually describe the data well enough, and they are much easier to interpret, visualize and less computationally expensive.