Predicting the future using Machine Learning part II

The polynomial model + implementation in Python

In this second part of the “Predicting the future using Machine Learning”-series I would like to elaborate further on finding a better generic model for the same dataset I worked with in the previous part.

To form a polynomial model, polynomial data is needed. This can easily be retrieved by extending the linear dataset I used earlier to a higher order polynomial dataset by expanding the original input with the original input to the power of a higher order polynomial and adding additional weights to the model.

I wrote a function that creates a polynomial dataset as a matrix given the datapoints and the specified order, which I called k. For ease of calculation I expanded the dataset with a vector of 1’s, which represents the constant weight w0:

I chose order 2, so my polynomial dataset looked like this:

The hypothesis for a single input is of the form:

This can be written as a matrix multiplication for all inputs in a single equation:

Implementing this in Python gives:

Solving all partial derivative equations per weight can easily be done with matrix operations. The principle is exactly the same as for the linear model, where we also just compute w0 and w1. So the final equation for the weight vector is of the form:

With the numpy built-in functions for the transpose and inverse, this equation can easily be implemented in Python, like so:

The only thing left now is visualizing the fit:

As can be seen in the graph above, the polynomial fit is a way better fit to this dataset than the linear fit. The disadvantage that occurs with the polynomial model is that it is prone to overfit. In the next part of this series I will discuss a method that can be applied that chooses a model which is less prone to overfit than the polynomial model.

Related posts:

With this article series we have a ledge grab but when we do the ledge grab it can only go to one spot. This is because of the vector that we have in our code. How to fix this? We need to make it to…