The focus of this article is to illustrate the application of linear regression using the sklearn library in python. Work in progress (5/30/2018)
Overview:
1. Polynomials in Python.
2. Generating sample data.
3. Fit a linear regressor and evaluate the R2 score.
4. Polynomial Regression.
5. K-Fold cross-validation.
6. Multivariate Regression.
Polynomials in Python:
Before delving into linear regression, let us create a function that evaluates polynomials using the matrix form of a polynomial.

Notice that each row represents a single data point; the row is passed through by taking the dot product of the Vandermonde matrix and the coefficient matrix. The result of the first dot product is:
![]()
A0 is the linear combination of all the terms in the polynomial evaluated at x0. Using this method we can evaluate a polynomial by passing a list of values x:[n_points] where the result of every value in the list is mapped out as the following:
![]()
we can code this up in python with a couple of lines of code.
| """ | |
| Polynomials | |
| @author: Abdullah Alnuaimi | |
| """ | |
| import numpy as np | |
| import matplotlib.pyplot as plt | |
| def fit_poly(a,k): | |
| '''returns a function of the dot product (A=V.a) ''' | |
| A=lambda x,a=a,k=k:[[a*n**k for a,k in zip(a,k)] for n in x] | |
| return A | |
| def evaluate_poly(x,A): | |
| ''' evaluates A=V.a,stores it in matrix form, and | |
| returns a list y(x)=[A0,..An]''' | |
| y=[sum(i) for i in A(x)] | |
| return y,A(x) | |
| ############################## Main ######################################### | |
| # y(x)=x+x^2-0.2x^3 | |
| coefficients=[0,1,1,-.2] # polynomial | |
| degree=[0,1,2,3] | |
| A=fit_poly(coefficients,degree) # returns A(x0)...A(Xn) | |
| # Evaluate the functions and returns y(x) | |
| x=np.linspace(0,5,20) | |
| y,_=evaluate_poly(x,A) | |
| #Plotting | |
| plt.plot(x,y,label='p(x)=x+x^2-0.2x^3') | |
| plt.xlabel('x') | |
| plt.ylabel('y(x)') | |
| plt.legend() | |
| plt.grid() |

Now whenever we need to create a polynomial function all we have to do is specify the coefficients and the order of the polynomial.
I should add the calling the function fit_poly() instantiates A(x) however it’s evaluated once evalute_poly() is called and fed with input data. This adds some flexibility without creating classes.
(which we just might…)