Linear Regression | Trading with Machine Learning Regression Models | Quantra Course
Part of the course on Trading with Machine Learning Regression:
https://quantra.quantinsti.com/course...
Welcome to this video lecture! In this video we will understand ‘Linear Regression’.
Let us begin with ‘Scatter Plots’. A scatter plot is a collection of points on a graph where each point represents the value of two variables (i.e. an X/Y pair). If we assume that ‘x’ is the return on the S&P 500 index and ‘y’ is the return on Stock ABC, then graphically the x/y plot will look like this. The degree to which these variables, i.e. return on S&P 500 index or ‘x’ and return on stock ABC or ‘y’, move together is called covariance.
Covariance captures the linear relationship between two random variables. Here, a positive covariance implies that the market and the stock tend to move in the same direction whereas a negative covariance implies that they tend to move in the opposite direction. But covariance has it’s limitations.
It is sensitive to the scale of the variables and it’s value is in terms of squared units which may range from positive to negative infinity. Hence making it difficult to compare two covariance values. To overcome these limitations we calculate the correlation coefficient of these variables after calculating the covariance.
Correlation measures the strength of linear relationship between ‘x’ and ‘y’ It’s a pure measure and has no units Correlation has a range of minus 1 to plus 1
Let us look at the 5 scatter plots of the stock ABC returns and index returns to understand correlation Limitation to correlation analysis is that
it may be affected by outliers leading to spurious correlation and as a result it does not capture strong nonlinear relationships between variables
When one tries to capture a mathematical relationship between ‘x’ and ‘y’ variables by fitting a line polynomial or a curve through these scatter plots
such that if one should be able to make a reasonably good prediction
of ‘y’ given ‘x’ then the mathematical process of deriving such an equation between x and y is called regression analysis
To break it down further regression explains the variation in a dependent variable in terms of an independent variable. The dependent variable ‘y’ is the variable that you want to predict. The independent variable 'x’ is the input variable that we use to predict the dependent variable.
In our example the Stock ABC’s returns are dependent and explained using the market index returns. If we try to fit a line through this scatter plot, that “best” explains the observed values of ‘y’ in terms of observed values of ‘x’, we get a simple linear regression model.
Linear regression assumes a linear relationship between the dependent and independent variables. The following regression equation describes that relation:
Yi = b0 + b1 Xi + ei
This equation states that the dependent variable ‘Y’ is equal to the intercept ‘b0’, plus a slope coefficient ‘b1’ times the independent variable ’X’, plus an error term ‘e’. The error term represents the portion of the dependent variable that cannot be explained by the independent variable. We refer to the intercept ‘b0’ and slope coefficient ‘b1’ as the regression coefficients. All the points on the regression equation line, predict the ‘y’ values for the corresponding ‘x’ values. However, the optimal regression line is the one for which the sum of the squared differences or (vertical distances) or the sum of squared errors or SSE between the ‘y’ values predicted by the regression equation/line and the actual ‘y’ values is minimal.
This is the reason why simple linear regression is also called as Ordinary Least Squares or OLS, and the estimated (predicted) values by the regression equation i.e. y predicted are called least squares estimates. The slope coefficient ‘b1’ of the regression line is calculated as covariance of x and y divided by the variance of x and the intercept coefficient is the line's intersection with the y axis at x = 0.
To understand the coefficients more intuitively, let us consider the returns for Stock ABC vs. S&P 500 index. The slope coefficient in a regression line
is called the stock’s beta as it measures the relative amount of systematic
or undiversifiable risk in ABC's returns.
The intercept term is the ex-post alpha i.e. the measure of excess returns of stock ABC as compared to market index returns. If the intercept term is negative, it means ABC has underperformed S&P on a risk adjusted basis and a positive intercept means it has had excess returns on risk adjusted basis.
Quantra is an online education portal that specializes in Algorithmic and Quantitative trading. Quantra offers various bite-sized, self-paced and interactive courses that are perfect for busy professionals, seeking implementable knowledge in this domain.
Find more info on - https://quantra.quantinsti.com/
Like us on Facebook: / goquantra
Follow us on Twitter: / goquantra