REGRESSION ANALYSIS general idea | DATA ANALYSIS #16
Regression Analysis General Idea. How to Conduct Multiple Regression Analysis? The goal is to build a regression model. The general purpose of multiple regression (the term was first used by Pearson, 1908) is to analyze the relationship between several independent variables (also called regressors or predictors) and a dependent variable.
The term regression in statistics was first used by Francis Galton (1886) in connection with the study of the inheritance of human physical characteristics. One of the characteristics was height; it was found that, in general, sons of tall fathers, not surprisingly, were taller than sons of short fathers. More interestingly, the spread in the height of sons was smaller than the spread in the height of fathers. This was a manifestation of the tendency for the height of sons to return to the mean (regression to mediocrity), that is, "regression". This fact was demonstrated by calculating the average height of sons of fathers whose height is 56 inches, calculating the average height of sons of fathers whose height is 58 inches, and so on. The results were then plotted on a plane with the average height of sons on the ordinate and the average height of fathers on the abscissa. The points (approximately) fell on a straight line with a positive slope of less than 45°; it is important that the regression was linear.
Aims of Regression Analysis
Determining the degree of determinacy of the variation of the criterion (dependent) variable by the predictors (independent variables)
Predicting the value of the dependent variable using the independent(s)
Determining the contribution of individual independent variables to the variation of the dependent
Regression analysis cannot be used to determine the presence of a relationship between variables, since the presence of such a relationship is a prerequisite for using the analysis.
Regression analysis is a statistical method that allows you to build a model with one dependent variable (response) and one or more independent variables (factors).
Regression analysis allows you to:
1) Identify which factors affect the response and which do not.
2) Rank factors by the degree of influence on the response.
3) Predict the response value for certain factor values.
To ensure acceptable accuracy of the model, the minimum sample size should not be less than the value of "number of factors multiplied by 10".
Here are some examples of problem statements for regression analysis.
Determine which factors affect electricity consumption at the enterprise, and build a forecast of electricity costs for the next quarter.
The construction of a new shopping center is planned. It is required to predict the "passability" of the sections of the future shopping center in order to justify the rental rate and the optimal area of the premises.
Based on the realtor database of sold real estate objects, build a forecast of the cost of an apartment taking into account the area, amenities, type of house and other factors.
Identify the factors that determine the market share of a brand of certain goods.
When buying a car, you need to choose a model that will lose little in value after three years of service on the secondary market.
The model built using regression analysis is an equation of the form:
Y = b_0 + b_1 X_1 + b_2 X_2 + ⋯ + b_k X_k
where X_1, X_2, ..., X_k are factors, Y is the response, b_0, b_1, ..., b_k are the regression parameters (coefficients).
Mathematically, the essence of regression analysis comes down to finding the regression parameters, checking their significance and assessing the acceptability of the entire constructed linear model as a whole.
Let us explain using the example of a model with one quantitative factor. Let us find out whether the area of a real estate object (cu) affects its price (sq. m). Let's plot a graph of the price (along the vertical axis) versus the area (along the horizontal axis). Such a graph is called a correlation field or a scatter diagram.
The line passing through the concentration of points is called the regression line. It is plotted using the least squares method, which consists of minimizing the vertical distance of all points of the correlation field from the regression line.
If the model, in multiple regression analysis, is adequate, i.e. reflects the true strength of the relationship between price and area, then the regression line can be used to predict the price value for a specific value of the area of the real estate object.