Categorical variable encoding
In this video, we implement different categorical encoding techniques, and we compare their effect on the performance of different machine learning algorithms.
Categorical variables are those that take strings, or categories as values, instead of numbers. For example, the variable color, with values red, blue and green is a categorical variable. If we use Scikit-learn to build machine learning models, we need to transform the strings of categorical variables into numbers. There are multiple techniques that we can use to do so.
In one hot encoding, we derive dummy variables from the categories of the variable, so that each category now becomes a variable with values 1 or 0 indicating if the category is present in each observation.
In ordinal or integer encoding, we replace the categories by integers. Usually first category seen in the variable get the value 0 and so on. A twist in this encoding is to assign the values based on the mean of the target per category. Thus, the category where the mean of the target is the smallest is replaced by 0, and that where the mean of the target is the higher, is replaced by n, n being the number of different categories.
In target encoding or mean encoding, we replace the category by the mean target value in that category.
Another technique used in finance is the weight of evidence, where the category is replaced by the natural logarithm of the percentage of positive instances divided the percentage of negative instances, positive and negative instances being the target equal to 1 or 0, respectively.
Different encoding techniques are orientated to improve the performance of different algorithms. For example, one hot encoding and the techniques that generate monotonic encoding tend to improve linear model performance, whereas for decision trees a random assignment of numbers to the categories may work just as well.
In this video, we compare the implementation of different technique using pandas. But keep in mind that this can be smoothed by using open source packages like Feature-engine or Scikit-learn as shown in these articles:
https://www.trainindata.com/post/feat...
https://www.trainindata.com/post/feat...