Steps to evaluate a feature engineering procedure

Here are the steps to evaluate a feature engineering procedure:

1. Obtain a baseline performance of the machine learning model before applying any feature engineering procedures
2. Apply feature engineering and combinations of feature engineering procedures
3. For each application of feature engineering, obtain a performance measure and compare it to our baseline performance
4. If the delta (change in) performance precedes a threshold (usually defined by the human), we deem that procedure helpful and apply it to our machine learning pipeline
5. This change in performance will usually be measured as a percentage (if the baseline went from 40% accuracy to 76% accuracy, that is a 90% improvement)

In terms of performance, this idea varies between machine learning algorithms. Most good primers on machine learning will tell you that there are dozens of accepted metrics when practicing data science.

In our case, because the focus of this book is not necessarily on machine learning and rather on the understanding and transformation of features, we will use baseline machine learning algorithms and associated baseline metrics in order to evaluate the feature engineering procedures.