Machine learning has multiple benefits. The first is consistency—аn algorithm can evaluate hundreds of variables, find indiscernible relationships, and make the same prediction every time. Humans, on the other hand, can become overwhelmed by the amount of information and make erratic predictions. When dealing with high volumes of data, a human may come to different conclusions about predicted outcomes from the same dataset without realizing it.

Learn when you should and shouldn’t use machine learning. We teach you when in your organization’s lifecycle to implement ML and what real-life problems you can solve. We also cover the types of ML problems and show you the difference between product analytics vs. data products.

The most important step in machine learning is not any fancy technique—it’s properly defining the problem. An incorrect definition can result in months—or even years—of wasted resources. That is why, before we can leverage the power of ML, we need a reason to use it. Whether it’s a directive from a manager, a pain point from a stakeholder, or a gap in the business, you need to properly frame the problem first and foremost.

Most projects won’t give you the data in a neat and tidy .csv file like they do in class or on Kaggle. If you work at a big company, it is likely that you will have plenty of data readily available to you. However, you’ll still need to pull it yourself. In this section, we will talk about data collection techniques from most structured to least structured.

After defining our problem, we need to preprocess our data. Keep in mind that different models have different requirements. For example, linear regression cannot handle null values, whereas some tree-based models can. Additionally, deviant data points can significantly skew the results of some models and not others, e.g., linear models are affected by outliers whereas tree-based models—not as much. In that case, you may want to remove or adjust outliers in order to get the best performance from our models. After all, we want to make sure our data is of the highest quality before we start building our models.

Exploratory data analysis is an important prerequisite to the modeling process. With EDA, we get a feel for the assumptions so we can choose the best model and get the most accurate results. The reason we perform this step is to understand the shape of our data and the relationships between the variables.

The feature engineering step will likely reap the highest accuracy gains in our modeling process—even more than the selection of the algorithm itself. The core principle is to find variables that have predictive power. So, regardless of how complex the feature is or how fancy of a technique, if it does not add predictive power to our model, then it doesn’t do anything for your model. Remember: complexity doesn’t create value. Creative and domain-specific features create value.

Cross-validation allows us to test our model on data that the model has not been trained on. If it performs well on this unseen data, we can be more confident that our predictions will generalize well.

Feature selection is the process of reducing the model’s complexity by selecting only the highest signal features in the dataset. Typically, there are three classes of feature selection techniques. Wrapper methods require us to exhaustively search through all features and measure performance. They give us the best accuracy but are the most computationally expensive out of the three. Next, we have filter methods that pick up feature properties through univariate statistics. These tend to be faster and easier to check since they don’t rely on cross-validation. Lastly, the embedded methods are baked into the ML algorithm. We cover these in the companion course, Machine Learning Algorithms.

How do we deal with imbalance in our data? This is where oversampling, undersampling, and other techniques become useful. We show you how to perform these to optimize your model’s performance.

In this section of the course, we’ll focus on ML modeling techniques such as the baseline model, model selection, hyperparameter tuning, and assembling.

We can’t measure the value of our model without proper evaluation criteria. Earlier in this course, we touched on two types of Supervised ML problems—regression and classification. Since these problems use different target variables, we use different metrics to evaluate the quality of the model.

The goal of our work is to make it useful to other people. How do we do that? Through something called productionization. It is a little broader than simply deploying the model into your product; rather it is the delivery of our model to our end user in whatever form.

As the course goes through different steps of the modeling process, you’re probably wondering, “How am I supposed to remember each step?” The good news is that building an ML model typically follows a very similar process each time: (1) Problem Framing/Business Understanding; (2) Data Cleaning/Preparation; (3) Exploratory Data Analysis; (4) Feature Engineering; (5) Cross-Validation; (6) Modeling; (7) Evaluation