How’s it going, 365 family? My name is Andrew from DataLeap and I'm happy to team up with 365 Data Science in their series 365 Data Use Cases. So, what’s my favorite data use case? Recommender systems!
Recommender Systems Use Cases: Google, YouTube, Facebook, Instagram, and Amazon
When you think of FAANG companies, what pops into your mind? Netflix and their newest dating show suggestion, Google and YouTube’s home page, Facebook, and Instagram’s feed. For B2C companies like Amazon, selling you a recommended product directly lifts their bottom line: sales revenue. But for companies that rely on content, connections, and curation, things are a bit different. Giving you a recommended video or mobile game keeps you on their platform longer. This is beneficial in a subtler way, usually increasing CTR (click-through rate) which leads to higher watch time or session time to lift ads revenue and in-app purchases.
You can check out everything I have to say on the topic in the video below or just scroll down to keep on reading.
YouTube Recommendation Algorithm
Before we get too technical, think about what data Google can even use for a YouTube recommender model: the infamous YouTube Recommendation Algorithm. Google has a sense of what videos you like based on your behavior on (or off) YouTube. Besides, they have a sense of what type of users you are similar to based on your demographic and information you’ve willingly shared with Google. If your friend Clair went to the same high school as you, has a similar search history for French cooking tutorials, and shared a video about eclairs on YouTube, then you will likely see that same video on your home page since Google recommendation engine is aware you compare to Claire and their eclectic eclair shares. So take care where you share your personal preferences.
Google Recommendation Engine
Let’s understand recommender systems from the source, the King Regent of Recommendations: Google. Google provides free courses on how to pass their developer interviews. In the section about recommendation systems, Google considers the value add of a model that helps users find compelling new content among millions of Google store apps and billions of YouTube videos.
Recommendation System Architecture
Now let’s get technical. One common recommendation system architecture looks like this:
1. Candidate generation
In the candidate generation stage, the recommendation system considers a large group of videos. This is called a corpus. Reducing billions into hundreds is the goal, and evaluating whether a video is a good candidate quickly is the key. Rigor and speed are competing against each other, especially since a recommender system might incorporate many nominators that vote on videos for better results but slower performance.
2. Scoring
The next stage involves a more precise model parsing through fewer than 100 candidates, ranking them so that the user sees the best first. The scoring section, of course, takes in your personal data as I mentioned earlier, but it also thinks about the following:
- "Local" vs "distant" items; that is, taking geographic information into account
- Popular or trending items
- A social graph; that is, items liked or recommended by friends
Now hold on, you might ask. Why not let the candidate generator score as well. There’s a couple of reasons:
- Some systems rely on multiple candidate generators. The scores of these different generators might not be comparable.
- With a smaller pool of candidates, the system can afford to use more features and a more complex model that may better capture context.
This goes hand in hand with the final stage: re-ranking.
3. Re-ranking
Re-ranking takes into consideration content that the user has already disliked, how new the content is, and other factors. This is how YouTube promotes diversity, freshness, and fairness.
Now you have a good initial idea of what recommender systems are and how the biggest and the best companies use them to reach their business goals.
However, if you enjoy comedy and compelling data storytelling with corgis, head over to my YouTube channel DataLeap and my second channel The DataLeap Andrew Show.
And, if you want to dive deeper into the subject of recommender systems and machine learning, check out the 365 Data Science course that's perfect for beginners, Machine Learning in Python.
