In this first section, we will discuss what the course covers, why you need to learn Web Scraping and give you some notes on the ethics of scraping.

In this part of the course, we will explain to you how to set up Python 3 and then load up Jupyter. We’ll also show you what the Anaconda Prompt is and how we use it to download and import new modules.

Here we will introduce what APIs are and how to use them. In order to do that, we will discuss the popular data exchange format JSON, as well as HTTP requests and the Python library to submit them – ‘requests’. At the end of the section we will show you how to deal with an API that requires a registration.

Web Scraping relies on extracting information from the source code of webpages. Thus, a general understanding of HTML is required. This section is a short crash course for those that are not familiar with HTML. It is meant as an intuitive look into the basics, not a comprehensive guide.

After familiarizing with HTML, we are ready to delve into the Web Scraping itself. We will now introduce the “Beautiful Soup” package and explore its capabilities.

Now that we’ve seen what Beautiful Soup can do, we will devote this section to practicing our newly formed skills. We are going to obtain information about movies from a ‘Rotten Tomatoes’ rank list.

In this short section, we will discuss an easy way to scrape HTML tables.

Although we have done a decent amount of scraping so far in the course, this is one of those topics that can depend very much so on the website we choose. Different websites present specific problems. Thus, in this section, we will discuss what are the most common problems that you will have to deal with and give you solutions and workarounds.

Here, we will introduce another Web Scraping package – ‘Requests-HTML’. We are doing it because it has one big advantage over Beautiful Soup – the ability to execute JavaScript. Thus, this allows us to extract dynamically generated content which is exactly what we will do.