Web Scraping and API Fundamentals in Python
Web Scraping and API Fundamentals in Python offers an introduction to the techniques of data extraction from the web. In this course, you will learn how to use one of the most powerful tools on the Internet â APIs. We will also discuss in depth how to obtain information directly from websites using the BeautifulSoup Python package. There will be a short HTML crash course for those not familiar with it. Finally, we will introduce the Requests-HTML package in order to extract dynamically generated JavaScript content.
4 hours 
8 exercises 
26 quizzes 
44 downloadables
Sign up to
preview the program
for FREE!
Create a free account and start learning data science today.
create free account
Our graduates work at exciting places:
Section 1
Course Introduction
In this first section, we will discuss what the course covers, why you need to learn Web Scraping and give you some notes on the ethics of scraping.
What does the course cover?
What is Web Scraping?
Ethics of Scraping
Section 2
Setting Up the Environment
In this part of the course, we will explain to you how to set up Python 3 and then load up Jupyter. Weâll also show you what the Anaconda Prompt is and how we use it to download and import new modules.
Setting up the environment - Do not skip, please!
Why Python and why Jupyter?
Installing Anaconda
Jupyter Dashboard - Part 1
Jupyter Dashboard - Part 2
Installing the packages
Section 3
Working with APIs
Here we will introduce what APIs are and how to use them. In order to do that, we will discuss the popular data exchange format JSON, as well as HTTP requests and the Python library to submit them â ârequestsâ. At the end of the section, we will show you how to deal with an API that requires registration.
API overview
HTTP requests: GET and POST requests
JSON: preferred data exchange format for APIs
Exchange rates API: GETting a JSON response
Incorporating parameters in a GET request
Additional API functionalities
Creating a simple currency converter
iTunes API
iTunes API: Structuring and exporting the data
GitHub API: Pagination
EDAMAM API: Initial setup and registration
EDAMAM API: Sending a POST request
Downloading files with Requests
Section 4
HTML Overview
Web Scraping relies on extracting information from the source code of webpages. Thus, a general understanding of HTML is required. This section is a short crash course for those that are not familiar with HTML. It is meant as an intuitive look into the basics, not a comprehensive guide.
What is HTML?
Structure of HTML
Syntax of HTML. Tags
Tag attributes
Popular tags
CSS and JavaScript
Character encoding
XHTML and code style
Section 5
Web Scraping with Beautiful Soup
After familiarizing with HTML, we are ready to delve into the Web Scraping itself. We will now introduce the âBeautiful Soupâ package and explore its capabilities.
Introduction to the Beautiful Soup package
Workflow of Web Scraping
Setting up your first scraper
Searching and navigating the HTML tree
Searching the HTML tree by attributes
Exctracting data from the HTML tree
Extracting text from an HTML tag
Practical example: dealing with links
Extracting data from nested HTML tags
Scraping multiple pages automatically
Section 6
Practical Project: Scraping Rotten Tomatoes
Now that weâve seen what Beautiful Soup can do, we will devote this section to practicing our newly formed skills. We are going to obtain information about movies from a âRotten Tomatoesâ rank list.
Setting up your scraper
Extracting the title and year of each movie
Extracting the rest of the information
Dealing with the cast of the movies
Storing and exporting the data in a structured form
Section 7
Scraping HTML Tables
In this short section, we will discuss an easy way to scrape HTML tables.
Scraping HTML tables with the help of Pandas
Section 8
Common Roadblocks when Scraping
Although we have done a decent amount of scraping so far in the course, this is one of those topics that can depend very much so on the website we choose. Different websites present specific problems. Thus, in this section, we will discuss what are the most common problems that you will have to deal with and give you solutions and workarounds.
Common roadblocks when Web Scraping
Section 9
The Requests-HTML Package
Here, we will introduce another Web Scraping package â âRequests-HTMLâ. We are doing it because it has one big advantage over Beautiful Soup â the ability to execute JavaScript. Thus, this allows us to extract dynamically generated content which is exactly what we will do.
Introduction to the requests-html package
Exploring the capabilities of requests-html for Web Scraping
Searching for text
CSS selectors
Scraping JavaScript
Advanced Specialization
This course is part of Module 4 of the 365 Data Science Program. The complete training consists of four modules, each building upon your knowledge from the previous one. Module 4 is focused on developing a specialized, industry-relevant skill set, and students are encouraged to complete Modules 1, 2, and 3 before they start this part of the training. Here, you will learn how to perform Credit Risk Modeling for banks, Customer Analytics for retail or other commercial companies, and Time Series Analysis for finance and stock data.
See All Modules
Why Choose the 365 Data Science Program?
Practice
Real-life project and data. Solve them on your own computer as you would in the office.
Q&A Hub
Our expert instructors are happy to help. Post a question and get a personal answer by one of our instructors.
Certificates
Earn a verifiable certificate after each completed course. Celebrate your successes and share your progress with your professional network!
Trust the other 500,000 students
The course is in-depth and is delivered at a steady pace with eye catching visuals. The instructors go through all the basics really well. They try not to over-simplify the material, you get a good sense Đ°of how deep Data Science is in the course. Great job!!!
This course is amazing! After watching the video carefully and doing all the exercises, I am even capable of having discussions with Machine learning major Masterâs students! High standard course with reasonable pricing.
Very clear and in-depth explanation of data science and how all the inter-related concepts apply in real life business environment. Absolutely great for beginners! Best data science course I have come across so far!
I would highly recommend the course to any beginner who wants to venture into the world of Data Science. The concepts are very well explained and there is an emphasis on practical application which really helps create a better understanding of the concepts.
Ready to start?
Sign up today for FREE!
