Data Preprocessing with NumPy

with Viktor Mehandzhiyski

4.8/5

(718)

This course will guide you through one of Python’s most notable packages – NumPy. We’ll explain why it’s so popular and discuss the numerous applications of its crown jewel – the ndarray class.

8 hours 68 lessons

Start course

68 High Quality Lessons

6 Practical Tasks

8 Hours of Content

Certificate of Achievement

Course Overview

This course is designed to show you how to work with one of Python’s fundamental packages – NumPy. You will learn what a “package” is and see how to install, upgrade and import it. By the time you finish the course, you’ll be comfortable with NumPy’ ndarray class, how to slice and reduce the dimensions of its instances, as well as how to quickly refer to the documentation. Furthermore, you’ll be ready to take advantage of NumPy’s various built-in functions and methods, which we’ll use to generate random and non-random data, import and export data to and from Python, find statistical values for a dataset, and clean and preprocess ndarrays.

Topics covered

data analysisData processingProgrammingPython

What You'll Learn

Do you already know how to use Python? Improve your skills by adding another tool to your arsenal – one of the most notable packages, NumPy!

Create arrays with NumPy

Use basic NumPy syntax

Generate data with NumPy

Implement statistical functions

Substitute missing values in Ndarrays

Explore ways to clean and preprocess data in NumPy

Curriculum

Introduction to NumPy Free

6 Lessons 19 Min

This introductory section presents the NumPy package and its applications. You’ll learn how to install and upgrade NumPy, before quickly learning about its most important assets – “arrays”. We’ll also go over how to use the documentation - an extremely useful component for our work later on in the course.

Course Introduction Free

The NumPy Package and Its Applications Free

Installing and Upgrading NumPy Free

What is an array? Free

Using The NumPy Documentation Free

Frequently Asked Questions Free
Why do we use NumPy? Free

3 Lessons 20 Min

This section follows NumPy’s role in the development of Python and takes a closer look at ndarrays. We discuss what makes them so useful and compare them to another similarly-looking data structure – NumPy lists.

History of NumPy Free

Ndarrays Free

Arrays vs Lists Free
NumPy Fundamentals

6 Lessons 29 Min

Here, we focus on the basic NumPy syntax. You’ll learn about “indexing” and the different ways of assigning values to an array. This section also explains the elementwise properties of arrays, as we go over the different types of data we can store in them. In addition, we’ll take a look at some of the most important characteristics and properties of NumPy functions.

Indexing

Assigning Values

Elementwise Properties

Types of Data Supported by NumPy

Characteristics of NumPy Functions - Part 1

Characteristics of NumPy Functions - Part 2
Working with Arrays

4 Lessons 27 Min

This section explores the concept of slicing and how its many variations can be applied to ndarrays. You’ll grasp what “dimensions” are when it comes to arrays and learn how the “reduce” function and method work.

Basic Slicing

Stepwise Slicing

Conditional Slicing

Dimensions and the Squeeze Function
Generating Data with NumPy

7 Lessons 32 Min

This part of the course explains how to generate arrays of random and non-random data. We begin by creating “empty” arrays, as well as basic arrays of 1s and 0s, before moving on to random generators. Then, we introduce NumPy’s capabilities of generating pseudo-random data pulled from a probability distribution. The section concludes with the applications of generating pseudo-random data.

Arrays of 0s and 1s

"_like" functions in NumPy

A Non-Random Sequence of Numbers

Random Generators and Seeds

Basic Random Functions in NumPy

Probability Distributions in NumPy

Applications of Random Data in NumPy
Importing and Saving Data with NumPy

6 Lessons 39 Min

This part of the course explains how to generate arrays of random and non-random data. We begin by creating “empty” arrays, as well as basic arrays of 1s and 0s, before moving on to random generators. Then, we introduce NumPy’s capabilities of generating pseudo-random data pulled from a probability distribution. The section concludes with the applications of generating pseudo-random data.

np.loadtxt() vs np.genfromtxt()

Simple Cleaning when Importing

String vs Object vs Numbers

np.save()

np.savez()

np.savetxt()
Statistics with NumPy

8 Lessons 42 Min

In this section of the course, we focus on importing and exporting, also known as saving data using the NumPy package. We discuss the differences between “np.loadtxt()” and “np.genfromtxt()” and their applications. We’ll examine NumPy’s capabilities to partially clean datasets as we import them. Later in the section, you’ll learn why you need to import a file into a specific datatype and how choosing the incorrect one can affect your results. We continue with the topic of saving ndarrays to external files where you’ll discover what N-P-Y and N-P-Z files are and when (and how) to export arrays in those formats. Finally, we provide you with a more conventional approach and showcase how to save arrays as text files.

Using Statistical Functions in NumPy

Minimal and Maximal Values in NumPy

Statistical Order Functions in NumPy

Averages and Variance in NumPy

Covariance and Correlation in NumPy

Histograms in NumPy (Part 1)

Histograms in NumPy (Part 2)

NAN Equivalent Functions in NumPy
Data Manipulation with NumPy

13 Lessons 95 Min

This section revolves around NumPy’s capabilities to compute important characteristics or statistics from an array. These include minimal and maximal values, various forms of averages, covariances, correlations as well as histograms. In addition, you’ll also learn about nan equivalent functions and how to use them.

Checking for Missing Values in Ndarrays

Substituting Missing Values in Ndarrays

Reshaping Ndarrays

Removing Values from Ndarrays

Sorting Ndarrays

Argument Sort in NumPy

Argument Where in NumPy

Shuffling Ndarrays

Casting Ndarrays

Striping Values from Ndarrays

Stacking Ndarrays

Concatenating Ndarrays

Finding Unique Vaules in Ndarrays
A Loan Data Practical Example with NumPy

15 Lessons 88 Min

In this part of the NumPy course, we explore ways to clean and preprocess data in NumPy. You’ll understand how to find and fill missing values, reshape an array, delete excess data as well as sort, shuffle and cast ndarrays. The section also explains what argument functions are and why they are so useful, and introduces ways to combining arrays by stacking and concatenating them. Finally, you’ll discover how to extract the unique values of an array and why this can be important for your analysis.

Setting Up: Introduction to the Practical Example

Setting Up: Importing the Data Set

Setting Up: Checking for Incomplete Data

Setting Up: Splitting the Dataset

Setting Up: Creating Checkpoints

Manipulating Text Data: Issue Date

Manipulating Text Data: Loan Status and Term

Manipulating Text Data: Grade and Sub Grade

Manipulating Text Data: Verification Status & URL

Manipulating Text Data: State Address

Manipulating Text Data: Converting Strings and Creating a Checkpoint

Manipulating Numeric Data: Substitute Filler Values

Manipulating Numeric Data: Currency Change – The Exchange Rate

Manipulating Numeric Data: Currency Change - From USD to EUR

Completing the Dataset:

Student feedback

4.8/5

718 ratings

5 stars

624 (87%)

4 stars

78 (11%)

3 stars

13 (2%)

2 stars

3 (0%)

1 star

0 (0%)

Rafael Rodrigues de Siqueira

09.08.2022

This last section with the practical example is the cherry of the cake. It helps widen the conceptualization taught in the course, beyond common grounds. People have different way of learning and the more approach you offer, the better. I have had a short journey in Data Science, yet it is evident that practice plays a huge part on it. Such activity takes you from 0 to 100 and greatly increases your confidence in the subject. Food for thought: I don't know how plausible it is, but if the same could be applied for the career track courses, where we could have all subjects wrapped up into a big work, that would be exceptional.

SHUVA MALLIK

21.04.2023

Overall the course is good for beginners. Problems could be of different types. Random generator section has achieved the prize of worst section of the entire course. I barely could understand the section. The instructor just uses poisson, logistics etc. distributions and said we should have statistical knowledge for this. But he could just explain the trials, events, occurrences etc. to us rather than just showing some code.

Eslam Mohamed Helmi

06.10.2022

Many of the questions asked in Q&A sections are not having a reply from the instructor for months Also, some of the videos have outdated information like that in section 5 [Generating Data with NumPy]

Jannery Rivas

18.11.2022

It feels really complete. After doing this course, you will feel confident in your Python and Numpy skills. After this, you should be able to complete a project by yourself.

Oskar Ciolka

13.04.2023

It's the best course I came onto yet. Shows everything straightforward, no humongous amounts of exercises like in an SQL course. Pretty thorough. Nice.

“A large portion of a data analyst’s work is dedicated to preprocessing datasets. Unquestionably, this involves tons of mathematical and statistical techniques that NumPy is renowned for. NumPy can be described as a computationally stable state-of-the-art Python instrument that provides flexibility and can take your analysis to the next level.”

Viktor Mehandzhiyski

Content Creator at 365 Data Science

Courses You May Like

Data Preprocessing with NumPy

with Viktor Mehandzhiyski

Start Course

Data Preprocessing with NumPy

Course Overview

Topics covered

What You'll Learn

Curriculum

Courses You May Like

The Complete Data Visualization Course with Python, R, Tableau, and Excel

Introduction to R Programming

Data Cleaning and Preprocessing with pandas

SQL

Python Programmer Bootcamp

Introduction to Jupyter

SQL + Tableau

Git and GitHub

Introduction to Python

Dates and Times in Python

Power Query and Data Modeling

Introduction to DAX

Building Business Reports Using Power BI

Working with Text Files in Python

Intermediate Python Programming

Advanced SQL