Data Manipulation using pandas

Data Manipulation using pandas

0%

1. Preprocessing Data: Part I

The data received from different sources can be messy, and to use it in the future, you must ensure it is convenient. In the first section, you will learn what data preprocessing is and will deal with some logical inconsistencies.

What is Data Preprocessing?

Types consistency

Poor Data Presentation

Manipulating Strings

Replacing Specific Elements

Simultaneous Replacement

2. Preprocessing Data: Part II

Missing or NA values, outliers, and inconsistencies are other types of problematic data. Throughout the second section of the course, you will learn how to deal with such issues.

Logical Inconsistency

Filling NA values

3. Grouping Data

As a data analyst, you will need to draw compact conclusions based on large amounts of data. In order to achieve that, you need to understand the data grouping idea and how to apply it to examples.

What is Grouping Data?

Grouping in pandas [1/2]

Grouping in pandas [2/2]

Grouping by Several Columns

4. Aggregating and Visualizing Data

Sometimes one built-in function is not enough to draw a complete conclusion, so you need to use something more complex. This section will teach you how to apply multiple functions while grouping. Also, you will learn to visualize data using the pandas library only and will be acquainted with the main plot types. As a data analyst, you will need to draw compact conclusions based on large amounts of data. In order to achieve that, you need to understand the data grouping idea and how to apply it to examples.

Advanced Aggregation [1/2]

Advanced Aggregation [2/2]

Bar and Scatter Plots

Other Types of Graphs

5. Joining Data

As mentioned before, sometimes you may need to work with data received from multiple sources. This section will teach you how to join two dataframes using different techniques.

What is Joining Data?

Contenido del Curso

Data Manipulation using pandas

Data Manipulation using pandas

1. Preprocessing Data: Part I

What is Data Preprocessing?Types consistency Poor Data Presentation Manipulating Strings Challenge Replacing Specific Elements Simultaneous Replacement Challenge

2. Preprocessing Data: Part II

Logical Inconsistency Removing Rows Challenge Outliers Challenge Missing Values Filling NA values Challenge

3. Grouping Data

What is Grouping Data?Grouping in pandas [1/2]Challenge Grouping in pandas [2/2]Challenge Grouping by Several Columns Challenge

4. Aggregating and Visualizing Data

Advanced Aggregation [1/2]Challenge Advanced Aggregation [2/2]Challenge Histograms Challenge Bar and Scatter Plots Other Types of Graphs Challenge 1 Challenge 2

5. Joining Data

What is Joining Data?Left Join Right Join Inner Join Outer Join Concatenation

Manipulating Strings

If you want to replace specific symbol in string, you need to apply the .replace() method, passing the element you want to replace as the first parameter, and the element that should take its place as the second parameter. For instance,


              12345
            
# Initial value with comma
string = "35600,0"
print(string)
# Replacing comma with dot
print(string.replace(',','.'))

As you can see, the comma was replaced by the dot symbol. But how to perform replacements for the whole column? On one side, we can run loop, and perform a replacement for every element. But it's very irrational.

pandas allows you to apply string methods to the whole column within one action. To do it, you need to set the str accessor to column, and then specify the method you want to perform. For instance, we can replace the - symbols in the address column with underscore _ characters.


              1234567
            
# Importing the library
import pandas as pd

# Reading the file
df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/f2947b09-5f0d-4ad9-992f-ec0b87cd4b3f/data.csv')
# Replacing - characters by _
print(df.address.str.replace('-', '_'))

And finally, you need to convert modified values into new type. pandas also allows us to perform that within one action by applying the .astype() method. The parameter of the method should be the type you want to convert to (int, float, str, etc.).

¿Todo estuvo claro?

¡Gracias por tus comentarios!

Sección 1. Capítulo 4

some-alt