Data Manipulation using pandas

Poor Data Presentation

One of the reasons that can cause types inconsistency may be poor data presentation. For instance, values of weight column may have also the measurment unit (like, 25kg, 14lb). In this case Python will understand these values as strings.

Let's see what is wrong with the values in the columns we considered to have wrong type.

# Importing the library

import pandas as pd

# Reading the file

df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/f2947b09-5f0d-4ad9-992f-ec0b87cd4b3f/data.csv')

# Output values of 'problematic' columns

print(df.loc[:,['totinch', 'morgh', 'valueh', 'grosrth', 'omphtotinch']])


              1234567
            
# Importing the library
import pandas as pd

# Reading the file
df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/f2947b09-5f0d-4ad9-992f-ec0b87cd4b3f/data.csv')
# Output values of 'problematic' columns
print(df.loc[:,['totinch', 'morgh', 'valueh', 'grosrth', 'omphtotinch']])

We found the root of the problem. All the columns but 'totinch' use dots . as indicator for missing values, while values in the 'totinch' column use commas , as the decimal separator. This may happen due to data origin, for instance. This problem can be solved by replacing commas with dots, and converting to float type.

Note, if you try to convert existing values into numeric type, then the error ValueError will be raised.

War alles klar?

Danke für Ihr Feedback!

Abschnitt 1. Kapitel 3

Fragen Sie AI

Fragen Sie alles oder probieren Sie eine der vorgeschlagenen Fragen, um unser Gespräch zu beginnen

some-alt

Data Manipulation using pandas

0%

1. Preprocessing Data: Part I

The data received from different sources can be messy, and to use it in the future, you must ensure it is convenient. In the first section, you will learn what data preprocessing is and will deal with some logical inconsistencies.

What is Data Preprocessing?

Types consistency

Poor Data Presentation

Manipulating Strings

Replacing Specific Elements

Simultaneous Replacement

2. Preprocessing Data: Part II

Missing or NA values, outliers, and inconsistencies are other types of problematic data. Throughout the second section of the course, you will learn how to deal with such issues.

Logical Inconsistency

Filling NA values

3. Grouping Data

As a data analyst, you will need to draw compact conclusions based on large amounts of data. In order to achieve that, you need to understand the data grouping idea and how to apply it to examples.

What is Grouping Data?

Grouping in pandas [1/2]

Grouping in pandas [2/2]

Grouping by Several Columns

4. Aggregating and Visualizing Data

Sometimes one built-in function is not enough to draw a complete conclusion, so you need to use something more complex. This section will teach you how to apply multiple functions while grouping. Also, you will learn to visualize data using the pandas library only and will be acquainted with the main plot types. As a data analyst, you will need to draw compact conclusions based on large amounts of data. In order to achieve that, you need to understand the data grouping idea and how to apply it to examples.

Advanced Aggregation [1/2]

Advanced Aggregation [2/2]

Bar and Scatter Plots

Other Types of Graphs

5. Joining Data

As mentioned before, sometimes you may need to work with data received from multiple sources. This section will teach you how to join two dataframes using different techniques.

What is Joining Data?