Svep för att visa menyn

Replace Numerical Missing Data with Values

In the previous chapter, you dropped all the records containing NaN. If everything was done correct, the shape of new dataframe was (183, 12) - we lost 709 rows out of 891!

To avoid data loss because of NaNs, use different approaches to replace the missing data. The choice depends on the amount of missing data, its distribution(one or multiple columns, homogeneous or not, etc.).

The popular approaches to deal with NaNs for numerical data are:

replace with the mean value: good for the normal data distribution and when the number of NaNs is small.
replace with the mode value: good for exponential distributions and a small amount of NaNs.
replace with the max or min value: good if you are sure there are no outliers that may affect the result.
replace with some const value: for example, 0 or 1, if the possible value is either 0 or 1.

To replace the NaNs, you can use fillna():


              123
            
data.fillna(some_val) # replaces NaN with some_val
# or
data.fillna(some_val, inplace=True) # change the data in-place

Also, you can use replace(old_val, new_val) to replace not only NaNs, but any other values:


              123
            
data['Age'].replace(np.nan, 0) # replaces NaN with 0
# or
data['Age'].replace(np.nan, 0, inplace=True)

Do you remember that titanic dataset contains missing values in the Age column? Instead of dropping rows, let's think how to replace NaNs and save data.

Uppgift

Swipe to start coding

If share of NaNs is low enough, replace them with value - but which one? Do the following:

Calculate the share of missing values in Age column. Round this value to 2 decimal places.
Build the histogram of Age distribution. Use matplotlib.pyplot and method hist().

Lösning

Well, hope you have the similar histogram:

If yes, move on to the next chapter to deal with NaNs.

Byt till skrivbordet för praktisk övningFortsätt där du är med ett av alternativen nedan

Var allt tydligt?

Tack för dina kommentarer!

Avsnitt 2. Kapitel 3

Fråga AI

Fråga vad du vill eller prova någon av de föreslagna frågorna för att starta vårt samtal

Replace Numerical Missing Data with Values

In the previous chapter, you dropped all the records containing NaN. If everything was done correct, the shape of new dataframe was (183, 12) - we lost 709 rows out of 891!

The popular approaches to deal with NaNs for numerical data are:

replace with the mean value: good for the normal data distribution and when the number of NaNs is small.
replace with the mode value: good for exponential distributions and a small amount of NaNs.
replace with the max or min value: good if you are sure there are no outliers that may affect the result.
replace with some const value: for example, 0 or 1, if the possible value is either 0 or 1.

To replace the NaNs, you can use fillna():


              123
            
data.fillna(some_val) # replaces NaN with some_val
# or
data.fillna(some_val, inplace=True) # change the data in-place

Also, you can use replace(old_val, new_val) to replace not only NaNs, but any other values:


              123
            
data['Age'].replace(np.nan, 0) # replaces NaN with 0
# or
data['Age'].replace(np.nan, 0, inplace=True)

Do you remember that titanic dataset contains missing values in the Age column? Instead of dropping rows, let's think how to replace NaNs and save data.

Uppgift

Swipe to start coding

If share of NaNs is low enough, replace them with value - but which one? Do the following:

Calculate the share of missing values in Age column. Round this value to 2 decimal places.
Build the histogram of Age distribution. Use matplotlib.pyplot and method hist().

Lösning

Well, hope you have the similar histogram:

If yes, move on to the next chapter to deal with NaNs.

Byt till skrivbordet för praktisk övningFortsätt där du är med ett av alternativen nedan

Var allt tydligt?

Tack för dina kommentarer!

Avsnitt 2. Kapitel 3

Byt till skrivbordet för praktisk övningFortsätt där du är med ett av alternativen nedan