Calculating the Pearson Coefficient Using NumPy and Pandas

Let's look at how we can calculate the correlation coefficient if our data's type is np.array. The library has many statistics routines which simplify the calculations. We will use the method np.corrcoef(). It works with 2 arrays of the same length of our data:

# Import the libraries

import numpy as np

# Define np.arrays

x = np.array([1, 2, 3, 5, 7, 8, 10, 11, 13, 15])

y = np.array([2, 4, 7, 8, 10, 15, 20, 21, 23, 30])

# Find correlation

r = np.corrcoef(x, y)


              123456789
            
# Import the libraries
import numpy as np

# Define np.arrays
x = np.array([1, 2, 3, 5, 7, 8, 10, 11, 13, 15])
y = np.array([2, 4, 7, 8, 10, 15, 20, 21, 23, 30])

# Find correlation
r = np.corrcoef(x, y)


python
9
1
2
3
Output:
 [[1.         0.98661904]
  [0.98661904 1.        ]]

This function returns the correlation matrix (2-dimensional array) of correlation coefficients. Here is a more convenient version of the array:

The upper right value corresponds to the correlation coefficient for y and x, while the lower-left value is the correlation coefficient for x and y. These values we will always need. The other ones are the correlation coefficients between x and x, y and y. They are always equal to one.

If you want just the Pearson coefficient between x and y use this:

print(np.corrcoef(x, y)[0,1])


              1
            
print(np.corrcoef(x, y)[0,1])

Pandas correlation calculations also has a function to calculate the correlation coefficient for two of the same length Series objects. You can use .corr() method:

# Import the libraries

import pandas as pd

# Define series

x = pd.Series([1, 2, 3, 5, 7, 8, 10, 11, 13, 15])

y = pd.Series([2, 4, 7, 8, 10, 15, 20, 21, 23, 30])

# Print correlation coeffitients

print(x.corr(y))

print(y.corr(x))


              12345678910
            
# Import the libraries
import pandas as pd

# Define series
x = pd.Series([1, 2, 3, 5, 7, 8, 10, 11, 13, 15])
y = pd.Series([2, 4, 7, 8, 10, 15, 20, 21, 23, 30])

# Print correlation coeffitients
print(x.corr(y))
print(y.corr(x))


python
9
1
2
3
Output:
  0.9866190374718473
  0.9866190374718473

Task

Swipe to start coding

You have the initial dataset of Abyssinian cats' weight and height (x and y arrays, respectively). Find the correlation coefficient between x and y using all functions we discussed in this chapter.

[Lines #2-3] Import the pandas, numpy libraries.
[Lines #10-11] Change the type of arrays to np.arrays, find the correlation coefficient between x and y.
[Line #12] Print the correlation coefficient you have found in a such way.
[Lines #15-16] Change the type of arrays to Pandas Series , find the correlation coefficient between x and y.
[Line #17] Print the correlation coefficient you have found in a such way.

Solution

# Import the libraries

import pandas as pd

import numpy as np

# Initialize the data

x = [8, 10, 9.2, 8.4, 9.1, 9.6, 8, 10.2, 9.3, 9.4, 9.9, 8.7]

y = [3.6, 5.4, 4.8, 3.9, 4.2, 5.2, 3.5, 5.5, 4.4, 4.7, 5.1, 3.7]

# Find and print the correlation coefficient with np.arrays

np_x = np.array(x)

np_y = np.array(y)

print(np.corrcoef(np_x, np_y)[0,1])

# Find and print the correlation coefficient with pd.Series

pd_x = pd.Series(x)

pd_y = pd.Series(y)

print(pd_x.corr(pd_y))

Everything was clear?

Thanks for your feedback!

Section 2. Chapter 3

# Import the libraries

import pandas as ___

import ___ as np

# Initialize the data

x = [8, 10, 9.2, 8.4, 9.1, 9.6, 8, 10.2, 9.3, 9.4, 9.9, 8.7]

y = [3.6, 5.4, 4.8, 3.9, 4.2, 5.2, 3.5, 5.5, 4.4, 4.7, 5.1, 3.7]

# Find and print the correlation coefficient with np.arrays

np_x = np.array(___)

np_y = ___

print(np.___(np_x, np_y)[0,1])

# Find and print the correlation coefficient with pd.Series

pd_x = ___

pd_y = ___

print(___)

Ask AI

Ask anything or try one of the suggested questions to begin our chat