High-Dimensional Data and the Curse of Dimensionality
High-dimensional data has many features, or columns. As you add more dimensions, data points spread farther apart, and the space becomes increasingly empty. This makes it hard to find patterns, because the distances between points lose meaning. This is called the curse of dimensionalityβthe challenge of analyzing data when there are too many features.
1234567891011121314151617181920212223242526272829import numpy as np import matplotlib.pyplot as plt # Generate random points in 2D np.random.seed(0) points_2d = np.random.rand(100, 2) # Generate random points in 3D points_3d = np.random.rand(100, 3) fig = plt.figure(figsize=(12, 5)) # Plot 2D points ax1 = fig.add_subplot(1, 2, 1) ax1.scatter(points_2d[:, 0], points_2d[:, 1], color='blue', alpha=0.6) ax1.set_title('100 Random Points in 2D') ax1.set_xlabel('X') ax1.set_ylabel('Y') # Plot 3D points ax2 = fig.add_subplot(1, 2, 2, projection='3d') ax2.scatter(points_3d[:, 0], points_3d[:, 1], points_3d[:, 2], color='red', alpha=0.6) ax2.set_title('100 Random Points in 3D') ax2.set_xlabel('X') ax2.set_ylabel('Y') ax2.set_zlabel('Z') plt.tight_layout() plt.show()
Thanks for your feedback!
Ask AI
Ask AI
Ask anything or try one of the suggested questions to begin our chat
Can you explain more about why distances lose meaning in high dimensions?
What are some ways to deal with the curse of dimensionality?
Can you give real-world examples where the curse of dimensionality is a problem?
Awesome!
Completion rate improved to 8.33High-Dimensional Data and the Curse of Dimensionality
Swipe to show menu
High-dimensional data has many features, or columns. As you add more dimensions, data points spread farther apart, and the space becomes increasingly empty. This makes it hard to find patterns, because the distances between points lose meaning. This is called the curse of dimensionalityβthe challenge of analyzing data when there are too many features.
1234567891011121314151617181920212223242526272829import numpy as np import matplotlib.pyplot as plt # Generate random points in 2D np.random.seed(0) points_2d = np.random.rand(100, 2) # Generate random points in 3D points_3d = np.random.rand(100, 3) fig = plt.figure(figsize=(12, 5)) # Plot 2D points ax1 = fig.add_subplot(1, 2, 1) ax1.scatter(points_2d[:, 0], points_2d[:, 1], color='blue', alpha=0.6) ax1.set_title('100 Random Points in 2D') ax1.set_xlabel('X') ax1.set_ylabel('Y') # Plot 3D points ax2 = fig.add_subplot(1, 2, 2, projection='3d') ax2.scatter(points_3d[:, 0], points_3d[:, 1], points_3d[:, 2], color='red', alpha=0.6) ax2.set_title('100 Random Points in 3D') ax2.set_xlabel('X') ax2.set_ylabel('Y') ax2.set_zlabel('Z') plt.tight_layout() plt.show()
Thanks for your feedback!
