Pandas - Mati & Caleb
This lesson introduces students to the Pandas library in Python for data analysis and manipulation, covering topics such as data loading, table creation, manipulation, and visualization using real-world examples.
- before we start this portion of the lesson:
- Overview:
- Learning Objectives:
- what is pandas?
- Question #2 & 3:
- but why is pandas useful?
- Question #4:
- how do i flipping use it? its so hard, my puny brain cant understand it
- example code on how to load a csv into a chart
- how to manipulate the data in pandas.
- how do i put it into a chart 😩
- Hacks
- note
before we start this portion of the lesson:
check if you have pip installed since we are going to be installing some libraries today!!!!!! if you arnt sure if you have pip, check it by running this command:
pip
if your terminal says "command not found" or something else on linux, run this:
python3 -m ensurepip --default-pip
Overview:
Pandas is a powerful tool in Python that is used for data analysis and manipulation. In this lesson, we will explore how to use Pandas to work with datasets, analyze them, and visualize the results.
Learning Objectives:
By the end of this lesson, students should be able to:
- Understand what Pandas is and why it is useful for data analysis
- Load data into Pandas and create tables to store it
- Use different functions in Pandas to manipulate data, such as filtering, sorting, and grouping
- Visualize data using graphs and charts
what is pandas?
this:
- Pandas is a Python library used for data analysis and manipulation.
- it can handle different types of data, including CSV files and databases.
- it also allows you to create tables to store and work with your data.
- it has functions for filtering, sorting, and grouping data to make it easier to work with.
- it also has tools for visualizing data with graphs and charts.
- it is widely used in the industry for data analysis and is a valuable skill to learn.
- companies that use Pandas include JPMorgan Chase, Google, NASA, the New York Times, and many others.
but why is pandas useful?
- it can provides tools for handling and manipulating tabular data, which is a common format for storing and analyzing data.
- it can handle different types of data, including CSV files and databases.
- it allows you to perform tasks such as filtering, sorting, and grouping data, making it easier to analyze and work with.
- it has functions for handling missing data and can fill in or remove missing values, which is important for accurate data analysis.
- it also has tools for creating visualizations such as graphs and charts, making it easier to communicate insights from the data.
- it is fast and efficient, even for large datasets, which is important for time-critical data analysis.
- it is widely used in the industry and has a large community of users and developers, making it easy to find support and resources.
import pandas as pd
df = pd.read_csv('files/example.csv')
print(df.head())
print("Average age:", df['Age'].mean())
females = df[df['Gender'] == 'Female']
print(females)
sorted_data = df.sort_values(by='Salary', ascending=False)
print(sorted_data)
uh oh!!! no pandas 😢
if see this error, enter these into your terminal:
pip install wheel
pip install pandas
on stack overflow, it said pandas is disturbed through pip as a wheel. so you need that too.
link to full forum if curious: https://stackoverflow.com/questions/33481974/importerror-no-module-named-pandas
ps: do this for this to work on ur laptop:
wget https://raw.githubusercontent.com/KKcbal/amongus/master/_notebooks/files/example.csv
import pandas as pd
# read the CSV file
df = pd.read_csv('files/example.csv')
# print the first five rows
print(df.head())
# define a function to assign each age to an age group
def assign_age_group(age):
if age < 30:
return '<30'
elif age < 40:
return '30-40'
elif age < 50:
return '40-50'
else:
return '>50'
# apply the function to the Age column to create a new column with age groups
df['Age Group'] = df['Age'].apply(assign_age_group)
# group by age group and count the number of people in each group
age_counts = df.groupby('Age Group')['Name'].count()
# print the age group counts
print(age_counts)
import pandas as pd
# load the csv file
df = pd.read_csv('example.csv')
# print the first five rows
print(df.head())
# filter the data to include only people aged 30 or older
df_filtered = df[df['Age'] >= 30]
# sort the data by age in descending order
df_sorted = df.sort_values('Age', ascending=False)
# group the data by gender and calculate the mean age for each group
age_by_gender = df.groupby('Gender')['Age'].mean()
# print the filtered data
print(df_filtered)
# print the sorted data
print(df_sorted)
# print the mean age by gender
print(age_by_gender)
import pandas as pd
import matplotlib.pyplot as plt
# read the CSV file
df = pd.read_csv('files/example.csv')
# create a bar chart of the number of people in each age group
age_groups = ['<30', '30-40', '40-50', '>50']
age_counts = pd.cut(df['Age'], bins=[0, 30, 40, 50, df['Age'].max()], labels=age_groups, include_lowest=True).value_counts()
plt.bar(age_counts.index, age_counts.values)
plt.title('Number of people in each age group')
plt.xlabel('Age group')
plt.ylabel('Number of people')
plt.show()
# create a pie chart of the gender distribution
gender_counts = df['Gender'].value_counts()
plt.pie(gender_counts.values, labels=gender_counts.index, autopct='%1.1f%%')
plt.title('Gender distribution')
plt.show()
# create a scatter plot of age vs. income
plt.scatter(df['Age'], df['Income'])
plt.title('Age vs. Income')
plt.xlabel('Age')
plt.ylabel('Income')
plt.show()
uh oh!!!! another error!??!!??!?! install this library:
pip install matplotlib
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
# read the CSV file
df = pd.read_csv('files/example.csv')
# define age groups
age_groups = ['<30', '30-40', '40-50', '>50']
# create a new column with the age group for each person
df['Age Group'] = pd.cut(df['Age'], bins=[0, 30, 40, 50, np.inf], labels=age_groups, include_lowest=True)
# group by age group and count the number of people in each group
age_counts = df.groupby('Age Group')['Name'].count()
# create a bar chart of the age counts
age_counts.plot(kind='bar')
# set the title and axis labels
plt.title('Number of People in Each Age Group')
plt.xlabel('Age Group')
plt.ylabel('Number of People')
# show the chart
plt.show()
magic!!!!!!
Hacks
- make your own data using your brian, google or chatgpt, should look different than mine.
- modify my code or write your own
- output your data other than a bar graph.
- write an 850+ word essay on how pandas, python or irl, affected your life. If AI score below 85%, then -1 grading point (CRINGE DONT USE)
- answer the questions below, the more explained the better.
Questions
- What are the two primary data structures in pandas and how do they differ?
- How do you read a CSV file into a pandas DataFrame?
- How do you select a single column from a pandas DataFrame?
- How do you filter rows in a pandas DataFrame based on a condition?
- How do you group rows in a pandas DataFrame by a particular column?
- How do you aggregate data in a pandas DataFrame using functions like sum and mean?
- How do you handle missing values in a pandas DataFrame?
- How do you merge two pandas DataFrames together?
- How do you export a pandas DataFrame to a CSV file?
- What is the difference between a Series and a DataFrame in Pandas?
note
all hacks due saturday night, the more earlier you get them in the higher score you will get. if you miss the due date, you will get a 0. there will be no tolerance.
no questions answered
Tonight- 2.9
Friday Night- 2.8
Saturday Night - 2.7
Sunday Night - 0.0
questions answered
Tonight- 3.0
Friday Night- 2.9
Saturday Night - 2.8
Sunday Night - 0.0
wdfasdf