Search for a command to run...
👋 Hi there, I am Nirmal. I am a former software engineer with a keen interest in data science and analytics domains. Besides, I love to contribute to open source and help others to understand various tech stuffs.
No comments yet. Be the first to comment.
This series will explore on the latest trends and practical applications in data science and analytics, empowering professionals with expert perspectives and actionable insights.
The era of Big Data has ushered in a new paradigm of data-driven decision-making, revolutionizing industries across the globe. To harness the power of Big Data effectively, organizations rely on a diverse array of tools and technologies that cover da...
Bias in Machine Learning Bias refers to the simplifying assumptions made by a model to make the target function easier to learn. High bias can lead to underfitting Represents the error introduced by approximating a real-world problem Example: A li...
When it comes to developing software applications, choosing the appropriate architectural style is a critical decision. Among the popular options are microservices and monolithic architecture. By understanding the trade-offs, readers can make well-i...
In a world where information is everywhere, businesses have tons of data at their fingertips. This data is like a goldmine waiting to be used for smart decisions. That's where Big Data analytics and Data Science come in. They're like the experts who ...
NumPy is a powerful library for numerical computing in Python, offering robust support for large, multi-dimensional arrays and an extensive collection of mathematical functions. Creating Arrays To begin using NumPy, import the library and create arra...
Python is a powerful tool for data professionals, largely due to its extensive collection of open-source libraries and packages. In this blog, we'll explore the fundamentals of pandas, with a special focus on its core data structures: Series and DataFrame.
A Series in pandas is akin to a column in a spreadsheet or a one-dimensional NumPy array. It’s a labeled array that can hold any data type, with each element having an associated label called an index. This indexing feature allows for efficient and intuitive data manipulation.
Here’s a quick look at how to create and use a Series:
import pandas as pd
# Creating a Series
s = pd.Series([1, 2, 3, 4, 5])
print(s)
Output:
0 1
1 2
2 3
3 4
4 5
dtype: int64
A DataFrame is the heart of pandas, representing a two-dimensional labeled data structure with columns and rows, similar to a table or spreadsheet. Each column in a DataFrame is a Series.
Here’s how to create a DataFrame:
# Creating a DataFrame from a dictionary
data = {'col1': [1, 2], 'col2': [3, 4]}
df = pd.DataFrame(data)
print(df)
Output:
col1 col2
0 1 3
1 2 4
# Creating a DataFrame from a NumPy array
import numpy as np
df2 = pd.DataFrame(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), columns=['a', 'b', 'c'])
print(df2)
Output:
a b c
0 1 2 3
1 4 5 6
2 7 8 9
DataFrames come with a plethora of built-in attributes and methods that simplify common data analysis tasks. Here are some of the most commonly used ones:
columns: Returns the column labels of the DataFrame.
dtypes: Returns the data types of the columns.
iloc: Accesses rows and columns using integer-based indexing.
loc: Accesses rows and columns by labels or Boolean arrays.
shape: Returns a tuple representing the dimensionality.
values: Returns a NumPy representation of the DataFrame.
Examples:
print(df2.columns)
print(df2.dtypes)
print(df2.shape)
print(df2.values)
Output:
Index(['a', 'b', 'c'], dtype='object')
a int64
b int64
c int64
dtype: object
(3, 3)
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
apply(): Applies a function along an axis.
copy(): Creates a copy of the DataFrame.
describe(): Provides descriptive statistics.
drop(): Drops specified labels from rows or columns.
groupby(): Groups DataFrame using a mapper or by a Series of columns.
head(): Returns the first n rows.
info(): Prints a concise summary of the DataFrame.
isna(): Detects missing values.
sort_values(): Sorts by the values along an axis.
value_counts(): Returns a Series containing counts of unique values.
Example:
print(df2.describe())
Output:
a b c
count 3.000000 3.000000 3.000000
mean 4.000000 5.000000 6.000000
std 3.000000 3.000000 3.000000
min 1.000000 2.000000 3.000000
25% 2.500000 3.500000 4.500000
50% 4.000000 5.000000 6.000000
75% 5.500000 6.500000 7.500000
max 7.000000 8.000000 9.000000
Selecting and manipulating data within a DataFrame is a crucial skill. Pandas offers several ways to achieve this.
loc[]:Select rows by label:
df = pd.DataFrame({
'A': ['alpha', 'apple', 'arsenic', 'angel', 'android'],
'B': [1, 2, 3, 4, 5],
'C': ['coconut', 'curse', 'cassava', 'cuckoo', 'clarinet'],
'D': [6, 7, 8, 9, 10]
}, index=[0, 1, 2, 3, 4])
print(df.loc[1])
print(df.loc[[1, 3]])
Output:
A apple
B 2
C curse
D 7
Name: 1, dtype: object
A B C D
1 apple 2 curse 7
3 angel 4 cuckoo 9
iloc[]:Select rows by position:
print(df.iloc[1])
print(df.iloc[[0, 2]])
Output:
A apple
B 2
C curse
D 7
Name: 1, dtype: object
A B C D
0 alpha 1 coconut 6
2 arsenic 3 cassava 8
print(df['C'])
print(df[['A', 'C']])
Output:
0 coconut
1 curse
2 cassava
3 cuckoo
4 clarinet
Name: C, dtype: object
A C
0 alpha coconut
1 apple curse
2 arsenic cassava
3 angel cuckoo
4 android clarinet
print(df.A)
Output:
0 alpha
1 apple
2 arsenic
3 angel
4 android
Name: A, dtype: object
loc[]:print(df.loc[0:2, ['A', 'C']])
Output:
A C
0 alpha coconut
1 apple curse
2 arsenic cassava
iloc[]:print(df.iloc[[2, 4], 0:3])
Output:
A B C
2 arsenic 3 cassava
4 android 5 clarinet
Pandas DataFrames are an essential tool for working with tabular data. Each row and column in a DataFrame is represented by a pandas Series, making data manipulation intuitive and efficient. The robust suite of methods and attributes available in pandas allows for sophisticated data operations with minimal code. As you gain experience with pandas, you’ll find it an invaluable tool in your data science toolkit.
For more detailed information, refer to the official pandas documentation:
By mastering the fundamentals of pandas, you'll be on your way to becoming a proficient and effective data professional.