DATA SCIENCE USING PYTHON LAB

1. Creating a NumPy Array

A. Basic ndarray
In NumPy, a ndarray (n-dimensional array) is the primary data
structure used to store and manipulate numerical data.

import numpy as np
arr1 = [Link]([1, 2, 3, 4, 5])
print(arr1)

Output:
[1 2 3 4 5]
B. Array of Zeroes
You can create an array of zeroes in NumPy using the [Link]()
function.

import numpy as np
arr1 = [Link](5)
print(arr1)

Output:
[0. 0. 0. 0. 0.]
C. Array of ones
You can create an array of ones in NumPy using the [Link]()
function.

import numpy as np
arr1 = [Link](5)
print(arr1)

Output:
 [1. 1. 1. 1. 1.]
D. Random numbers in ndarray
You can generate random numbers in a NumPy array using the
[Link].

import numpy as np
arr1=[Link](3, 4) # 3x4 array of random floats between 0 and
1
print(arr1)

Output (varies each time due to randomness):

[[0.37454012 0.95071431 0.73199394 0.59865848]
[0.15601864 0.15599452 0.05808361 0.86617615]
[0.60111501 0.70807258 0.02058449 0.96990985]]
F. Imatrix in NumPy
In NumPy, you can create an identity matrix using the [Link]()
function. It produces square matrices with 1s on the diagonal and 0s
elsewhere.

import numpy as np
i_matrix = [Link](4) # Creates a 4x4 identity matrix
print(i_matrix)

Output:
[[1. 0. 0. 0.]
[0. 1. 0. 0.]
 [0. 0. 1. 0.]
[0. 0. 0. 1.]]
G. Evenly spaced ndarray
You can create an evenly spaced ndarray in NumPy using either the
[Link]()

import numpy as np
arr = [Link](0, 10, 5) # 5 evenly spaced numbers from 0 to 10
print(arr)

Output:
[ 0. 2.5 5. 7.5 10. ]

2. The Shape and Reshaping of NumPy Array

A. Dimensions of NumPy Array
In NumPy, the dimensions of an array refer to the number of axes (or
levels of indexing) in the array. You can determine the dimensions using
the .ndim attribute.

import numpy as np
# 1D array
arr1 = [Link]([1, 2, 3, 4])
print([Link]) # Output: 1
# 2D array
arr2 = [Link]([[1, 2, 3], [4, 5, 6]])
print([Link]) # Output: 2
 # 3D array
arr3 = [Link]([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
print([Link]) # Output: 3
B. Shape of NumPy Array
The shape of a NumPy array tells you the size of each dimension.
import numpy as np
arr = [Link]([[1, 2, 3], [4, 5, 6]])
print([Link]) # Output: (2, 3)
This output means:
The array has 2 rows and 3 columns.
C. Size of NumPy Array
The size gives the total number of elements in the array.
arr = [Link]([[1, 2, 3], [4, 5, 6]])
print([Link]) # Output: 6
D. Reshaping a NumPy Array
In NumPy, the .reshape() function is used to change the shape of an
array without modifying its data.
Reshaping a 1D Array to 2D:
import numpy as np
arr = [Link]([1, 2, 3, 4, 5, 6])
reshaped_arr = [Link](2, 3) # Reshape to 2 rows and 3 columns
print(reshaped_arr)

Output:
[[1 2 3]
 [4 5 6]]
E. Flattening a NumPy Array
Flattening a NumPy array means converting a multi-dimensional array
into a 1D array.
Flattening a 2D Array:

import numpy as np
arr = [Link]([[1, 2, 3], [4, 5, 6]])
# Using flatten()
flat1 = [Link]()
print("Using flatten():", flat1)

Output:
Using flatten(): [1 2 3 4 5 6]
F. Transpose of a NumPy Array
In NumPy, the transpose of an array is obtained by swapping its axes.

import numpy as np
arr = [Link]([[1, 2, 3], [4, 5, 6]])
print("Original Array:")
print(arr)
print("\nTransposed Array (using transpose()):")
print([Link](arr))

Output:
 Original Array:
[[1 2 3]
[4 5 6]]
Transposed Array (using transpose()):
[[1 4]
[2 5]
[3 6]]

3. Expanding and Squeezing a NumPy Array

A. Expanding a NumPy Array
Adding a New Axis (Changing Shape)
You can add a new axis to change the dimensionality of an array using
np.expand_dims()

import numpy as np
arr = [Link]([1, 2, 3])
# Add a new axis at position 0 (convert 1D to 2D)
expanded_arr = np.expand_dims(arr, axis=0)

# Output: [[1 2 3]]

B. Squeezing a NumPy Array
Squeezing a NumPy array means removing dimensions. You can do this
using [Link]()

import numpy as np
arr = [Link]([[[1], [2], [3]]]) # Shape: (1, 3, 1)
 squeezed_arr = [Link](arr)
print(squeezed_arr.shape) # Output: (3,)
print(squeezed_arr)
# [1 2 3]
C. Sorting in NumPy Arrays
import numpy as np
arr = [Link]([3, 1, 2, 5, 4])
sorted_arr = [Link](arr)
print(sorted_arr)

# Output: [1 2 3 4 5]
4. Indexing and Slicing of NumPy Array
A. Slicing 1-D NumPy arrays
Slicing in python means taking elements from one given index to
another given index.
We pass slice instead of index like this: [start:end].

import numpy as np
arr = [Link]([1, 2, 3, 4, 5, 6, 7])
print(arr[1:5])

Output:
[2 3 4 5]
B. Slicing 2-D NumPy arrays
 import numpy as np
arr = [Link]([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]])
print(arr[1, 1:4])

Output:
[7 8 9]
C. Slicing 3-D NumPy arrays
import numpy as np
# Creating a 3-D array (2x3x4) arr
= [Link]([[[1, 2, 3, 4],
[5, 6, 7, 8],
[9, 10, 11, 12]],
[[13, 14, 15, 16],
[17, 18, 19, 20],
[21, 22, 23, 24]]])
print("Original array:\n", arr)
# Extracting elements from the first "depth" layer print("\
nFirst depth layer (arr[0]):\n", arr[0])

Output: Original
array: [[[ 1 2 3
4]
[ 5 6 7 8]
[ 9 10 11 12]]
[[13 14 15 16]
[17 18 19 20]
 [21 22 23 24]]]
First depth layer (arr[0]):
[[ 1 2 3 4]
[ 5 6 7 8]
[ 9 10 11 12]]
D. Negative slicing of NumPy arrays
Use the minus operator to refer to an index from the end:
Slice from the index 3 from the end to index 1 from the end:

import numpy as np
arr = [Link]([1, 2, 3, 4, 5, 6, 7])
print(arr[-3:-1])

Output:
[5 6]

5. Stacking and Concatenating NumPy Arrays

A. Stacking ndarrays
Vertical Stacking (vstack): Stacks arrays in sequence vertically (row-wise).
import numpy as np
a = [Link]([1, 2, 3])
b = [Link]([4, 5, 6])
result = [Link]((a, b))
print(result)

Output:
 [[1 2 3]
[4 5 6]]

Horizontal Stacking (hstack): Stacks arrays in sequence horizontally

(column-wise).
result = [Link]((a, b))
print(result)

Output:
[1 2 3 4 5 6]
B. Concatenating ndarrays
Concatenating arrays in NumPy is done using the [Link]()
function. This function joins a sequence of arrays along an existing axis.
Concatenate Along Rows (Axis 0):

import numpy as np
a = [Link]([[1, 2], [3, 4]])
b = [Link]([[5, 6], [7, 8]])
result = [Link]((a, b), axis=0)
print(result)

Output:
[[1 2]
[3 4]
[5 6]
[7 8]]

Concatenate Along Columns (Axis 1):

result = [Link]((a, b), axis=1)
print(result)
 Output:
[[1 2 5 6]
[3 4 7 8]]

C. Broadcasting in NumPy Arrays

Broadcasting in NumPy is a powerful mechanism that allows NumPy to
perform arithmetic operations on arrays of different shapes
Scalar and Array Broadcasting:

import numpy as np
a = [Link]([1, 2, 3])
b = 2 # Scalar
result = a * b
print(result)

Output:
[2 4 6]

Broadcasting with Different Shapes:

a = [Link]([[1, 2, 3], [4, 5, 6]])
b = [Link]([10, 20,
30]) result = a + b
print(result)

Output:
[[11 22 33]
[14 25 36]]
6. Perform following operations using pandas
A. Creating Dataframe
A DataFrame is essentially a 2-dimensional, size-mutable, and
heterogeneous data structure with labeled axes (rows and columns)

import pandas as pd data

={
"Name": ["Alice", "Bob", "Charlie"], "Age":
[25, 30, 35],
"City": ["New York", "Los Angeles", "Chicago"]
}
df = [Link](data)
print(df)

Output:
Name Age City
0 Alice 25 New York
1 Bob 30 Los Angeles
2 Charlie 35 Chicago
B. concat()
The [Link]() function is used to concatenate (combine) two or
more DataFrames along a particular axis (either rows or columns). It’s
super flexible and efficient when you need to merge or join data.

Concatenating DataFrames Vertically (Row-wise):

import pandas as pd
df1 = [Link]({"Name": ["Alice", "Bob"], "Age": [25, 30]})
 df2 = [Link]({"Name": ["Charlie", "David"], "Age": [35, 40]})
result = [Link]([df1, df2])
print(result)

Output:
Name Age
0 Alice 25
1 Bob 30
0 Charlie 35
1 David 40
C. Setting Conditions:
In Pandas, you can set conditions to filter, update, or manipulate
DataFrames based on specific criteria.
1. Filtering Rows Based on a Condition

import pandas as pd
data = {
"Name": ["Alice", "Bob", "Charlie", "David"],
"Age": [25, 30, 35, 40],
"Salary": [50000, 60000, 70000, 80000]
}

df = [Link](data)

# Get rows where Age is greater than 30

filtered_df = df[df["Age"] > 30]
print(filtered_df)

Output:
Name Age Salary
 2 Charlie 35 70000
3 David 40 80000

2. Updating Values Based on a Condition

The most straightforward and efficient way to update values based
on a condition is to use loc:

import pandas as pd
# Sample DataFrame
data = {
"Name": ["Alice", "Bob", "Charlie", "David"],
"Age": [25, 30, 35, 40],
"Salary": [50000, 60000, 70000, 80000]
}
df = [Link](data)

# Update salary where age is greater than 30

[Link][df["Age"] > 30, "Salary"] = 90000
print(df)

Output:
Name Age Salary
0 Alice 25 50000
1 Bob 30 60000
2 Charlie 35 90000
3 David 40 90000

D. Adding a new column

1. Adding a Column with a Constant Value
You can directly assign a constant value to a new column:

import pandas as pd
 # Sample DataFrame
data = {
"Name": ["Alice", "Bob", "Charlie", "David"],
"Age": [25, 30, 35, 40]
}
df = [Link](data)
# Add a new column with a constant value
df["Salary"] = 50000
print(df)

Output:

Name Age Salary

0 Alice 25 50000
1 Bob 30 50000
2 Charlie 35 50000
3 David 40 50000

7. Perform following operations using Pandas

A. Filling NaN with string
You can fill NaN values in a Pandas DataFrame with a specific string using
the fillna() method.

import pandas as pd
import numpy as np
# Create a sample DataFrame with NaN values
 data = {
'Name': ['Alice', 'Bob', [Link], 'David'],
'Age': [24, [Link], 30, [Link]],
'City': ['New York', [Link], 'Chicago', 'Los Angeles']
}
df = [Link](data)
# Fill NaN values with a specific string
df_filled = [Link]('Unknown')
print(df_filled)

Output:
Name Age City
0 Alice 24 New York
1 Bob Unknown Unknown
2 Unknown 30 Chicago
3 David Unknown Los Angeles

B. Sorting based on column values

In Pandas, you can sort a DataFrame based on the values of one or more
columns using the sort_values() method.
Syntax:
df.sort_values(by='column_name', ascending=True)
by: The column name(s) to sort by.
ascending: If True, sorts in ascending order; if False, sorts in descending
order.
Sorting by a Single Column:
 import pandas as pd
data = {
'Name': ['Alice', 'Bob', 'Charlie', 'David'],
'Age': [24, 30, 22, 29],
'Score': [88, 95, 92, 85]
}
df = [Link](data)
# Sort by 'Age' in ascending order
sorted_df = df.sort_values(by='Age')
print(sorted_df)

Output:

Name Age Score

2 Charlie 22 92
0 Alice 24 88
3 David 29 85
1 Bob 30 95

C. groupby()
The groupby() function in Pandas is used to group data based on one or
more columns and then perform aggregation or transformation
operations on the grouped data. It is similar to the "GROUP BY" clause in
SQL.
Grouping and Summing

import pandas as pd
 data = {
'Department': ['HR', 'IT', 'HR', 'Finance', 'IT', 'Finance'],
'Employee': ['Alice', 'Bob', 'Charlie', 'David', 'Eve', 'Frank'],
'Salary': [50000, 60000, 55000, 70000, 62000, 68000]
}
df = [Link](data)
# Group by Department and sum the salaries
grouped = [Link]('Department').sum()
print(grouped)

Output:
Department Salary
Finance 138000
HR 105000
IT 122000

8. Read the following file formats using pandas

A. Text files
B. CSV files
In Pandas, you can read text files (like CSV or plain text) using the
pd.read_csv() function.
Reading a Comma-Separated File (CSV)
Suppose you have a file called [Link] with the following content:
Name, Age, City
Alice, 24, New York
 Bob, 30, San Francisco
Charlie, 22, Chicago

Code:
df = pd.read_csv('[Link]')
print(df)

Output:
Name Age City
0 Alice 24 New York
1 Bob 30 San Francisco
2 Charlie 22 Chicago

B. Excel Files
You can read Excel files in Pandas using the pd.read_excel() function. It
works with both .xls and .xlsx file formats.
Reading a Single Sheet
Assuming you have an Excel file [Link] with the following data in
Sheet1:

Name Age City

Alice 24 New York

Bob 30 San Francisco

Charlie 22 Chicago

Code:
 df = pd.read_excel('[Link]', sheet_name='Sheet1')
print(df)

Output:
Name Age City
0 Alice 24 New York
1 Bob 30 San Francisco
2 Charlie 22 Chicago

Installing Required Library

Make sure you have the openpyxl library installed for .xlsx files:
pip install openpyxl
For older Excel files (.xls), you may also need xlrd:
pip install xlrd
D. JSON FILES
You can read JSON files in Pandas using the pd.read_json() function. It
can handle various JSON formats and convert them into DataFrames.

import pandas as pd
df = pd.read_json('[Link]')
[Link]: Path to your JSON file.
Returns: A DataFrame containing the parsed JSON data.
Reading a Simple JSON File
Assuming you have a file called [Link] with the following content:
[
 {"Name": "Alice", "Age": 24, "City": "New York"},
{"Name": "Bob", "Age": 30, "City": "San Francisco"},
{"Name": "Charlie", "Age": 22, "City": "Chicago"}
]
Code:
import pandas as pd
# Reading the JSON file
df = pd.read_json('[Link]')
print(df)

Output:
Name Age City
0 Alice 24 New York
1 Bob 30 San Francisco
2 Charlie 22 Chicago
9. Read the following file formats
a. Pickle files
Serialization ([Link]): Converts the Python object into a byte
stream and writes it to the file.
Deserialization ([Link]): Reads the byte stream from the file and
converts it back into a Python object.
Binary Mode (wb, rb): Pickle files are written and read in binary mode.
1. Creating and Saving a Pickle File (Serialization)
import pickle
# Sample data (a dictionary in this case)
data = {'name': 'Alice', 'age': 30}
#Saving the data to a pickle file
with open('[Link]', 'wb') as file:
[Link](data, file)
print("Data serialized and saved to [Link]")

2. Loading a Pickle File

(Deserialization) import pickle
# Reading the data from the pickle file
with open('[Link]', 'rb') as file:
loaded_data = [Link](file)
print("Deserialized data:")
print(loaded_data)

Output:
Data serialized and saved to [Link]
Deserialized data:
{'name': 'Alice', 'age': 30 }
b. Image files using PIL
If you haven't installed it yet, you can install it using pip:
pip install Pillow
Reading an Image File using Pillow

from PIL import Image

# Open an image file
image = [Link]("[Link]")
# Display the image
[Link]()
# Print some basic info
print(f"Format: {[Link]}")
print(f"Size: {[Link]}")
print(f"Mode: {[Link]}")

c. Multiple files using Glob

Reading Multiple Text Files

import glob
# Get all text files in the "data" folder
text_files = [Link]("data/*.txt")
for file in text_files:
with open(file, 'r') as f:
content = [Link]()
print(f"File: {file}\nContent:\n{content}\n")

d. Importing data from database

SQLAlchemy is a powerful SQL toolkit and Object Relational Mapping
(ORM) library. Pandas can directly read SQL queries using SQLAlchemy.
Installation:
pip install sqlalchemy pandas
# Add database-specific driver, e.g.,
pip install pymysql # for MySQL

MySQL:
import pandas as pd
from sqlalchemy import create_engine
# Create a database connection
engine=create_engine("mysql+pymysql://username:password@localhos
t/mydatabase")
# Import data from a table into a DataFrame
df = pd.read_sql("SELECT * FROM my_table", engine)
print([Link]()) # Display the first few rows

10. Demonstrate web scraping using python

Step 1: Install Required Libraries
Make sure you have BeautifulSoup and requests installed:
pip install beautifulsoup4 requests
Step 2: Write a Python Script
Here’s a simple script to scrape the latest news headlines from a website
(like BBC News):

import requests
from bs4 import BeautifulSoup
# URL of the website to scrape
url = "[Link]
# Send a GET request to the website
response = [Link](url)
# Check if the request was successful
if response.status_code == 200:
# Parse the HTML content using BeautifulSoup
 soup = BeautifulSoup([Link], "[Link]")

# Find all headline tags (BBC typically uses <h3> tags for headlines)
headlines = soup.find_all("h3")
print("Latest News Headlines from BBC:")
for headline in headlines[:10]: # Display the first 10 headlines
print(f"- {headline.get_text(strip=True)}")
else:
print("Failed to retrieve the webpage.")

12. Perform following visualizations using matplotlib

Open your terminal or command prompt and run:
pip install matplotlib
a. Bar Graph
This following code will create a bar graph with four categories
(Apples, Bananas, Cherries, Dates) and their corresponding values.

import [Link] as plt

# Sample data
categories = ['Apples', 'Bananas', 'Cherries', 'Dates']
values = [30, 45, 20, 35]
# Create a bar graph
[Link](figsize=(8, 5))
[Link](categories, values, color='blue')
# Add title and labels
 [Link]('Fruit Sales')
[Link]('Fruits')
[Link]('Quantity Sold')
# Display the bar graph
[Link]()

Output:

b. Pie Chart
Represents hypothetical market share percentages of programming
languages.
labels show the language names.
sizes represent the market share of each language.
autopct='%1.1f%%' displays the percentage on each slice.
startangle=140 rotates the pie for a better view.

import [Link] as plt

# Sample data
labels = ['Python', 'JavaScript', 'Java', 'C#', 'Ruby']
sizes = [40, 30, 15, 10, 5]
colors = ['lightblue', 'lightgreen', 'coral', 'violet', 'gold']
# Create a pie chart
[Link](figsize=(6, 6))
[Link](sizes, labels=labels, colors=colors, autopct='%1.1f%%',
startangle=140)
# Add a title
[Link]('Programming Language Popularity')
# Display the pie chart
[Link]()

Output:
c. Box Plot
import [Link] as plt
# Sample data (e.g., exam scores of students)
data = [45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100]
# Create a simple box plot
[Link](data)
# Add title and labels
[Link]('Simple Box Plot')
[Link]('Scores')
# Display the box plot
[Link]()
Output:

d. Histogram
Uses [Link]() to plot the histogram.
bins=5 divides the data into 5 intervals.
color='blue' and edgecolor='black' improve visibility.

import [Link] as plt

# Sample data
data = [5, 10, 15, 10, 20, 25, 20, 15, 10, 5, 30, 35, 30, 25, 20]
# Create a histogram
[Link](data, bins=5, color='blue', edgecolor='black')
# Add title and labels
[Link]('Simple Histogram')
[Link]('Value')
[Link]('Frequency')
# Display the histogram
[Link]()

Output:

e. Line Chart and Subplots

Uses plot() to draw the lines and adds titles and labels.

import [Link] as plt

# Sample data
x = [1, 2, 3, 4, 5]
y = [10, 20, 15, 25, 30]
# Create a simple line chart
[Link](x, y, marker='o', color='blue', linestyle='-')
# Add title and labels
[Link]('Simple Line Chart')
[Link]('X-axis')
[Link]('Y-axis')
# Display the chart
[Link]()

Output:
f. Scatter Plot
Uses [Link]() to plot individual points.
color='blue' sets the point color.
marker='o' makes circular markers.

import [Link] as plt

# Sample data
x = [1, 2, 3, 4, 5]
y = [10, 15, 20, 25, 30]
# Create a scatter plot
[Link](x, y, color='blue', marker='o')
# Add title and labels
[Link]('Simple Scatter Plot')
[Link]('X-axis')
[Link]('Y-axis')
# Display the scatter plot
[Link]()

Output:
13. Getting started with NLTK, install NLTK using PIP
Run the following command in your terminal or command
prompt: pip install nltk

14. Python program to implement with Python Sci Kit-Learn & NLTK
simple Python program using NLTK and Scikit-Learn to classify text as
positive or negative using Naïve Bayes.

import nltk
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
# Sample dataset
texts = ["I love Python", "Python is great", "I hate bugs", "Debugging is
frustrating"]
labels = ["positive", "positive", "negative", "negative"]
# Convert text to numeric features using CountVectorizer
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(texts)
# Train a simple Naïve Bayes model
model = MultinomialNB()
[Link](X, labels)
# Predict on a new text
new_text = ["I love debugging"]
X_new = [Link](new_text)
prediction = [Link](X_new)[0]
print(f"Prediction for '{new_text[0]}':
{prediction}")

Output:
Prediction for 'I love debugging': positive