Python Basics

Introduction

Python is a widely-used, open-source programming language that has been around for over 20 years. It is cross-platform, meaning you can easily develop code on Linux, Windows, or Mac OS.

Why Learn Python?

In today’s world, where data plays a critical role in nearly every field, Python stands out as a top choice for Data Engineering, Data Science, Machine Learning, and Deep Learning. With the increasing demand for machine learning technologies, learning Python is a great way to stay relevant and competitive in the industry.

Key Areas Where Python is Commonly Used

Data Engineering: Managing and processing large datasets.
Machine Learning: Building intelligent models that learn from data.
Data Science: Analyzing and extracting insights from data.
Deep Learning: Developing complex AI systems and neural networks.

Advantages of Python

Python offers a variety of benefits that make it a preferred choice for developers:

Simplified and Efficient Development: Python’s concise syntax, dynamic typing, and interpreted nature speed up development, allowing you to write less code while achieving more.
Beginner-Friendly and Readable: Its clean structure and straightforward syntax make Python easy to learn and use, even for beginners, while enhancing collaboration in team projects.
Versatile and Extensible: Python supports object-oriented programming, integrates seamlessly with other languages, and is suitable for a wide range of applications, from web development to data analysis.
Robust Memory Management: Automatic memory allocation and garbage collection reduce the complexity of managing resources.
Open Source and Accessible: Being free to use, modify, and distribute ensures Python is accessible to individuals and organizations globally.

Python Data Types Overview

Python offers a variety of built-in data types to manage different kinds of information. Here's a breakdown of the main categories:

1. Numeric Data Types

These are used to represent numbers in Python:

int: Represents whole numbers.

age = 25

float: Represents decimal (floating-point) numbers.

pi = 3.14

complex: Represents complex numbers, which have both a real and an imaginary part (written as a + bj).

z = 3 + 5j  # Complex number
print(z.real)  # Output: 3.0
print(z.imag)  # Output: 5.0

2. String Data Type (`str`)

Used to represent a sequence of characters (text).

name = "Alice"

3. Sequence Types

These data types are used to store ordered collections of items.

list: An ordered and mutable collection of items, which can be of different types.

numbers = [1, 2, 3, 4]

tuple: Similar to lists but immutable.

coordinates = (1, 2, 3)

range: Represents an immutable sequence of numbers. Often used in loops.

numbers = range(1, 10, 2)  # Creates a range from 1 to 9 with a step of 2
for number in numbers:
    print(number)  # Output: 1, 3, 5, 7, 9

4. Binary Types

These types deal with binary data and byte manipulation:

bytes: Immutable sequence of bytes (8-bit values).
bytearray: Mutable sequence of bytes.
memoryview: Provides a view of data without copying it.

5. Mapping Data Type

dict: A collection of key-value pairs, where keys must be unique.

person = {"name": "Alice", "age": 25}

6. Boolean Type (`bool`)

Represents one of two possible values: True or False.

is_active = True

7. Set Data Types

These types store unordered collections of unique items.

set: An unordered collection with no duplicate elements.

fruits = {"apple", "banana", "cherry"}

frozenset: An immutable version of a set.

frozen_fruits = frozenset({"apple", "banana"})

8. Data Structures (Collections)

These are Python’s collection types for managing multiple items:

list
tuple
range
set
frozenset
dict

Complex Numbers in Python

Complex numbers have a real and an imaginary part, represented as a + bj, where a is the real part and b is the imaginary part. You can create complex numbers using the complex() function.

z = complex(3, 5)  # Represents 3 + 5j
print(z.real)  # Output: 3.0
print(z.imag)  # Output: 5.0

Range Data Type

The range() function generates a sequence of numbers, often used in loops. You can define the start, stop, and step parameters.

Syntax:

range(start, stop, step)

Example:

numbers = range(1, 10, 2)  # Generates 1, 3, 5, 7, 9
for num in numbers:
    print(num)  # Output: 1, 3, 5, 7, 9

Identifiers in Python

In Python, an identifier is the name assigned to various objects in a program, such as variables, lists, tuples, sets, dictionaries, functions, classes, and modules. Simply put, it serves as a label to refer to these entities later.

Rules for Identifiers in Python

Must begin with a letter (A-Z or a-z) or an underscore (_).
Can contain letters, digits (0-9), and underscores (_).
Cannot start with a digit.
Cannot use Python reserved keywords (e.g., if, while, class).
Case-sensitive (e.g., name and Name are different).
No special characters allowed (e.g., @, #, $, %).
Can be of any length.

Variables in Python

A variable is a name that refers to a memory location where data is stored and can be changed over time, hence the term "variable." When a value is assigned to a variable, Python allocates memory to store it. Variables act as references or labels for the data they hold, and their values can be modified during the program’s execution. In Python, the equal sign (=) is used to assign values to variables.

Examples

# Assigning a value to a variable
age = 25  # 'age' is a variable storing the value 25

name = "John"  # 'name' is a variable storing the string "John"

Multiple Assignment in Python

Python allows you to assign a single value to multiple variables at once, making your code more concise and readable. The syntax is simple:

# Multiple assignment example
a = b = c = 10  # All three variables (a, b, c) are assigned the value 10

Note: While identifiers and variables are often used interchangeably, they are not exactly the same. A variable is a specific type of identifier used to hold data or a value. It is a memory location where data can be stored and later retrieved.

Variable Type Change (Casting) in Python

In Python, variables are dynamically typed, meaning their data type is assigned automatically based on the value they are assigned. However, you can explicitly convert (cast) one data type to another using the following functions:

`int()`: Converts a Value into an Integer

Can convert an integer literal, a float literal (by removing the decimal part), or a string literal that represents a whole number.

Example:

# Converting a float to an integer
num = int(10.5)  # Result: 10

# Converting a string to an integer
num_str = int("20")  # Result: 20

`float()`: Converts a Value into a Float

Can convert an integer, a float literal, or a string literal that represents a float or integer.

Example:

# Converting an integer to a float
num = float(10)  # Result: 10.0

# Converting a string to a float
num_str = float("20.5")  # Result: 20.5

`str()`: Converts a Value into a String

Can convert any data type (integer, float, etc.) into a string.

Example:

# Converting an integer to a string
num = str(10)  # Result: "10"

# Converting a float to a string
num_float = str(20.5)  # Result: "20.5"

Working with Strings in Python

In Python, strings can be defined using either single quotes ('...') or double quotes ("..."), and both are equivalent. This gives flexibility in working with strings. If you need to include quotes within the string itself, you can escape them using the backslash (\).

Example:

single_quote_string = 'Hello, World!'
# This is a string using single quotes.

Example:

double_quote_string = "Python is fun!"
# This is a string using double quotes.

Example:

escaped_string = "She said, \"Python is amazing!\""
# This string uses an escape sequence to include quotes inside the string.

Working with Numbers in Python

Python can perform arithmetic operations like a simple calculator. It supports basic operators such as:

+ for addition
- for subtraction
* for multiplication
/ for division

Python also allows you to use parentheses (()) for grouping expressions.

Example:

result = (3 + 2) * 4
# Result will be 20.

Types of Division in Python

There are different types of division operators in Python:

Regular Division (/): Returns a float value.
Floor Division (//): Returns the integer quotient.
Modulus (%): Returns the remainder of the division.
Exponentiation (**): Returns the result of raising a number to the power of another.

Example of Regular Division:

result = 5 / 2
# Returns 2.5

Example of Floor Division:

result = 5 // 2
# Returns 2

Example of Modulus:

remainder = 5 % 2
# Returns 1

Example of Exponentiation:

power = 2 ** 3
# Returns 8 (2 raised to the power of 3)

String Concatenation and Integer Addition

The + operator is used for both string concatenation and integer addition in Python.

Example of String Concatenation:

greeting = "Hello, " + "World!"
# Results in "Hello, World!"

Example of Integer Addition:

sum_result = 5 + 3
# Results in 8

List

A list is one of the most versatile data structures in Python, designed to store a collection of items. Lists are ordered and changeable, allowing you to store and manipulate data in a sequence. You can create a list with any type of element, including integers, strings, or even other lists. However, it's common for lists to contain elements of the same type.

Key Characteristics of Lists

Ordered: The items in a list have a defined order, and the order will be preserved.
Changeable (Mutable): You can modify the list after it's created.
Allows Duplicate Members: Lists can contain multiple instances of the same element.

Creating and Using Lists

Lists are defined by placing items inside square brackets [] and separating them with commas.

# Creating a list
fruits = ['apple', 'banana', 'cherry']

Indexing and Slicing

Lists are indexed starting from 0. You can access individual elements using the index.

# Accessing an element by index
print(fruits[0])  # Output: 'apple'

Slicing allows you to access a range of elements within a list by specifying a start index and an end index:

# Slicing a list
print(fruits[1:3])  # Output: ['banana', 'cherry']

If you omit the start or end index, the slice will include elements from the beginning or up to the end of the list:

# Examples
print(fruits[:2])  # Output: ['apple', 'banana']
print(fruits[1:])  # Output: ['banana', 'cherry']

List Methods

Adding Elements

append(): Adds an item to the end of the list.
insert(): Adds an item at a specific index in the list.
extend(): Adds all items from another list to the end of the current list.

# Adding elements to a list
fruits.append('orange')
fruits.insert(1, 'grapes')
more_fruits = ['kiwi', 'mango']
fruits.extend(more_fruits)

Sorting Lists

sort(): Sorts the list in ascending order.
sort(reverse=True): Sorts the list in descending order.

# Sorting a list
fruits.sort()  # Ascending order
fruits.sort(reverse=True)  # Descending order

Finding List Length

# Get the number of elements in the list
print(len(fruits))  # Output: Number of elements in the list

Removing Elements

remove(): Removes the first occurrence of a specified value.
pop(): Removes and returns the last item from the list.
del: Removes an element at a specific index.
clear(): Removes all items from the list.

# Removing items from a list
fruits.remove('banana')  # Removes 'banana'
last_fruit = fruits.pop()  # Removes and returns the last item
del fruits[2]  # Removes the item at index 2
fruits.clear()  # Clears the entire list

Copying Lists

To copy a list, use the copy() method. Simply assigning a list to another variable will create a reference, so changes in the original list will affect the new one. Use copy() to avoid this:

# Copying a list
fruits_copy = fruits.copy()

Examples of List Operations

# Creating a list
fruits = ['apple', 'banana', 'cherry']

# Adding elements
fruits.append('orange')
fruits.insert(1, 'grapes')
more_fruits = ['kiwi', 'mango']
fruits.extend(more_fruits)

# Sorting and displaying the list
fruits.sort()
print(fruits)

# Getting length of the list
print(len(fruits))

# Removing items
fruits.remove('banana')
popped_fruit = fruits.pop()
del fruits[0]

# Clearing the list
fruits.clear()

# Copying the list
fruits_copy = fruits.copy()

Tuple

A tuple is a collection of items in Python that is ordered but immutable, meaning once it is created, its values cannot be changed. Tuples are often used when the integrity of the data needs to be preserved, and they are faster than lists for certain operations due to their immutability.

Key Characteristics of Tuples

Ordered: Tuples maintain the order of elements.
Immutable: You cannot modify a tuple after it's created (no adding, removing, or changing elements).
Allows Duplicate Members: Like lists, tuples can contain duplicate elements.

Creating a Tuple

Tuples are created by placing items inside parentheses () and separating them with commas:

# Creating a tuple
my_tuple = ('apple', 'banana', 'cherry')

You can also create a tuple without parentheses (parentheses are optional if you use commas to separate the items):

# Tuple without parentheses
my_tuple = 'apple', 'banana', 'cherry'

Indexing and Slicing

Tuples can be indexed and sliced similar to lists. The main difference is that tuples are immutable, meaning once a value is assigned at an index, it cannot be modified.

Indexing

Positive Indexing: The first element starts at index 0.
Negative Indexing: Tuples support negative indexing where -1 refers to the last item, -2 refers to the second-to-last item, and so on.

# Indexing examples
print(my_tuple[0])  # Output: 'apple'
print(my_tuple[-1])  # Output: 'cherry'

Slicing

You can specify a range of indexes by providing the start and end index. The return value will be a new tuple with the specified elements:

# Slicing examples
print(my_tuple[1:3])  # Output: ('banana', 'cherry')
print(my_tuple[:2])   # Output: ('apple', 'banana')
print(my_tuple[1:])   # Output: ('banana', 'cherry')

Changing Tuple Values

Since tuples are immutable, you cannot change the value of an element once it is assigned. However, you can work around this limitation by converting the tuple into a list, making changes, and then converting it back into a tuple:

# Modifying a tuple by conversion
my_tuple = ('apple', 'banana', 'cherry')
temp_list = list(my_tuple)
temp_list[1] = 'orange'  # Change 'banana' to 'orange'
my_tuple = tuple(temp_list)
print(my_tuple)  # Output: ('apple', 'orange', 'cherry')

Removing Items

Tuples are immutable, so you cannot remove items directly. However, you can delete the entire tuple using the del keyword:

# Deleting a tuple
del my_tuple  # Deletes the entire tuple

Joining Tuples

To combine two or more tuples, you can use the + operator:

# Joining tuples
tuple1 = ('apple', 'banana')
tuple2 = ('cherry', 'date')
combined_tuple = tuple1 + tuple2
print(combined_tuple)  # Output: ('apple', 'banana', 'cherry', 'date')

Nested Tuples

Tuples can also be nested (tuples inside tuples or lists inside tuples), allowing you to store more complex data structures:

# Nested tuple example
nested_tuple = (('apple', 'banana'), ('cherry', 'date'))
print(nested_tuple[0])  # Output: ('apple', 'banana')

Set

A set is an unordered collection of unique elements. Sets are commonly used for operations that involve membership testing, eliminating duplicate entries, and performing mathematical operations like union, intersection, difference, and symmetric difference.

Key Characteristics of Sets

Unordered: Items in a set do not have a specific order, so you cannot access them via indexes or keys.
No Duplicate Items: A set automatically eliminates duplicate entries.
Mutable: You can add or remove items from a set after it's created.
Supports Mathematical Operations: Sets support operations like union, intersection, difference, and symmetric difference.

Creating a Set

You can create a set using curly braces {} or the built-in set() function. To create an empty set, use set(), as {} creates an empty dictionary.

# Examples
my_set = {1, 2, 3}
another_set = set([4, 5, 6])
empty_set = set()

Accessing Items

Since sets are unordered, you cannot access elements by index. Instead, you can loop through the set or check for membership using the in keyword.

# Looping through a set
for item in my_set:
    print(item)

# Checking if an item is in the set
print(2 in my_set)  # Output: True
print(7 in my_set)  # Output: False

Adding Items

Use the add() method to add an item to a set. If the item already exists, the set remains unchanged.

# Adding an item
my_set.add(4)
print(my_set)  # Output: {1, 2, 3, 4}

Get the Length of a Set

To find the number of items in a set, use the len() function:

print(len(my_set))  # Output: 4

Removing Items from Sets

You can remove an item using the remove() or discard() method:

remove(): Removes the specified item. Raises a KeyError if the item is not found.
discard(): Removes the specified item. Does nothing if the item is not found.

# Removing items
my_set.remove(2)  # Removes 2 from the set
my_set.discard(3)  # Removes 3 from the set

Alternatively, use pop() to remove and return an arbitrary item:

# Popping an item
removed_item = my_set.pop()
print(removed_item)  # Output: An arbitrary item from the set

Clearing a Set

To empty a set, use the clear() method:

my_set.clear()  # Empties the set
print(my_set)  # Output: set()

Joining Two Sets

You can combine sets using union() or update():

union(): Returns a new set with all unique items from both sets.
update(): Adds items from one set to another, modifying the original set.

# Joining sets
set1 = {1, 2, 3}
set2 = {3, 4, 5}

union_set = set1.union(set2)
print(union_set)  # Output: {1, 2, 3, 4, 5}

set1.update(set2)
print(set1)  # Output: {1, 2, 3, 4, 5}

Mathematical Set Operations

Python sets support operations like intersection, difference, and symmetric difference:

Intersection: Common items in both sets.
Difference: Items in the first set but not in the second.
Symmetric Difference: Items in either set but not in both.

# Set operations
intersection_set = set1.intersection(set2)
print(intersection_set)  # Output: {3}

difference_set = set1.difference(set2)
print(difference_set)  # Output: {1, 2}

sym_diff_set = set1.symmetric_difference(set2)
print(sym_diff_set)  # Output: {1, 2, 4, 5}

Dictionary

A dictionary in Python is a collection of key-value pairs. It is an unordered, mutable (changeable), and indexed data structure. In dictionaries, keys must be unique and can be of any immutable type, such as strings, numbers, or tuples. The key-value pair allows you to store and retrieve values efficiently.

Key Characteristics of Dictionaries

Unordered: Dictionaries do not maintain the order of elements (although from Python 3.7 onwards, insertion order is preserved).
Mutable: You can change the content of a dictionary after it's created (add, remove, or modify items).
Indexed: You can access values through their associated keys.

Creating a Dictionary

You can create a dictionary using curly braces {} or with the built-in dict() function.

# Examples
my_dict = {"name": "Alice", "age": 25, "city": "New York"}
another_dict = dict(name="Bob", age=30, city="Los Angeles")

Accessing Items

To access values in a dictionary, you use the key inside square brackets []. Alternatively, you can use the get() method, which allows you to specify a default return value if the key does not exist.

# Examples
print(my_dict["name"])  # Output: Alice
print(my_dict.get("age"))  # Output: 25
print(my_dict.get("gender", "Not found"))  # Output: Not found

Changing Values

You can modify the value of an existing item by referencing its key.

# Example
my_dict["age"] = 26
print(my_dict)  # Output: {"name": "Alice", "age": 26, "city": "New York"}

Looping Through a Dictionary

You can loop through a dictionary to retrieve its keys, values, or both key-value pairs.

# Looping through keys
for key in my_dict:
    print(key)  # Output: name, age, city

# Looping through values
for value in my_dict.values():
    print(value)  # Output: Alice, 26, New York

# Looping through key-value pairs
for key, value in my_dict.items():
    print(key, value)  # Output: name Alice, age 26, city New York

Getting Key Names

Use the keys() method to retrieve all keys in a dictionary:

# Example
keys = my_dict.keys()
print(keys)  # Output: dict_keys(['name', 'age', 'city'])

Check if a Key Exists

You can check if a specific key exists in a dictionary using the in keyword.

# Examples
print("name" in my_dict)  # Output: True
print("gender" in my_dict)  # Output: False

Adding Items

To add a new item to a dictionary, assign a value to a new key.

# Example
my_dict["gender"] = "Female"
print(my_dict)  # Output: {"name": "Alice", "age": 26, "city": "New York", "gender": "Female"}

Removing Items

There are several methods to remove items from a dictionary:

pop(): Removes a specified key-value pair and returns the value.
popitem(): Removes the last inserted key-value pair (from Python 3.7 onwards).
del: Removes an item by its key.
clear(): Removes all items from the dictionary.

# Examples
age = my_dict.pop("age")  # Removes "age" key and returns its value
print(age)  # Output: 26
print(my_dict)  # Output: {"name": "Alice", "city": "New York", "gender": "Female"}

last_item = my_dict.popitem()  # Removes and returns the last inserted item
print(last_item)  # Output: ('gender', 'Female')

del my_dict["name"]  # Deletes the "name" key
print(my_dict)  # Output: {"city": "New York"}

my_dict.clear()  # Clears the dictionary
print(my_dict)  # Output: {}

Copying a Dictionary

To create a copy of a dictionary, use the copy() method. Direct assignment creates a reference, so changes to one dictionary will affect the other.

# Example
new_dict = my_dict.copy()
print(new_dict)

Conditional Execution

In Python, conditional statements allow you to execute certain blocks of code based on specific conditions. These are fundamental in making decisions in your program.

1. The if Statement

The simplest form of a conditional is the if statement. It checks a condition (a boolean expression), and if the condition is True, the indented block of code below it is executed.

# Example
x = 5
if x > 0:
   print("x is positive")

In this example, the condition x > 0 is checked. If it's true, the program prints "x is positive". If it's false, nothing happens.

2. Alternative Execution (if-else)

In some cases, you want to execute one block of code if the condition is true and another block if the condition is false. This is done using the else statement.

# Example
x = 5
if x > 0:
   print("x is positive")
else:
   print("x is non-positive")

In this example, if x > 0 is true, "x is positive" will be printed. Otherwise, "x is non-positive" will be printed.

3. Chained Conditionals (if-elif-else)

Sometimes, there are more than two possible outcomes. You can use elif (short for "else if") to check multiple conditions in a sequence.

# Example
x = 0
if x > 0:
   print("x is positive")
elif x < 0:
   print("x is negative")
else:
   print("x is zero")

In this example, the program checks if x is positive, negative, or zero and prints the corresponding message.

4. Nested if Statements

You can also have if statements inside other if statements, which are called nested if statements. This is useful when you need to check additional conditions within a block of code.

# Example
x = 10
y = 5
if x > 0:
   if y > 0:
      print("Both x and y are positive")
   else:
      print("x is positive, but y is non-positive")
else:
   print("x is non-positive")

In this example, the first if checks if x is positive, and if so, the program checks whether y is also positive.

5. Short-Hand if Statements

If you only have a single statement to execute, you can write the if statement in one line. This is useful for simple conditions.

# Example
x = 5
if x > 0: print("x is positive")

In this example, if x > 0 is true, the program prints "x is positive", all in one line.

6. Short-Hand if-else (Ternary Operator)

Python allows you to write an if-else statement in a single line using a ternary operator. This is useful for situations where you want to assign a value based on a condition.

# Example
x = 5
result = "Positive" if x > 0 else "Non-positive"
print(result)

In this example, the value of result will be "Positive" if x > 0, and "Non-positive" otherwise. This compact form is known as the ternary operator or conditional expression.

7. Using pass in Conditional Statements

Sometimes, you may need an if statement that doesn't do anything when the condition is met. In that case, you can use the pass statement, which is a placeholder for "do nothing".

# Example
x = -1
if x > 0:
   pass # Do nothing if x is positive
else:
   print("x is non-positive")

In this example, if x > 0 is true, nothing will happen because of the pass statement. If x is not greater than zero, it will print "x is non-positive".

Loops

In Python, there are two main types of loops used for repetitive execution: for loops and while loops. These loops help automate tasks that require repeating a block of code.

1. For Loop

A for loop is typically used when you want to iterate over a sequence (like a list, tuple, or string) or when you know beforehand how many times you want to execute a block of code.

Basic Structure of a For Loop

for var in range(num): 
    # code to execute

In this structure, range(num) generates a sequence of numbers from 0 to num-1, and var takes on each value in that sequence one by one.

Example: For Loop with range()

for i in range(3):
    print("Hello, World!")

This will print "Hello, World!" three times because the loop runs three times, with i taking values 0, 1, and 2.

For Loop with Lists

fruits = ["apple", "banana", "cherry"]
for fruit in fruits:
    print(fruit)

This will print:

apple
banana
cherry

The break Statement

The break statement can be used inside loops to stop the loop from running, even if the loop hasn't iterated through all items.

for i in range(5):
    if i == 3:
        break  # Exit the loop when i equals 3
    print(i)

Output:

The continue Statement

The continue statement can be used to stop the current iteration of the loop and continue with the next iteration.

for i in range(5):
    if i == 2:
        continue  # Skip the rest of the code when i equals 2
    print(i)

Output:

2. While Loop

A while loop is used to repeat a block of code as long as a specified condition is true. It’s useful when you don’t know beforehand how many times the loop should run, but you know the condition under which it should continue.

Basic Structure of a While Loop

while condition:
    # code to execute

The loop continues to execute as long as the condition is true.

Example: While Loop

x = 0
while x < 5:
    print(x)
    x += 1  # Increment x after each iteration

This will print:

The break Statement in a While Loop

The break statement can also be used in a while loop to exit the loop early if a certain condition is met.

x = 0
while x < 5:
    if x == 3:
        break  # Exit the loop when x equals 3
    print(x)
    x += 1

Output:

The continue Statement in a While Loop

The continue statement in a while loop works the same way as in a for loop: it skips the rest of the code inside the loop for the current iteration and moves to the next iteration.

x = 0
while x < 5:
    x += 1
    if x == 3:
        continue  # Skip printing when x equals 3
    print(x)

Output:

3. The else Statement in Loops

Both for and while loops can have an else block. The code inside the else block is executed when the loop terminates normally (i.e., not due to a break statement).

Example: else with a For Loop

for i in range(3):
    print(i)
else:
    print("Loop completed without break.")

Output:

0
1
2
Loop completed without break.

Example: else with a While Loop

x = 0
while x < 3:
    print(x)
    x += 1
else:
    print("While loop completed without break.")

Output:

0
1
2
While loop completed without break.

4. Creating a List using For Loop, If Condition, and Range()

You can combine for loops, if conditions, and range() to create lists based on certain conditions. Here's an example:

# Create a list of even numbers from 0 to 10
even_numbers = [i for i in range(11) if i % 2 == 0]
print(even_numbers)

Output:

[0, 2, 4, 6, 8, 10]

5. Finding Sum from 0 to 1000

To find the sum of numbers from 0 to 1000 using a loop:

total_sum = 0
for i in range(1001):
    total_sum += i
print("Sum from 0 to 1000:", total_sum)

Output:

Sum from 0 to 1000: 500500

Functions in Python

In Python, a function is a block of reusable code that performs a specific task. Functions help organize and modularize code, making it easier to maintain and debug. They allow us to group statements together and execute them when needed.

1. Defining a Function

To define a function, you use the def keyword, followed by the function name and parentheses. The statements inside the function's body must be indented.

Basic Structure of a Function

def function_name(parameters):
    # body of the function
    # statements to execute

def: Defines the function.
function_name: The name of the function.
parameters: Optional values you can pass to the function (also known as arguments).
Indented Block: The code inside the function is indented.

Example: Simple Function

def greet():
    print("Hello, World!")
    
greet()  # Calling the function

Output:

Hello, World!

2. Function Arguments (Parameters)

Functions can accept data (called arguments or parameters) that are passed when the function is called. There are different ways to pass arguments into a function:

Positional Arguments

Positional arguments are processed in the order they are passed to the function.

def greet(name, age):
    print(f"Hello, {name}! You are {age} years old.")

greet("Alice", 30)

Output:

Hello, Alice! You are 30 years old.

Keyword Arguments

With keyword arguments, you specify the value for a parameter by its name.

def greet(name, age):
    print(f"Hello, {name}! You are {age} years old.")

greet(name="Bob", age=25)

Output:

Hello, Bob! You are 25 years old.

3. Default Parameter Value

You can define default values for parameters, which will be used if no argument is passed for those parameters when calling the function.

def greet(name, age=18):
    print(f"Hello, {name}! You are {age} years old.")

greet("Charlie")  # Uses default value for age
greet("David", 25)  # Passes a value for age

Output:

Hello, Charlie! You are 18 years old.
Hello, David! You are 25 years old.

4. Variable Number of Arguments (*args)

Sometimes, you may want to pass an unknown number of arguments to a function. You can use *args to handle this scenario. *args allows the function to accept any number of positional arguments as a tuple.

def greet(*names):
    for name in names:
        print(f"Hello, {name}!")

greet("Alice", "Bob", "Charlie")

Output:

Hello, Alice!
Hello, Bob!
Hello, Charlie!

5. Mixing Positional, Keyword, and *args

You can mix positional arguments, keyword arguments, and *args in a function, but there are rules for the order of these:

Positional arguments first
Then, any keyword arguments
Finally, *args

def greet(message, *names, punctuation="!"):
    for name in names:
        print(f"{message}, {name}{punctuation}")

greet("Hello", "Alice", "Bob", punctuation="?")

Output:

Hello, Alice?
Hello, Bob?

6. Return Statement

A function can return a value as a result. The return keyword is used to send a value back from the function to the caller.

def add(a, b):
    return a + b

result = add(3, 4)
print(result)

Output:

7.lambda function in Python

A lambda function in Python is a small, anonymous function defined using the lambda keyword. It is typically used for short, simple operations where defining a full function would be unnecessary.

Syntax of a Lambda Function

lambda arguments: expression

lambda: The keyword that defines an anonymous function.
arguments: The inputs to the function (can be one or more).
expression: A single expression that the function computes and returns.

Lambda functions are concise, and their body is limited to a single expression.

Example: Lambda Function

add = lambda x, y: x + y
result = add(5, 3)
print(result)

Output:

Lambda with Multiple Arguments

multiply = lambda a, b, c: a * b * c
print(multiply(2, 3, 4))

Output:

Local vs Global Variables

In Python, the scope of variables can be classified as either local or global:

Local Variables: Variables defined inside a function. They are only accessible within that function.
Global Variables: Variables defined outside any function. They are accessible throughout the program.

Accessing Global Variables Inside a Function

If you want to modify a global variable from within a function, you can use the global keyword.

Example: Without global Keyword

x = 10

def modify():
    x = 20  # Local variable
    print(x)  # Prints local variable

modify()
print(x)  # Prints global variable

Output:

20
10

Example: With global Keyword

x = 10

def modify():
    global x  # Referencing global variable
    x = 20  # Modifies global variable
    print(x)

modify()
print(x)  # Prints modified global variable

Output:

20
20

Exception and Error Handling

In Python, there are two primary types of errors:

Syntax Errors: Errors that occur when the code does not follow the correct syntax.
Exceptions: Errors that occur during execution, even if the syntax is correct.

Exceptions

Exceptions are runtime errors that happen when Python encounters an issue during execution. They do not always cause the program to terminate immediately.

Try...Except Block

To handle exceptions, Python provides the try...except block. This allows you to handle errors gracefully without crashing the program.

try:
    x = 10 / 0
except ZeroDivisionError:
    print("You can't divide by zero!")

Output:

You can't divide by zero!

Multiple Exception Handling

You can handle multiple types of exceptions in separate except blocks:

try:
    a = 10 / 0
except ZeroDivisionError:
    print("You can't divide by zero!")
except Exception as e:
    print(f"An error occurred: {e}")

Output:

You can't divide by zero!

Raising Exceptions

You can use the raise statement to trigger an exception manually:

def check_age(age):
    if age < 18:
        raise ValueError("Age must be 18 or older")
    else:
        print("Age is valid")

check_age(16)

Output:

ValueError: Age must be 18 or older

Finally Clause

The finally clause can be added to the try...except block to ensure that certain code is always executed, regardless of whether an exception occurred or not. This is typically used for cleanup actions like closing files or releasing resources.

try:
    x = 10 / 2
finally:
    print("This block runs no matter what")

Output:

This block runs no matter what

Finally with Return Statement

If a return statement is inside the try block, the finally block will execute before returning the value:

def test_func():
    try:
        return "Returned from try"
    finally:
        print("This is from finally")

print(test_func())

Output:

This is from finally
Returned from try

Else Clause in Try-Except

The else clause in a try...except block runs only if no exception is raised in the try block. It is useful for code that should run when the try block is successful and does not raise an exception.

try:
    x = 10 / 2
except ZeroDivisionError:
    print("Can't divide by zero")
else:
    print("Division successful")

Output:

Division successful

Reading and Writing Files

Python provides built-in functions for file handling, primarily using the open() function. This function allows reading from and writing to files. Here's an overview of how to handle files in Python.

Opening a File: open() Function

The open() function is used to open a file and returns a file object. It requires at least one argument, the filename, and optionally, a mode for how the file will be handled.

Syntax: open(filename, mode)

filename: The name of the file you want to open (can include the file path).
mode: The mode in which to open the file. If not provided, it defaults to "r" (read).

File Modes

"r" - Read (default): Opens the file for reading. If the file does not exist, it raises an error.
"a" - Append: Opens the file for appending. If the file doesn't exist, it creates a new one.
"w" - Write: Opens the file for writing. If the file exists, it is overwritten.
"x" - Create: Creates a new file. If the file already exists, it raises an error.

Reading Files

Once a file is opened, you can use various methods to read its contents:

read(): Reads the entire content of the file.
readline(): Reads one line at a time, including the newline character \n.
readlines(): Reads all the lines of the file and returns them as a list of strings.

Example of Reading a File

with open("example.txt", "r") as file:
    content = file.read()
    print(content)

Example of Reading Line by Line

with open("example.txt", "r") as file:
    for line in file:
        print(line.strip())  # Use strip() to remove newline characters

Writing to Files

To write to a file, you can use the write() or writelines() methods. Writing will overwrite the file content if opened in write mode ("w").

Example of Writing to a File

with open("example.txt", "w") as file:
    file.write("Hello, World!\nThis is a test file.")

Example of Appending to a File

with open("example.txt", "a") as file:
    file.write("\nThis line is appended.")

File Handling and readline()

When using readline(), it reads one line at a time from the file. If the file ends, it returns an empty string.

with open("example.txt", "r") as file:
    line = file.readline()
    while line:
        print(line.strip())  # Strip newline character
        line = file.readline()

Regular Expressions in Python

Regular expressions (regex) are patterns used to match character combinations in strings. Python's re module provides functions to perform regex operations, such as searching, matching, splitting, and replacing parts of strings.

Basic Functions in re Module

findall(): Returns a list of all matches found in the string.
search(): Searches for the first match in the string and returns a match object if found.
split(): Splits the string at each match of the pattern.
sub(): Replaces matches with a specified string.

Example of findall()

import re
result = re.findall(r'\d+', "There are 2 apples and 12 oranges.")
print(result)  # Output: ['2', '12']

Example of search()

import re
match = re.search(r'\d+', "There are 2 apples.")
if match:
    print(match.group())  # Output: '2'

Example of split()

import re
result = re.split(r'\s+', "Hello  World  Python!")
print(result)  # Output: ['Hello', 'World', 'Python!']

Example of sub()

import re
result = re.sub(r'\d+', 'X', "There are 2 apples and 12 oranges.")
print(result)  # Output: 'There are X apples and X oranges.'

Example: Using Regular Expressions to Extract Data

import re

text = "John's phone number is 123-456-7890 and Sarah's is 987-654-3210."

# Find all phone numbers in the text
phone_numbers = re.findall(r'\d{3}-\d{3}-\d{4}', text)
print(phone_numbers)  # Output: ['123-456-7890', '987-654-3210']

Here, \d{3}-\d{3}-\d{4} is a regular expression that matches phone numbers in the format XXX-XXX-XXXX.

Python Keywords

In Python, keywords are reserved words that have special meaning in the language. These words cannot be used as regular identifiers (e.g., variable names, function names, etc.) because they are reserved for specific functionality. Python keywords are case-sensitive, meaning they must be written exactly as they are.

Example Keywords

Some common Python keywords include:

False, await, else, import, pass, None, break, except, in, raise, True, class, finally, is, return, and, continue, for, lambda, try, as, def, from, nonlocal, while

Role of Keywords in Python

These keywords are predefined and have specific roles in the Python programming language. For example:

if, else, elif are used for conditional statements.
def is used to define a function.
class is used to define a class.
return is used to return a value from a function.

Because these keywords are reserved, you cannot use them as names for variables, functions, or any other objects.

String Format

String formatting allows you to embed variables and expressions inside strings in a readable and maintainable way. Python provides several methods to achieve string formatting, including f-strings, the str.format() method, and the old-style % operator.

F-strings in Python

F-strings (formatted string literals) provide a concise way to embed expressions inside string literals, evaluated at runtime. F-strings are prefixed with the letter f, and expressions inside the string are enclosed in curly braces {}.

Example:

name = "Alice"
age = 25
message = f"My name is {name} and I am {age} years old."
print(message)  # Output: "My name is Alice and I am 25 years old."

format() Function in Python

The str.format() method allows for advanced string formatting by substituting placeholders in a string with values. You can use positional {0}, named {name}, or empty {} placeholders.

Syntax:

"string {} format".format(value)

Example:

name = "Alice"
age = 25
message = "My name is {} and I am {} years old.".format(name, age)
print(message)  # Output: "My name is Alice and I am 25 years old."

Example with Positional Indexes:

message = "My name is {0} and I am {1} years old. {0} loves programming.".format(name, age)
print(message)  # Output: "My name is Alice and I am 25 years old. Alice loves programming."

String Formatting Using % Operator

Python also supports C-style string formatting, where the % operator is used to format variables within a string. You can use special format specifiers to control how data is displayed.

%s: For strings
%d: For decimal integers (base-10)
%f: For floating point numbers
%c: For characters
%b: For binary numbers
%o: For octal numbers
%x: For hexadecimal (lowercase)
%X: For hexadecimal (uppercase)
%e: For exponent notation

Example:

name = "Alice"
age = 25
height = 5.6
formatted_string = "Name: %s, Age: %d, Height: %.1f" % (name, age, height)
print(formatted_string)  # Output: "Name: Alice, Age: 25, Height: 5.6"

Python for Data Engineer

Python Basics

Introduction

Why Learn Python?

Key Areas Where Python is Commonly Used

Advantages of Python

Python Data Types Overview

1. Numeric Data Types

2. String Data Type (str)

3. Sequence Types

4. Binary Types

5. Mapping Data Type

6. Boolean Type (bool)

7. Set Data Types

8. Data Structures (Collections)

Complex Numbers in Python

Range Data Type

Identifiers in Python

Rules for Identifiers in Python

Variables in Python

Multiple Assignment in Python

Variable Type Change (Casting) in Python

int(): Converts a Value into an Integer

float(): Converts a Value into a Float

str(): Converts a Value into a String

Working with Strings in Python

Working with Numbers in Python

Types of Division in Python

String Concatenation and Integer Addition

List

Key Characteristics of Lists

Creating and Using Lists

Indexing and Slicing

List Methods

Adding Elements

Sorting Lists

Finding List Length

Removing Elements

Copying Lists

Tuple

Key Characteristics of Tuples

Creating a Tuple

Indexing and Slicing

Indexing

Slicing

Changing Tuple Values

Removing Items

Joining Tuples

Nested Tuples

Set

Key Characteristics of Sets

Creating a Set

Accessing Items

Adding Items

Get the Length of a Set

Removing Items from Sets

Clearing a Set

Joining Two Sets

Mathematical Set Operations

Dictionary

Key Characteristics of Dictionaries

Creating a Dictionary

Accessing Items

Changing Values

Looping Through a Dictionary

Getting Key Names

Check if a Key Exists

Adding Items

Removing Items

Copying a Dictionary

Conditional Execution

1. The if Statement

2. Alternative Execution (if-else)

3. Chained Conditionals (if-elif-else)

4. Nested if Statements

5. Short-Hand if Statements

6. Short-Hand if-else (Ternary Operator)

7. Using pass in Conditional Statements

Loops

1. For Loop

2. String Data Type (`str`)

6. Boolean Type (`bool`)

`int()`: Converts a Value into an Integer

`float()`: Converts a Value into a Float

`str()`: Converts a Value into a String