Lesson 26: Multithreading and Multiprocessing in Python

🎯 Lesson Objective

To understand how to execute multiple tasks concurrently or in parallel in Python, using threads and processes.
You’ll learn the difference between multithreading and multiprocessing, when to use each, and how to build efficient, scalable programs.

---

🧩 1. What Are Threads and Processes?

Concept	Description	Example
Thread	A lightweight unit of a process that shares the same memory space.	Running background tasks like updating UI or downloading files.
Process	An independent program with its own memory and resources.	Running multiple Python scripts at once or CPU-heavy tasks.

In simple terms:

• Threading = Multiple tasks sharing the same memory (good for I/O tasks)
• Multiprocessing = Multiple Python instances (good for CPU-heavy tasks)

---

🧠 2. Why Use Concurrency?

When your program performs multiple tasks such as:

• Downloading files from the internet
• Reading and writing files
• Performing data analysis on large datasets
• Handling multiple client requests on a server

you don’t want one task to block others.
That’s where multithreading and multiprocessing help by running tasks concurrently.

---

⚙️ 3. The Global Interpreter Lock (GIL)

Python’s GIL (Global Interpreter Lock) allows only one thread to execute Python bytecode at a time within a single process.
This means:

• Multithreading in Python doesn’t achieve true parallelism for CPU-bound tasks.
• But it’s perfect for I/O-bound tasks, like waiting for network responses or reading from files.

For CPU-bound tasks (e.g., complex calculations), use multiprocessing.

---

🧵 4. Multithreading in Python

Importing the Module

import threading
import time

Example 1 — Basic Multithreading

def print_numbers():
    for i in range(5):
        print(f"Number: {i}")
        time.sleep(1)

def print_letters():
    for ch in "ABCDE":
        print(f"Letter: {ch}")
        time.sleep(1)

t1 = threading.Thread(target=print_numbers)
t2 = threading.Thread(target=print_letters)

t1.start()
t2.start()

t1.join()
t2.join()

print("Both threads completed.")

🧾 Output:

Number: 0
Letter: A
Number: 1
Letter: B
...
Both threads completed.

✅ Both tasks run concurrently, sharing the same CPU core.

---

Example 2 — Threads with Arguments

def greet(name):
    print(f"Hello, {name}!")
    time.sleep(2)
    print(f"Goodbye, {name}!")

thread1 = threading.Thread(target=greet, args=("Sameer",))
thread2 = threading.Thread(target=greet, args=("Ali",))

thread1.start()
thread2.start()
thread1.join()
thread2.join()

🧾 Output (interleaved):

Hello, Sameer!
Hello, Ali!
Goodbye, Sameer!
Goodbye, Ali!

---

Example 3 — Thread Synchronization (Lock)

When multiple threads access shared data, race conditions can occur. Use a Lock to prevent this.

lock = threading.Lock()
shared_counter = 0

def increment():
    global shared_counter
    for _ in range(100000):
        with lock:
            shared_counter += 1

threads = [threading.Thread(target=increment) for _ in range(5)]

for t in threads: t.start()
for t in threads: t.join()

print("Final counter:", shared_counter)

✅ Without the lock, the result would be unpredictable.
✅ With the lock, shared_counter becomes exactly 500000.

---

🔄 5. Multiprocessing in Python

Importing the Module

from multiprocessing import Process
import os, time

Example 1 — Basic Multiprocessing

def worker(task_id):
    print(f"Task {task_id} running on process {os.getpid()}")
    time.sleep(2)
    print(f"Task {task_id} completed")

if __name__ == "__main__":
    processes = []
    for i in range(4):
        p = Process(target=worker, args=(i,))
        processes.append(p)
        p.start()

    for p in processes:
        p.join()

    print("All processes finished.")

🧾 Output:

Task 0 running on process 23456
Task 1 running on process 23457
...
All processes finished.

✅ Each task runs in its own process, truly in parallel on multiple CPU cores.

---

Example 2 — Using a Process Pool

The multiprocessing.Pool class helps manage multiple processes efficiently.

from multiprocessing import Pool
import time

def square(n):
    time.sleep(1)
    return n * n

if __name__ == "__main__":
    numbers = [1, 2, 3, 4, 5]
    with Pool(processes=3) as pool:
        results = pool.map(square, numbers)
    print("Squares:", results)

🧾 Output:

Squares: [1, 4, 9, 16, 25]

✅ Tasks are distributed among 3 worker processes.

---

Example 3 — Sharing Data Between Processes

from multiprocessing import Value, Lock, Process

def add(lock, counter):
    for _ in range(1000):
        with lock:
            counter.value += 1

if __name__ == "__main__":
    lock = Lock()
    counter = Value('i', 0)
    processes = [Process(target=add, args=(lock, counter)) for _ in range(5)]

    for p in processes: p.start()
    for p in processes: p.join()

    print("Final Counter:", counter.value)

✅ Uses shared memory variable (Value) and a Lock to synchronize updates.

---

⚡ 6. Comparing Threading vs. Multiprocessing

Feature	Multithreading	Multiprocessing
Type of Task	I/O-bound	CPU-bound
Execution Model	Concurrent	Parallel
Memory	Shared memory space	Separate memory for each process
Overhead	Low	High
Performance	Limited by GIL	True multi-core utilization
Example	File downloads, web requests	Image processing, math computations

---

🧠 7. Example — File Download Automation (Multithreading)

import threading, requests, time

urls = [
    "https://example.com/file1.jpg",
    "https://example.com/file2.jpg",
    "https://example.com/file3.jpg"
]

def download_file(url):
    print(f"Downloading {url}")
    response = requests.get(url)
    filename = url.split("/")[-1]
    with open(filename, "wb") as f:
        f.write(response.content)
    print(f"Completed: {filename}")

start = time.time()
threads = [threading.Thread(target=download_file, args=(url,)) for url in urls]

for t in threads: t.start()
for t in threads: t.join()

print("All downloads finished in", round(time.time() - start, 2), "seconds")

✅ Multiple files download concurrently, saving significant time.

---

🧮 8. Example — Parallel Image Resizing (Multiprocessing)

from multiprocessing import Pool
from PIL import Image
import os

def resize_image(filename):
    img = Image.open(filename)
    img = img.resize((300, 300))
    img.save(f"resized_{filename}")
    return filename

if __name__ == "__main__":
    images = [f for f in os.listdir() if f.endswith(".jpg")]
    with Pool(processes=4) as pool:
        pool.map(resize_image, images)
    print("All images resized.")

✅ Each image is processed in parallel across 4 CPU cores.

---

🧪 9. Practical Use Cases

Use Case	Technique	Example
Web scraping multiple pages	Threading	Use requests + threading
Large data computation	Multiprocessing	Use Pool.map()
File backup & organization	Threading	Handle multiple file operations
Video rendering	Multiprocessing	Use all CPU cores
Server handling multiple clients	Threading	Each client connection in a thread

---

💡 10. Key Tips

• Use threading when your tasks are mostly waiting (I/O-bound).
• Use multiprocessing for computation-heavy tasks.
• Always use .join() to ensure threads/processes finish before the program exits.
• Avoid shared mutable data between threads unless you use locks or queues.
• Use concurrent.futures for a simpler API: from concurrent.futures import ThreadPoolExecutor

---