37 – Real-World Python Projects – IoT Data Logger

Purpose

A Python-based system that collects data from real sensors (temperature, humidity, motion, gas, etc.), processes it, and stores it in:

✔ Local CSV
✔ Local SQLite database
✔ Cloud platform (Firebase / MQTT / Thingspeak)

This is a must-know IoT project used everywhere—homes, factories, agriculture, vehicles, warehouses.


🧠 What You Will Build

A system that:

  1. Connects to sensors (via Arduino/ESP32/Raspberry Pi)
  2. Receives sensor readings via Serial/WiFi/MQTT
  3. Stores readings into CSV + SQLite
  4. Generates automatic daily report
  5. Visualizes data (temperature graph)

🧰 Tech Stack

  • Python
  • PySerial (for reading sensor data)
  • SQLite3
  • Pandas
  • Matplotlib
  • (Optional) MQTT, Firebase

📡 Example Hardware Setup

You can use any one:

Option A: Arduino + DHT11/22 Sensor

Arduino → USB → Python

Option B: ESP32/NodeMCU (WiFi)

ESP32 → MQTT → Python

Option C: Raspberry Pi (direct GPIO)

Python directly reads sensors


📁 Folder Structure

IoT_DataLogger/
│── datalogger.py
│── sensor_reader.py
│── database.db
│── logs/
│    └── data_2025-11-20.csv

🧩 FULL WORKING PYTHON CODE

📌 Part 1 — Read Sensor Data via Serial (sensor_reader.py)

import serial
import time

def read_sensor(port="COM3", baud=9600):
    ser = serial.Serial(port, baud, timeout=1)
    time.sleep(2)

    while True:
        line = ser.readline().decode().strip()
        if line:
            return line

Your Arduino should print something like:

25.7, 60.2    # temperature, humidity

📌 Part 2 — Data Logger (datalogger.py)

import pandas as pd
import sqlite3
from datetime import datetime
from sensor_reader import read_sensor

def init_db():
    conn = sqlite3.connect("database.db")
    cur = conn.cursor()
    cur.execute("""
        CREATE TABLE IF NOT EXISTS sensor_logs(
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            timestamp TEXT,
            temperature REAL,
            humidity REAL
        )
    """)
    conn.commit()
    conn.close()

def log_to_db(temp, hum):
    conn = sqlite3.connect("database.db")
    cur = conn.cursor()
    cur.execute("INSERT INTO sensor_logs(timestamp, temperature, humidity) VALUES (?, ?, ?)",
                (datetime.now(), temp, hum))
    conn.commit()
    conn.close()

def log_to_csv(temp, hum):
    date = datetime.now().strftime("%Y-%m-%d")
    filename = f"logs/data_{date}.csv"
    df = pd.DataFrame([[datetime.now(), temp, hum]],
                      columns=['Timestamp', 'Temperature', 'Humidity'])
    df.to_csv(filename, mode='a', header=not pd.io.common.file_exists(filename), index=False)

init_db()

print("Starting IoT Data Logger...")

while True:
    data = read_sensor()
    temp, hum = map(float, data.split(","))

    log_to_db(temp, hum)
    log_to_csv(temp, hum)

    print(f"Logged → Temp: {temp}°C, Humidity: {hum}%")

📊 Graph Generator (optional)

import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv("logs/data_2025-11-20.csv")
plt.plot(df['Timestamp'], df['Temperature'])
plt.xticks(rotation=45)
plt.title("Temperature Trend")
plt.xlabel("Time")
plt.ylabel("Temperature (°C)")
plt.tight_layout()
plt.show()

🚀 Optional Cloud Integrations

✔ MQTT Publisher (Python)

Send data to an IoT cloud dashboard:

pip install paho-mqtt

Code:

import paho.mqtt.client as mqtt

client = mqtt.Client()
client.connect("broker.hivemq.com", 1883)
client.publish("sameer/home/temperature", temp)


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *