battery-tracker-app/MIGRATION.md

# Schema Migrations

## Adding battery metadata fields, device type, and history tables

These columns and tables were added over several feature commits. `create_all()` does not add columns to existing tables, so existing installations need the `ALTER TABLE` commands below. New tables (`capacity_test`, `charge_log`) are created automatically by `create_all()` on next restart for SQLite — the explicit SQL below is provided for reference and for MariaDB operators who manage schema manually.

**Always snapshot first:**
```bash
cp batteries.db batteries.db.$(date +%Y-%m-%d).snapshot
```

**SQLite:**
```bash
sqlite3 batteries.db "ALTER TABLE device ADD COLUMN device_type VARCHAR(50);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN size VARCHAR(20);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN chemistry VARCHAR(20);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN capacity_mah INTEGER;"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN tested_capacity_mah INTEGER;"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN tested_date VARCHAR(10);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN charge_cycles INTEGER;"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN purchase_date VARCHAR(10);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN storage_location VARCHAR(100);"
```

New tables (auto-created on restart, shown here for reference):
```python
import sqlite3
conn = sqlite3.connect('batteries.db')
conn.execute('''CREATE TABLE IF NOT EXISTS capacity_test (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  battery_id INTEGER NOT NULL REFERENCES battery(id) ON DELETE CASCADE,
  tested_capacity_mah INTEGER NOT NULL,
  tested_date VARCHAR(10) NOT NULL,
  notes TEXT
)''')
conn.execute('''CREATE TABLE IF NOT EXISTS charge_log (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  battery_id INTEGER NOT NULL REFERENCES battery(id) ON DELETE CASCADE,
  charged_date VARCHAR(10) NOT NULL,
  increment_cycles INTEGER NOT NULL DEFAULT 0,
  notes TEXT
)''')
conn.commit(); conn.close()
```

**MariaDB / MySQL:**
```sql
ALTER TABLE device ADD COLUMN device_type VARCHAR(50) NULL;
ALTER TABLE battery ADD COLUMN size VARCHAR(20) NULL;
ALTER TABLE battery ADD COLUMN chemistry VARCHAR(20) NULL;
ALTER TABLE battery ADD COLUMN capacity_mah INT NULL;
ALTER TABLE battery ADD COLUMN tested_capacity_mah INT NULL;
ALTER TABLE battery ADD COLUMN tested_date VARCHAR(10) NULL;
ALTER TABLE battery ADD COLUMN charge_cycles INT NULL;
ALTER TABLE battery ADD COLUMN purchase_date VARCHAR(10) NULL;
ALTER TABLE battery ADD COLUMN storage_location VARCHAR(100) NULL;

CREATE TABLE IF NOT EXISTS capacity_test (
  id INT AUTO_INCREMENT PRIMARY KEY,
  battery_id INT NOT NULL,
  tested_capacity_mah INT NOT NULL,
  tested_date VARCHAR(10) NOT NULL,
  notes TEXT,
  CONSTRAINT fk_ct_battery FOREIGN KEY (battery_id) REFERENCES battery(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

CREATE TABLE IF NOT EXISTS charge_log (
  id INT AUTO_INCREMENT PRIMARY KEY,
  battery_id INT NOT NULL,
  charged_date VARCHAR(10) NOT NULL,
  increment_cycles INT NOT NULL DEFAULT 0,
  notes TEXT,
  CONSTRAINT fk_cl_battery FOREIGN KEY (battery_id) REFERENCES battery(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```

---

## Adding battery_pct_log table

This table was added to track battery percentage change history.

**Always snapshot first:**
```bash
cp batteries.db batteries.db.$(date +%Y-%m-%d).snapshot
```

**SQLite (Python):**
```python
import sqlite3
conn = sqlite3.connect('batteries.db')
conn.execute('''CREATE TABLE IF NOT EXISTS battery_pct_log (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  battery_id INTEGER NOT NULL REFERENCES battery(id) ON DELETE CASCADE,
  percentage INTEGER NOT NULL,
  recorded_at VARCHAR(19) NOT NULL,
  source VARCHAR(10)
)''')
conn.commit(); conn.close()
```

**MariaDB / MySQL:**
```sql
CREATE TABLE IF NOT EXISTS battery_pct_log (
  id INT AUTO_INCREMENT PRIMARY KEY,
  battery_id INT NOT NULL,
  percentage INT NOT NULL,
  recorded_at VARCHAR(19) NOT NULL,
  source VARCHAR(10),
  CONSTRAINT fk_bpl_battery FOREIGN KEY (battery_id) REFERENCES battery(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
```

---

## Adding Home Assistant fields (ha_entity_id, battery_percentage)

These columns were added in the Home Assistant integration feature. Existing databases
need a manual migration — `create_all()` does not add columns to existing tables.

**Always snapshot first:**
```bash
cp batteries.db batteries.db.$(date +%Y-%m-%d).snapshot
```

**SQLite:**
```bash
sqlite3 batteries.db "ALTER TABLE device ADD COLUMN ha_entity_id VARCHAR(100);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN battery_percentage INTEGER;"
```

**MariaDB / MySQL:**
```sql
ALTER TABLE device ADD COLUMN ha_entity_id VARCHAR(100) NULL;
ALTER TABLE battery ADD COLUMN battery_percentage INT NULL;
```

Both columns are nullable with no default, which is valid on all supported databases.

---

# Migrating from SQLite to MariaDB

This guide covers moving the Battery Tracker database from SQLite to MariaDB/MySQL
without changing any application code — only the connection string changes.

---

## 1. Install PyMySQL

```bash
source .venv/bin/activate
pip install pymysql
pip freeze > requirements.txt
```

---

## 2. Create the MariaDB database and user

```sql
CREATE DATABASE batteries CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'battuser'@'localhost' IDENTIFIED BY 'strongpassword';
GRANT ALL PRIVILEGES ON batteries.* TO 'battuser'@'localhost';
FLUSH PRIVILEGES;
```

---

## 3. Update the connection string

Set the `DATABASE_URL` environment variable before starting the app:

```bash
export DATABASE_URL='mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4'
```

Or add it to a `.env` file and load it with your process manager / systemd service:

```ini
# In ~/.config/systemd/user/battery-tracker.service, add under [Service]:
Environment=DATABASE_URL=mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4
```

---

## 4. (Optional) Use Flask-Migrate for schema management

Flask-Migrate lets you apply schema changes incrementally via `flask db upgrade`
rather than dropping and recreating tables.

```bash
pip install Flask-Migrate
```

In `app.py`, after the `db` setup, add:

```python
from flask_migrate import Migrate
migrate = Migrate(app, db)   # pass your SQLAlchemy db object
```

Then:

```bash
flask db init       # first time only — creates migrations/ directory
flask db migrate -m "initial schema"
flask db upgrade
```

> **Note:** The app uses raw SQLAlchemy (not Flask-SQLAlchemy), so you will need
> to adapt the `Migrate(app, db)` call to wrap your engine/session. Alternatively,
> run `Base.metadata.create_all(engine)` directly for the initial schema — the app
> already does this on startup.

---

## 5. Run the migration script

`sbin/setup_mariadb.py` prompts interactively for credentials (or reads them
from environment variables), snapshots the SQLite database, creates all tables
on MariaDB, and migrates every record using the SQLAlchemy ORM.

```bash
# Interactive — prompts for host, port, user, password, database
python sbin/setup_mariadb.py

# Non-interactive — supply individual env vars
MARIADB_HOST=localhost MARIADB_PORT=3306 \
  MARIADB_USER=battuser MARIADB_PASSWORD=strongpassword \
  MARIADB_DATABASE=batteries \
  python sbin/setup_mariadb.py

# Non-interactive — supply a full URL
MARIADB_URL='mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4' \
  python sbin/setup_mariadb.py
```

The script:
1. Snapshots `batteries.db` → `batteries.db.YYYY-MM-DD.snapshot`
2. Creates all tables on MariaDB if they don't exist
3. Migrates Device, Battery, CapacityTest, ChargeLog, and BatteryPctLog records (preserving primary keys and foreign keys)
4. Resets `AUTO_INCREMENT` counters past the highest migrated ID
5. Prints a verification table comparing row counts
6. Prints the exact `DATABASE_URL` and systemd service line to configure

---

## 6. Verify row counts

The migration script prints a summary:

```
=== Verification ===
Table                SQLite   MariaDB    OK?
-----------------------------------------------
device                    5         5     OK
battery                  40        40     OK
capacity_test            12        12     OK
charge_log               87        87     OK
battery_pct_log         320       320     OK

Migration complete. All row counts match.
```

If you see `MISMATCH`, **do not** decommission SQLite. Re-run the script after
investigating the error output.

You can also verify manually in the MariaDB shell:

```sql
SELECT COUNT(*) FROM device;
SELECT COUNT(*) FROM battery;
SELECT COUNT(*) FROM capacity_test;
SELECT COUNT(*) FROM charge_log;
SELECT COUNT(*) FROM battery_pct_log;
```

---

## 7. MariaDB configuration notes

| Setting | Recommended value |
|---|---|
| Character set | `utf8mb4` |
| Collation | `utf8mb4_unicode_ci` |
| `innodb_strict_mode` | `ON` (default in MariaDB 10.2+) |
| `sql_mode` | Include `STRICT_TRANS_TABLES` |

Set these in `/etc/mysql/mariadb.conf.d/50-server.cnf` (or equivalent):

```ini
[mysqld]
character-set-server  = utf8mb4
collation-server      = utf8mb4_unicode_ci
```

The `?charset=utf8mb4` query parameter in the connection string ensures the
client connection also uses utf8mb4, regardless of the server default.

---

## 8. Decommission SQLite

Once you have confirmed the row counts match and the app is running correctly
against MariaDB:

```bash
# Optional: keep a backup
cp batteries.db batteries.db.bak

# Remove the SQLite file
rm batteries.db
```

The `DATABASE_URL` environment variable now fully controls which database is used —
no code changes were required.