battery-tracker-app/MIGRATION.md

Schema Migrations

Adding battery metadata fields, device type, and history tables

These columns and tables were added over several feature commits. create_all() does not add columns to existing tables, so existing installations need the ALTER TABLE commands below. New tables (capacity_test, charge_log) are created automatically by create_all() on next restart for SQLite — the explicit SQL below is provided for reference and for MariaDB operators who manage schema manually.

Always snapshot first:

cp batteries.db batteries.db.$(date +%Y-%m-%d).snapshot

SQLite:

sqlite3 batteries.db "ALTER TABLE device ADD COLUMN device_type VARCHAR(50);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN size VARCHAR(20);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN chemistry VARCHAR(20);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN capacity_mah INTEGER;"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN tested_capacity_mah INTEGER;"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN tested_date VARCHAR(10);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN charge_cycles INTEGER;"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN purchase_date VARCHAR(10);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN storage_location VARCHAR(100);"

New tables (auto-created on restart, shown here for reference):

import sqlite3
conn = sqlite3.connect('batteries.db')
conn.execute('''CREATE TABLE IF NOT EXISTS capacity_test (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  battery_id INTEGER NOT NULL REFERENCES battery(id) ON DELETE CASCADE,
  tested_capacity_mah INTEGER NOT NULL,
  tested_date VARCHAR(10) NOT NULL,
  notes TEXT
)''')
conn.execute('''CREATE TABLE IF NOT EXISTS charge_log (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  battery_id INTEGER NOT NULL REFERENCES battery(id) ON DELETE CASCADE,
  charged_date VARCHAR(10) NOT NULL,
  increment_cycles INTEGER NOT NULL DEFAULT 0,
  notes TEXT
)''')
conn.commit(); conn.close()

MariaDB / MySQL:

ALTER TABLE device ADD COLUMN device_type VARCHAR(50) NULL;
ALTER TABLE battery ADD COLUMN size VARCHAR(20) NULL;
ALTER TABLE battery ADD COLUMN chemistry VARCHAR(20) NULL;
ALTER TABLE battery ADD COLUMN capacity_mah INT NULL;
ALTER TABLE battery ADD COLUMN tested_capacity_mah INT NULL;
ALTER TABLE battery ADD COLUMN tested_date VARCHAR(10) NULL;
ALTER TABLE battery ADD COLUMN charge_cycles INT NULL;
ALTER TABLE battery ADD COLUMN purchase_date VARCHAR(10) NULL;
ALTER TABLE battery ADD COLUMN storage_location VARCHAR(100) NULL;

CREATE TABLE IF NOT EXISTS capacity_test (
  id INT AUTO_INCREMENT PRIMARY KEY,
  battery_id INT NOT NULL,
  tested_capacity_mah INT NOT NULL,
  tested_date VARCHAR(10) NOT NULL,
  notes TEXT,
  CONSTRAINT fk_ct_battery FOREIGN KEY (battery_id) REFERENCES battery(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

CREATE TABLE IF NOT EXISTS charge_log (
  id INT AUTO_INCREMENT PRIMARY KEY,
  battery_id INT NOT NULL,
  charged_date VARCHAR(10) NOT NULL,
  increment_cycles INT NOT NULL DEFAULT 0,
  notes TEXT,
  CONSTRAINT fk_cl_battery FOREIGN KEY (battery_id) REFERENCES battery(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

---

Adding battery_pct_log table

This table was added to track battery percentage change history.

Always snapshot first:

cp batteries.db batteries.db.$(date +%Y-%m-%d).snapshot

SQLite (Python):

import sqlite3
conn = sqlite3.connect('batteries.db')
conn.execute('''CREATE TABLE IF NOT EXISTS battery_pct_log (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  battery_id INTEGER NOT NULL REFERENCES battery(id) ON DELETE CASCADE,
  percentage INTEGER NOT NULL,
  recorded_at VARCHAR(19) NOT NULL,
  source VARCHAR(10)
)''')
conn.commit(); conn.close()

MariaDB / MySQL:

CREATE TABLE IF NOT EXISTS battery_pct_log (
  id INT AUTO_INCREMENT PRIMARY KEY,
  battery_id INT NOT NULL,
  percentage INT NOT NULL,
  recorded_at VARCHAR(19) NOT NULL,
  source VARCHAR(10),
  CONSTRAINT fk_bpl_battery FOREIGN KEY (battery_id) REFERENCES battery(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

---

Adding Home Assistant fields (ha_entity_id, battery_percentage)

These columns were added in the Home Assistant integration feature. Existing databases need a manual migration — create_all() does not add columns to existing tables.

Always snapshot first:

cp batteries.db batteries.db.$(date +%Y-%m-%d).snapshot

SQLite:

sqlite3 batteries.db "ALTER TABLE device ADD COLUMN ha_entity_id VARCHAR(100);"
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN battery_percentage INTEGER;"

MariaDB / MySQL:

ALTER TABLE device ADD COLUMN ha_entity_id VARCHAR(100) NULL;
ALTER TABLE battery ADD COLUMN battery_percentage INT NULL;

Both columns are nullable with no default, which is valid on all supported databases.

---

Migrating from SQLite to MariaDB

This guide covers moving the Battery Tracker database from SQLite to MariaDB/MySQL without changing any application code — only the connection string changes.

---

1. Install PyMySQL

source .venv/bin/activate
pip install pymysql
pip freeze > requirements.txt

---

2. Create the MariaDB database and user

CREATE DATABASE batteries CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'battuser'@'localhost' IDENTIFIED BY 'strongpassword';
GRANT ALL PRIVILEGES ON batteries.* TO 'battuser'@'localhost';
FLUSH PRIVILEGES;

---

3. Update the connection string

Set the DATABASE_URL environment variable before starting the app:

export DATABASE_URL='mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4'

Or add it to a .env file and load it with your process manager / systemd service:

# In ~/.config/systemd/user/battery-tracker.service, add under [Service]:
Environment=DATABASE_URL=mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4

---

4. (Optional) Use Flask-Migrate for schema management

Flask-Migrate lets you apply schema changes incrementally via flask db upgrade rather than dropping and recreating tables.

pip install Flask-Migrate

In app.py, after the db setup, add:

from flask_migrate import Migrate
migrate = Migrate(app, db)   # pass your SQLAlchemy db object

Then:

flask db init       # first time only — creates migrations/ directory
flask db migrate -m "initial schema"
flask db upgrade

Note: The app uses raw SQLAlchemy (not Flask-SQLAlchemy), so you will need to adapt the Migrate(app, db) call to wrap your engine/session. Alternatively, run Base.metadata.create_all(engine) directly for the initial schema — the app already does this on startup.

---

5. Run the migration script

The migrate_to_mariadb.py script reads every record from SQLite and inserts it into MariaDB using the SQLAlchemy ORM — no raw SQL, no CSV exports.

MARIADB_URL='mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4' \
    python migrate_to_mariadb.py

Or pass the URL as a positional argument:

python migrate_to_mariadb.py 'mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4'

The script:

1. Creates all tables on MariaDB if they don't exist
2. Inserts all Device records (preserving primary keys)
3. Inserts all Battery records (preserving primary keys and foreign keys)
4. Resets AUTO_INCREMENT counters past the highest migrated ID
5. Prints a verification table comparing row counts

---

6. Verify row counts

The migration script prints a summary:

=== Verification ===
Table        SQLite   MariaDB    OK?
--------------------------------------
device            5         5     OK
battery          40        40     OK

Migration complete. All row counts match.

If you see MISMATCH, do not decommission SQLite. Re-run the script after investigating the error output.

You can also verify manually in the MariaDB shell:

SELECT COUNT(*) FROM device;
SELECT COUNT(*) FROM battery;

---

7. MariaDB configuration notes

Setting	Recommended value
Character set	utf8mb4
Collation	utf8mb4_unicode_ci
innodb_strict_mode	ON (default in MariaDB 10.2+)
sql_mode	Include STRICT_TRANS_TABLES

Set these in /etc/mysql/mariadb.conf.d/50-server.cnf (or equivalent):

[mysqld]
character-set-server  = utf8mb4
collation-server      = utf8mb4_unicode_ci

The ?charset=utf8mb4 query parameter in the connection string ensures the client connection also uses utf8mb4, regardless of the server default.

---

8. Decommission SQLite

Once you have confirmed the row counts match and the app is running correctly against MariaDB:

# Optional: keep a backup
cp batteries.db batteries.db.bak

# Remove the SQLite file
rm batteries.db

The DATABASE_URL environment variable now fully controls which database is used — no code changes were required.