193 lines
4.8 KiB
Markdown
193 lines
4.8 KiB
Markdown
# Schema Migrations
|
|
|
|
## Adding Home Assistant fields (ha_entity_id, battery_percentage)
|
|
|
|
These columns were added in the Home Assistant integration feature. Existing databases
|
|
need a manual migration — `create_all()` does not add columns to existing tables.
|
|
|
|
**Always snapshot first:**
|
|
```bash
|
|
cp batteries.db batteries.db.$(date +%Y-%m-%d).snapshot
|
|
```
|
|
|
|
**SQLite:**
|
|
```bash
|
|
sqlite3 batteries.db "ALTER TABLE device ADD COLUMN ha_entity_id VARCHAR(100);"
|
|
sqlite3 batteries.db "ALTER TABLE battery ADD COLUMN battery_percentage INTEGER;"
|
|
```
|
|
|
|
**MariaDB / MySQL:**
|
|
```sql
|
|
ALTER TABLE device ADD COLUMN ha_entity_id VARCHAR(100) NULL;
|
|
ALTER TABLE battery ADD COLUMN battery_percentage INT NULL;
|
|
```
|
|
|
|
Both columns are nullable with no default, which is valid on all supported databases.
|
|
|
|
---
|
|
|
|
# Migrating from SQLite to MariaDB
|
|
|
|
This guide covers moving the Battery Tracker database from SQLite to MariaDB/MySQL
|
|
without changing any application code — only the connection string changes.
|
|
|
|
---
|
|
|
|
## 1. Install PyMySQL
|
|
|
|
```bash
|
|
source .venv/bin/activate
|
|
pip install pymysql
|
|
pip freeze > requirements.txt
|
|
```
|
|
|
|
---
|
|
|
|
## 2. Create the MariaDB database and user
|
|
|
|
```sql
|
|
CREATE DATABASE batteries CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
|
CREATE USER 'battuser'@'localhost' IDENTIFIED BY 'strongpassword';
|
|
GRANT ALL PRIVILEGES ON batteries.* TO 'battuser'@'localhost';
|
|
FLUSH PRIVILEGES;
|
|
```
|
|
|
|
---
|
|
|
|
## 3. Update the connection string
|
|
|
|
Set the `DATABASE_URL` environment variable before starting the app:
|
|
|
|
```bash
|
|
export DATABASE_URL='mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4'
|
|
```
|
|
|
|
Or add it to a `.env` file and load it with your process manager / systemd service:
|
|
|
|
```ini
|
|
# In ~/.config/systemd/user/battery-tracker.service, add under [Service]:
|
|
Environment=DATABASE_URL=mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4
|
|
```
|
|
|
|
---
|
|
|
|
## 4. (Optional) Use Flask-Migrate for schema management
|
|
|
|
Flask-Migrate lets you apply schema changes incrementally via `flask db upgrade`
|
|
rather than dropping and recreating tables.
|
|
|
|
```bash
|
|
pip install Flask-Migrate
|
|
```
|
|
|
|
In `app.py`, after the `db` setup, add:
|
|
|
|
```python
|
|
from flask_migrate import Migrate
|
|
migrate = Migrate(app, db) # pass your SQLAlchemy db object
|
|
```
|
|
|
|
Then:
|
|
|
|
```bash
|
|
flask db init # first time only — creates migrations/ directory
|
|
flask db migrate -m "initial schema"
|
|
flask db upgrade
|
|
```
|
|
|
|
> **Note:** The app uses raw SQLAlchemy (not Flask-SQLAlchemy), so you will need
|
|
> to adapt the `Migrate(app, db)` call to wrap your engine/session. Alternatively,
|
|
> run `Base.metadata.create_all(engine)` directly for the initial schema — the app
|
|
> already does this on startup.
|
|
|
|
---
|
|
|
|
## 5. Run the migration script
|
|
|
|
The `migrate_to_mariadb.py` script reads every record from SQLite and inserts
|
|
it into MariaDB using the SQLAlchemy ORM — no raw SQL, no CSV exports.
|
|
|
|
```bash
|
|
MARIADB_URL='mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4' \
|
|
python migrate_to_mariadb.py
|
|
```
|
|
|
|
Or pass the URL as a positional argument:
|
|
|
|
```bash
|
|
python migrate_to_mariadb.py 'mysql+pymysql://battuser:strongpassword@localhost/batteries?charset=utf8mb4'
|
|
```
|
|
|
|
The script:
|
|
1. Creates all tables on MariaDB if they don't exist
|
|
2. Inserts all Device records (preserving primary keys)
|
|
3. Inserts all Battery records (preserving primary keys and foreign keys)
|
|
4. Resets `AUTO_INCREMENT` counters past the highest migrated ID
|
|
5. Prints a verification table comparing row counts
|
|
|
|
---
|
|
|
|
## 6. Verify row counts
|
|
|
|
The migration script prints a summary:
|
|
|
|
```
|
|
=== Verification ===
|
|
Table SQLite MariaDB OK?
|
|
--------------------------------------
|
|
device 5 5 OK
|
|
battery 40 40 OK
|
|
|
|
Migration complete. All row counts match.
|
|
```
|
|
|
|
If you see `MISMATCH`, **do not** decommission SQLite. Re-run the script after
|
|
investigating the error output.
|
|
|
|
You can also verify manually in the MariaDB shell:
|
|
|
|
```sql
|
|
SELECT COUNT(*) FROM device;
|
|
SELECT COUNT(*) FROM battery;
|
|
```
|
|
|
|
---
|
|
|
|
## 7. MariaDB configuration notes
|
|
|
|
| Setting | Recommended value |
|
|
|---|---|
|
|
| Character set | `utf8mb4` |
|
|
| Collation | `utf8mb4_unicode_ci` |
|
|
| `innodb_strict_mode` | `ON` (default in MariaDB 10.2+) |
|
|
| `sql_mode` | Include `STRICT_TRANS_TABLES` |
|
|
|
|
Set these in `/etc/mysql/mariadb.conf.d/50-server.cnf` (or equivalent):
|
|
|
|
```ini
|
|
[mysqld]
|
|
character-set-server = utf8mb4
|
|
collation-server = utf8mb4_unicode_ci
|
|
```
|
|
|
|
The `?charset=utf8mb4` query parameter in the connection string ensures the
|
|
client connection also uses utf8mb4, regardless of the server default.
|
|
|
|
---
|
|
|
|
## 8. Decommission SQLite
|
|
|
|
Once you have confirmed the row counts match and the app is running correctly
|
|
against MariaDB:
|
|
|
|
```bash
|
|
# Optional: keep a backup
|
|
cp batteries.db batteries.db.bak
|
|
|
|
# Remove the SQLite file
|
|
rm batteries.db
|
|
```
|
|
|
|
The `DATABASE_URL` environment variable now fully controls which database is used —
|
|
no code changes were required.
|