Database Refactoring Complete: File-Based Storage
Overview
Successfully refactored TribeGen from a complex database-heavy architecture to a hybrid approach:
- Database (SQLite): User accounts only (users table)
- File System (JSON): All game data in
/data/saves/{userId}/{saveId}/directory structure
Architecture Changes
Storage Structure
/data/saves/
├── {userId}/
│ ├── 1/ # Save 1
│ │ ├── save.json # Tribe metadata (name, month, season, food_count, dates)
│ │ ├── kobolds.json # Array of all kobolds with stats/injuries/items
│ │ ├── items.json # Tribe storage items
│ │ └── npcs.json # NPC list
│ ├── 2/ # Save 2
│ └── kobolds/
│ ├── {koboldId}/
│ │ └── sprites/ # Modified sprite files for this kobold
Refactored Classes
SaveManager (NEW)
- Purpose: Central abstraction for all file I/O operations
- Key Methods:
getUserSavesPath($userId)- creates and returns user save directorygetSavePath($userId, $saveId)- creates and returns specific save directorygetKoboldSpritePath($userId, $koboldId)- creates and returns sprite storage pathsaveJson($filePath, $data)- writes JSON with error handling and pretty-printingloadJson($filePath)- reads JSON with null handling on missing fileslistUserSaves($userId)- scans directory, loads save.json files, returns sorted by last_playedgenerateSaveId($userId)- generates next numeric save ID by scanning directorydeleteSave($userId, $saveId)- recursive directory deletion- Location:
src/System/SaveManager.php(115 lines)
Tribe (REFACTORED)
- Previous: Used database for all save data persistence
- Now:
- Loads/saves all data from JSON files via SaveManager
create()- creates new save with SaveManager ID generationload()- loads from JSON files (requires userId and saveId now)save()- writes save.json (metadata), kobolds.json (array), items.json- Constructor now takes
$userIdand$saveIdparameters loadFromFile()- reads save.json on initializationloadKobolds()- reads kobolds.json and reconstructs kobold objectsloadStorage()- reads items.json- Location:
src/Classes/Tribe.php
Kobold (REFACTORED)
- Previous: Used database INSERT/UPDATE/SELECT for persistence
- Now:
- Stored as part of Tribe's kobolds.json array
- Constructor requires
$userIdparameter (new) - Uses
uniqid('kob_')for unique ID generation instead of database auto-increment - New methods:
fromArray()andtoArray()for JSON serialization save()is now a no-op (saves via Tribe.save())- Injuries now use
uniqid('inj_')for IDs instead of database PKs - Location:
src/Classes/Kobold.php
Item (REFACTORED)
- Previous: Used database for stat modifications lookup
- Now:
- Constructor includes
$statModificationsparameter - New methods:
fromArray()andtoArray()for JSON serialization - Removed
loadModifications()- all data passed in constructor - Location:
src/Classes/Item.php
NPC (REFACTORED)
- Previous: Used database INSERT/UPDATE for persistence
- Now:
- Creates with
uniqid('npc_')for IDs - New methods:
fromArray()andtoArray()for JSON serialization save()is now a no-op (saves via Tribe.save())- Location:
src/Classes/NPC.php
User (REFACTORED)
- No Change to Auth: Database still used for register/login (password hashing)
- Changed getSaves(): Now uses
SaveManager.listUserSaves()instead of SQL query - Location:
src/Classes/User.php
Database (SIMPLIFIED)
- Previous: 13 tables (saves, kobolds, injuries, items, kobold_items, item_modifications, events_enabled, injuries_enabled, npcs, etc.)
- Now: 1 table only -
users(id, email, password_hash, created_at) - Impact: Removed all game-related table creation from
initializeTables() - Location:
src/Database/Database.php
API Changes
game.php (UPDATED)
- Tribe::create(): Now requires
$userIdparameter - Tribe::load(): Now requires both
$userIdand$saveIdparameters - Kobold::generate(): Now requires
$userIdand$saveIdparameters - listUserSaves: Uses
SaveManager.listUserSaves()instead of database query - Generation endpoint: Creates initial 20 kobolds and saves via Tribe.save()
- Load endpoint: Retrieves from JSON files and constructs objects
- Location:
src/api/game.php
File Serialization Format
save.json (Tribe Metadata)
{
"name": "My Tribe",
"current_month": 0,
"current_season": "spring",
"food_count": 100,
"created_at": "2024-01-15 10:30:45",
"last_played": "2024-01-15 11:45:30"
}
kobolds.json (All Kobolds)
[
{
"id": "kob_12345abc",
"name": "Grok",
"age": 15,
"stats": {
"vitality": 75,
"intelligence": 60,
...
},
"sprite_morph": "default",
"is_leader": true,
"is_alive": true,
"is_expelled": false,
"injuries": [
{
"id": "inj_xyz",
"category": "wound",
"injury_type": "cut_arm",
"severity": 2
}
],
"items": [],
"father_id": null,
"mother_id": null
}
]
items.json (Tribe Storage)
[
{
"id": "itm_abc123",
"save_id": "1",
"name": "Iron Axe",
"description": "A sturdy iron axe",
"item_type": "weapon",
"stat_modifications": {
"crafting": 5
}
}
]
Key Improvements
1. Portability: All game saves are JSON files, easily portable and versionable with git 2. Inspectability: Can manually examine and edit save files 3. Reduced Database Load: Only user authentication uses database 4. Simpler Queries: No complex joins or foreign key constraints 5. Faster Iteration: No need for migrations for game data changes 6. Backup Friendly: Can zip/copy individual save directories
Migration Path
If existing saves exist in database: 1. Query the old tables 2. Construct Tribe/Kobold/Item/NPC objects 3. Save via Tribe.save() to create JSON files 4. Can then safely remove database tables
Testing Considerations
- Create new save: Tests Tribe.create() and JSON file creation
- Load save: Tests Tribe.load() and JSON parsing
- List saves: Tests SaveManager.listUserSaves() sorting and filtering
- End month: Tests Tribe.endMonth() and file persistence
- Food system: Already integrated with JSON storage
Backwards Compatibility
All game classes maintain the same public API:
Tribe::create(),Tribe::load()still existKobold.save(),addInjury(),removeInjury()still work- No changes to gameplay logic, only persistence layer
The only API change is method signatures requiring userId parameter for Tribe/Kobold operations.
Source
| 1 | # Database Refactoring Complete: File-Based Storage |
| 2 | |
| 3 | ## Overview |
| 4 | Successfully refactored TribeGen from a complex database-heavy architecture to a hybrid approach: |
| 5 | - **Database (SQLite)**: User accounts only (users table) |
| 6 | - **File System (JSON)**: All game data in `/data/saves/{userId}/{saveId}/` directory structure |
| 7 | |
| 8 | ## Architecture Changes |
| 9 | |
| 10 | ### Storage Structure |
| 11 | ``` |
| 12 | /data/saves/ |
| 13 | ├── {userId}/ |
| 14 | │ ├── 1/ # Save 1 |
| 15 | │ │ ├── save.json # Tribe metadata (name, month, season, food_count, dates) |
| 16 | │ │ ├── kobolds.json # Array of all kobolds with stats/injuries/items |
| 17 | │ │ ├── items.json # Tribe storage items |
| 18 | │ │ └── npcs.json # NPC list |
| 19 | │ ├── 2/ # Save 2 |
| 20 | │ └── kobolds/ |
| 21 | │ ├── {koboldId}/ |
| 22 | │ │ └── sprites/ # Modified sprite files for this kobold |
| 23 | ``` |
| 24 | |
| 25 | ### Refactored Classes |
| 26 | |
| 27 | #### SaveManager (NEW) |
| 28 | - **Purpose**: Central abstraction for all file I/O operations |
| 29 | - **Key Methods**: |
| 30 | - `getUserSavesPath($userId)` - creates and returns user save directory |
| 31 | - `getSavePath($userId, $saveId)` - creates and returns specific save directory |
| 32 | - `getKoboldSpritePath($userId, $koboldId)` - creates and returns sprite storage path |
| 33 | - `saveJson($filePath, $data)` - writes JSON with error handling and pretty-printing |
| 34 | - `loadJson($filePath)` - reads JSON with null handling on missing files |
| 35 | - `listUserSaves($userId)` - scans directory, loads save.json files, returns sorted by last_played |
| 36 | - `generateSaveId($userId)` - generates next numeric save ID by scanning directory |
| 37 | - `deleteSave($userId, $saveId)` - recursive directory deletion |
| 38 | - **Location**: `src/System/SaveManager.php` (115 lines) |
| 39 | |
| 40 | #### Tribe (REFACTORED) |
| 41 | - **Previous**: Used database for all save data persistence |
| 42 | - **Now**: |
| 43 | - Loads/saves all data from JSON files via SaveManager |
| 44 | - `create()` - creates new save with SaveManager ID generation |
| 45 | - `load()` - loads from JSON files (requires userId and saveId now) |
| 46 | - `save()` - writes save.json (metadata), kobolds.json (array), items.json |
| 47 | - Constructor now takes `$userId` and `$saveId` parameters |
| 48 | - `loadFromFile()` - reads save.json on initialization |
| 49 | - `loadKobolds()` - reads kobolds.json and reconstructs kobold objects |
| 50 | - `loadStorage()` - reads items.json |
| 51 | - **Location**: `src/Classes/Tribe.php` |
| 52 | |
| 53 | #### Kobold (REFACTORED) |
| 54 | - **Previous**: Used database INSERT/UPDATE/SELECT for persistence |
| 55 | - **Now**: |
| 56 | - Stored as part of Tribe's kobolds.json array |
| 57 | - Constructor requires `$userId` parameter (new) |
| 58 | - Uses `uniqid('kob_')` for unique ID generation instead of database auto-increment |
| 59 | - New methods: `fromArray()` and `toArray()` for JSON serialization |
| 60 | - `save()` is now a no-op (saves via Tribe.save()) |
| 61 | - Injuries now use `uniqid('inj_')` for IDs instead of database PKs |
| 62 | - **Location**: `src/Classes/Kobold.php` |
| 63 | |
| 64 | #### Item (REFACTORED) |
| 65 | - **Previous**: Used database for stat modifications lookup |
| 66 | - **Now**: |
| 67 | - Constructor includes `$statModifications` parameter |
| 68 | - New methods: `fromArray()` and `toArray()` for JSON serialization |
| 69 | - Removed `loadModifications()` - all data passed in constructor |
| 70 | - **Location**: `src/Classes/Item.php` |
| 71 | |
| 72 | #### NPC (REFACTORED) |
| 73 | - **Previous**: Used database INSERT/UPDATE for persistence |
| 74 | - **Now**: |
| 75 | - Creates with `uniqid('npc_')` for IDs |
| 76 | - New methods: `fromArray()` and `toArray()` for JSON serialization |
| 77 | - `save()` is now a no-op (saves via Tribe.save()) |
| 78 | - **Location**: `src/Classes/NPC.php` |
| 79 | |
| 80 | #### User (REFACTORED) |
| 81 | - **No Change to Auth**: Database still used for register/login (password hashing) |
| 82 | - **Changed getSaves()**: Now uses `SaveManager.listUserSaves()` instead of SQL query |
| 83 | - **Location**: `src/Classes/User.php` |
| 84 | |
| 85 | #### Database (SIMPLIFIED) |
| 86 | - **Previous**: 13 tables (saves, kobolds, injuries, items, kobold_items, item_modifications, events_enabled, injuries_enabled, npcs, etc.) |
| 87 | - **Now**: 1 table only - `users` (id, email, password_hash, created_at) |
| 88 | - **Impact**: Removed all game-related table creation from `initializeTables()` |
| 89 | - **Location**: `src/Database/Database.php` |
| 90 | |
| 91 | ### API Changes |
| 92 | |
| 93 | #### game.php (UPDATED) |
| 94 | - **Tribe::create()**: Now requires `$userId` parameter |
| 95 | - **Tribe::load()**: Now requires both `$userId` and `$saveId` parameters |
| 96 | - **Kobold::generate()**: Now requires `$userId` and `$saveId` parameters |
| 97 | - **listUserSaves**: Uses `SaveManager.listUserSaves()` instead of database query |
| 98 | - **Generation endpoint**: Creates initial 20 kobolds and saves via Tribe.save() |
| 99 | - **Load endpoint**: Retrieves from JSON files and constructs objects |
| 100 | - **Location**: `src/api/game.php` |
| 101 | |
| 102 | ### File Serialization Format |
| 103 | |
| 104 | #### save.json (Tribe Metadata) |
| 105 | ```json |
| 106 | { |
| 107 | "name": "My Tribe", |
| 108 | "current_month": 0, |
| 109 | "current_season": "spring", |
| 110 | "food_count": 100, |
| 111 | "created_at": "2024-01-15 10:30:45", |
| 112 | "last_played": "2024-01-15 11:45:30" |
| 113 | } |
| 114 | ``` |
| 115 | |
| 116 | #### kobolds.json (All Kobolds) |
| 117 | ```json |
| 118 | [ |
| 119 | { |
| 120 | "id": "kob_12345abc", |
| 121 | "name": "Grok", |
| 122 | "age": 15, |
| 123 | "stats": { |
| 124 | "vitality": 75, |
| 125 | "intelligence": 60, |
| 126 | ... |
| 127 | }, |
| 128 | "sprite_morph": "default", |
| 129 | "is_leader": true, |
| 130 | "is_alive": true, |
| 131 | "is_expelled": false, |
| 132 | "injuries": [ |
| 133 | { |
| 134 | "id": "inj_xyz", |
| 135 | "category": "wound", |
| 136 | "injury_type": "cut_arm", |
| 137 | "severity": 2 |
| 138 | } |
| 139 | ], |
| 140 | "items": [], |
| 141 | "father_id": null, |
| 142 | "mother_id": null |
| 143 | } |
| 144 | ] |
| 145 | ``` |
| 146 | |
| 147 | #### items.json (Tribe Storage) |
| 148 | ```json |
| 149 | [ |
| 150 | { |
| 151 | "id": "itm_abc123", |
| 152 | "save_id": "1", |
| 153 | "name": "Iron Axe", |
| 154 | "description": "A sturdy iron axe", |
| 155 | "item_type": "weapon", |
| 156 | "stat_modifications": { |
| 157 | "crafting": 5 |
| 158 | } |
| 159 | } |
| 160 | ] |
| 161 | ``` |
| 162 | |
| 163 | ## Key Improvements |
| 164 | |
| 165 | 1. **Portability**: All game saves are JSON files, easily portable and versionable with git |
| 166 | 2. **Inspectability**: Can manually examine and edit save files |
| 167 | 3. **Reduced Database Load**: Only user authentication uses database |
| 168 | 4. **Simpler Queries**: No complex joins or foreign key constraints |
| 169 | 5. **Faster Iteration**: No need for migrations for game data changes |
| 170 | 6. **Backup Friendly**: Can zip/copy individual save directories |
| 171 | |
| 172 | ## Migration Path |
| 173 | |
| 174 | If existing saves exist in database: |
| 175 | 1. Query the old tables |
| 176 | 2. Construct Tribe/Kobold/Item/NPC objects |
| 177 | 3. Save via Tribe.save() to create JSON files |
| 178 | 4. Can then safely remove database tables |
| 179 | |
| 180 | ## Testing Considerations |
| 181 | |
| 182 | - Create new save: Tests Tribe.create() and JSON file creation |
| 183 | - Load save: Tests Tribe.load() and JSON parsing |
| 184 | - List saves: Tests SaveManager.listUserSaves() sorting and filtering |
| 185 | - End month: Tests Tribe.endMonth() and file persistence |
| 186 | - Food system: Already integrated with JSON storage |
| 187 | |
| 188 | ## Backwards Compatibility |
| 189 | |
| 190 | All game classes maintain the same public API: |
| 191 | - `Tribe::create()`, `Tribe::load()` still exist |
| 192 | - `Kobold.save()`, `addInjury()`, `removeInjury()` still work |
| 193 | - No changes to gameplay logic, only persistence layer |
| 194 | |
| 195 | The only API change is method signatures requiring `userId` parameter for Tribe/Kobold operations. |