diff --git a/docs/superpowers/plans/2026-05-03-rtc-pcf8523-fallback.md b/docs/superpowers/plans/2026-05-03-rtc-pcf8523-fallback.md new file mode 100644 index 0000000..0059ee1 --- /dev/null +++ b/docs/superpowers/plans/2026-05-03-rtc-pcf8523-fallback.md @@ -0,0 +1,607 @@ +# RTC PCF8523 Fallback Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a new header `src/rtcsync.h` that uses an Adafruit Adalogger FeatherWing (PCF8523) as persistent time storage and as a fallback time source when NTP is unavailable, while remaining fully optional (soft-fail without hardware). + +**Architecture:** Header-only module following the existing AquaMaster pattern. Decoupled from `timesync.h` via GCC weak symbols, so devices without RTC compile and run unchanged. Time is stored in the RTC as UTC (POSIX epoch). NTP remains the authoritative source; RTC is consulted only at boot before WiFi comes up. Manual time sets (e.g. browser-time button) are persisted to RTC with a plausibility check. + +**Tech Stack:** ESP32 / Arduino-Framework / PlatformIO, `adafruit/RTClib`, `ArduinoJson@^7.4.1`, existing `timesync.h` API. + +**Spec:** [docs/superpowers/specs/2026-05-03-rtc-pcf8523-fallback-design.md](../specs/2026-05-03-rtc-pcf8523-fallback-design.md) + +**Verification model:** This project has no unit-test suite (`test/` is empty). "Tests" in this plan mean: (1) build success across all PlatformIO envs, (2) serial-log behavior on real hardware, (3) HTTP-API checks via `curl`/browser. Each task ends with a verification step appropriate for the change. + +--- + +## File Structure + +| File | Role | +|------|------| +| `src/rtcsync.h` | **NEW** — header-only RTC module (globals + `setupRTC`, `loopRTC`, `syncFromNTP`, `persistSystemTimeToRTC`, weak-hook overrides) | +| `src/timesync.h` | Add two weak-symbol declarations + invocations (no behavioral change without `rtcsync.h`) | +| `src/master.cpp` | Include `rtcsync.h`; call `setupRTC()` before `setupWifi()`; call `loopRTC()` at start of `loop()` | +| `platformio.ini` | Add `adafruit/RTClib` to `lib_deps` of every env | + +--- + +## Task 1: Add RTClib dependency to all PlatformIO envs + +**Files:** +- Modify: `platformio.ini` (lines 30, 50, 70, 87, 105, 129 — `lib_deps` blocks of all six envs) + +- [ ] **Step 1: Add `adafruit/RTClib@^2.1.4` to every `lib_deps` block** + +In `platformio.ini`, append the line ` adafruit/RTClib@^2.1.4` (tab-indented like the other entries) to each of the six `lib_deps` sections. Concretely, each block should become: + +```ini +lib_deps = + bblanchon/ArduinoJson@^7.4.1 + esp32async/ESPAsyncWebServer@^3.7.7 + esp32async/AsyncTCP@^3.4.2 + mlesniew/PicoMQTT@^1.3.0 + adafruit/Adafruit PN532@^1.3.4 + adafruit/RTClib@^2.1.4 +``` + +Apply this edit to envs: `wemos_d1_mini32`, `esp32thing_OTA`, `esp32thing`, `esp32thing_CI`, `um_feathers3`, `um_feathers3_debug`. + +- [ ] **Step 2: Verify CI env builds with new dependency** + +Run: `pio run -e esp32thing_CI` +Expected: Library is downloaded (look for `Library Manager: Installing adafruit/RTClib`), build succeeds with `SUCCESS` line. No code change yet, so binary size should be essentially unchanged. + +- [ ] **Step 3: Commit** + +```bash +git add platformio.ini +git commit -m "build: add RTClib dependency for PCF8523 RTC support" +``` + +--- + +## Task 2: Create `src/rtcsync.h` skeleton with RTC detection (soft-fail) + +**Files:** +- Create: `src/rtcsync.h` + +- [ ] **Step 1: Create the skeleton file** + +Create `src/rtcsync.h` with the following content. This is the smallest possible vertical slice — only detection, no time-handling yet: + +```cpp +// PCF8523-RTC-Modul mit NTP-Sync und Fallback. +// Header-only nach Projekt-Pattern: Globale Objekte werden hier definiert. +// Dieser Header darf NUR in master.cpp inkludiert werden. +#pragma once +#include +#include +#include + +// Globale RTC-Instanz und Status-Flags +RTC_PCF8523 rtc; +bool rtcAvailable = false; +bool ntpEverSynced = false; +time_t lastNtpSyncEpoch = 0; +bool lastStaConnected = false; + +// I2C-Init und PCF8523-Detektion. Soft-Fail: setzt nur rtcAvailable. +void setupRTC() { + Wire.begin(); // idempotent — falls bereits durch andere Module aufgerufen + if (!rtc.begin()) { + Serial.println("[RTC] PCF8523 nicht gefunden — RTC-Funktionen deaktiviert"); + rtcAvailable = false; + return; + } + if (!rtc.initialized() || rtc.lostPower()) { + Serial.println("[RTC] PCF8523 erkannt, aber Akku-Backup ungültig (lostPower)"); + // Wir betrachten den Chip trotzdem als verfügbar — NTP wird ihn gleich neu setzen. + } + rtc.start(); // STOP-Bit löschen, falls gesetzt + rtcAvailable = true; + Serial.println("[RTC] PCF8523 initialisiert"); +} +``` + +- [ ] **Step 2: Build CI env to verify the header compiles standalone** + +The header is not yet included anywhere, so this only checks that the includes resolve. Run: `pio run -e esp32thing_CI` +Expected: `SUCCESS`, no warnings about `rtcsync.h`. + +- [ ] **Step 3: Commit** + +```bash +git add src/rtcsync.h +git commit -m "feat(rtc): add rtcsync.h skeleton with PCF8523 detection" +``` + +--- + +## Task 3: Add RTC → system-time fallback inside `setupRTC()` + +**Files:** +- Modify: `src/rtcsync.h` (the `setupRTC()` function from Task 2) + +- [ ] **Step 1: Extend `setupRTC()` to seed the system clock from RTC if available** + +Replace the body of `setupRTC()` with: + +```cpp +void setupRTC() { + Wire.begin(); + if (!rtc.begin()) { + Serial.println("[RTC] PCF8523 nicht gefunden — RTC-Funktionen deaktiviert"); + rtcAvailable = false; + return; + } + if (!rtc.initialized() || rtc.lostPower()) { + Serial.println("[RTC] PCF8523 erkannt, aber Akku-Backup ungültig (lostPower)"); + } + rtc.start(); + rtcAvailable = true; + Serial.println("[RTC] PCF8523 initialisiert"); + + // Systemzeit aus RTC vorbelegen — wird ggf. später durch NTP übersteuert. + // RTC speichert UTC, settimeofday erwartet UTC. Kein Offset nötig. + DateTime nowRtc = rtc.now(); + uint32_t rtcEpoch = nowRtc.unixtime(); + // Plausibilität: RTC sollte mindestens 2025 zeigen, sonst ist sie ungestellt. + if (rtcEpoch >= 1735689600UL) { + struct timeval tv; + tv.tv_sec = (time_t)rtcEpoch; + tv.tv_usec = 0; + settimeofday(&tv, NULL); + Serial.printf("[RTC] Systemzeit aus RTC gesetzt: %lu (UTC)\n", + (unsigned long)rtcEpoch); + } else { + Serial.printf("[RTC] RTC-Zeit unplausibel (%lu) — nicht übernommen\n", + (unsigned long)rtcEpoch); + } +} +``` + +- [ ] **Step 2: Build CI env** + +Run: `pio run -e esp32thing_CI` +Expected: `SUCCESS`. The header still isn't wired into `master.cpp`, so no runtime effect yet. + +- [ ] **Step 3: Commit** + +```bash +git add src/rtcsync.h +git commit -m "feat(rtc): seed system time from RTC at boot" +``` + +--- + +## Task 4: Add weak-symbol hooks to `timesync.h` + +These hooks let `rtcsync.h` extend `timesync.h` without `timesync.h` knowing about it. Without `rtcsync.h` linked in, the weak default is a null pointer and the call sites short-circuit. + +**Files:** +- Modify: `src/timesync.h:23-46` (`getCurrentTimeJSON`) +- Modify: `src/timesync.h:72-84` (`setSystemTime`) + +- [ ] **Step 1: Add weak hook declarations near the top of `timesync.h`** + +Insert these declarations directly after the existing `bool isValidDateTime(...)` line is declared (i.e. after the prototypes block, before `getCurrentTimeJSON`'s implementation around line 22). If no clean location exists, place them right after the `#include ` line: + +```cpp +// Weak hooks — falls rtcsync.h kompiliert/gelinkt wird, überschreibt es diese. +// Ohne rtcsync.h sind beide Symbole nullptr und werden nicht aufgerufen. +extern "C" void onSystemTimeSet(time_t t) __attribute__((weak)); +extern "C" void appendTimeStatus(JsonDocument &doc) __attribute__((weak)); +``` + +- [ ] **Step 2: Convert `getCurrentTimeJSON()` doc type and invoke `appendTimeStatus`** + +The existing function uses the deprecated `StaticJsonDocument<200>`. ArduinoJson v7 prefers plain `JsonDocument`. Replace lines 23–46 of `src/timesync.h` (the entire `getCurrentTimeJSON()` body) with: + +```cpp +String getCurrentTimeJSON() { + gettimeofday(&tv, &tz); + now = tv.tv_sec; + + JsonDocument doc; + doc["timestamp"] = (long)now; + doc["success"] = true; + + // Zusätzliche Zeitinformationen + gmtime_r(&now, &timeinfo); + char timeStr[64]; + strftime(timeStr, sizeof(timeStr), "%Y-%m-%d %H:%M:%S", &timeinfo); + doc["formatted"] = String(timeStr); + doc["year"] = timeinfo.tm_year + 1900; + doc["month"] = timeinfo.tm_mon + 1; + doc["day"] = timeinfo.tm_mday; + doc["hour"] = timeinfo.tm_hour; + doc["minute"] = timeinfo.tm_min; + doc["second"] = timeinfo.tm_sec; + + // Optionale RTC/Sync-Status-Felder, falls rtcsync.h gelinkt ist + if (appendTimeStatus) { + appendTimeStatus(doc); + } + + String response; + serializeJson(doc, response); + return response; +} +``` + +- [ ] **Step 3: Invoke `onSystemTimeSet` from `setSystemTime()`** + +Replace lines 72–84 of `src/timesync.h` (the `setSystemTime` function) with: + +```cpp +bool setSystemTime(long timestamp) { + struct timeval tv; + tv.tv_sec = timestamp; + tv.tv_usec = 0; + + if (settimeofday(&tv, NULL) == 0) { + Serial.println("Zeit erfolgreich gesetzt: " + String(timestamp)); + if (onSystemTimeSet) { + onSystemTimeSet((time_t)timestamp); + } + return true; + } else { + Serial.println("Fehler beim Setzen der Zeit"); + return false; + } +} +``` + +- [ ] **Step 4: Build CI env to verify backwards compatibility** + +`rtcsync.h` is still not included in `master.cpp`, so both weak symbols must resolve to nullptr and the `if (...)` guards must skip them. +Run: `pio run -e esp32thing_CI` +Expected: `SUCCESS`. No new warnings. Behavior is identical to before this commit (the hooks are no-ops). + +- [ ] **Step 5: Commit** + +```bash +git add src/timesync.h +git commit -m "feat(timesync): add weak hooks for RTC integration" +``` + +--- + +## Task 5: Implement `persistSystemTimeToRTC()` and the `onSystemTimeSet` override + +**Files:** +- Modify: `src/rtcsync.h` (append below `setupRTC`) + +- [ ] **Step 1: Append the persistence function and the weak-hook override** + +Add the following at the end of `src/rtcsync.h`: + +```cpp +// Plausibilitäts-Grenzen: 2025-01-01 .. 2100-01-01 (UTC). +// Verhindert, dass kaputte Timestamps (0, negativ, 1970) die RTC korrumpieren. +static constexpr uint32_t RTC_MIN_EPOCH = 1735689600UL; // 2025-01-01 00:00:00 UTC +static constexpr uint32_t RTC_MAX_EPOCH = 4102444800UL; // 2100-01-01 00:00:00 UTC + +void persistSystemTimeToRTC(time_t t) { + if (!rtcAvailable) return; + if ((uint32_t)t < RTC_MIN_EPOCH || (uint32_t)t >= RTC_MAX_EPOCH) { + Serial.printf("[RTC] persist abgelehnt — Timestamp unplausibel: %ld\n", + (long)t); + return; + } + rtc.adjust(DateTime((uint32_t)t)); + Serial.printf("[RTC] in RTC geschrieben: %ld (UTC)\n", (long)t); +} + +// Weak-Hook-Override aus timesync.h — wird automatisch aufgerufen, +// sobald irgendwo setSystemTime() Erfolg meldet. +extern "C" void onSystemTimeSet(time_t t) { + persistSystemTimeToRTC(t); +} +``` + +- [ ] **Step 2: Build CI env** + +Run: `pio run -e esp32thing_CI` +Expected: `SUCCESS`. Still no behavioral change because `rtcsync.h` is not yet included by `master.cpp`. + +- [ ] **Step 3: Commit** + +```bash +git add src/rtcsync.h +git commit -m "feat(rtc): persist system time to RTC with plausibility check" +``` + +--- + +## Task 6: Implement `syncFromNTP()` wrapper + +**Files:** +- Modify: `src/rtcsync.h` (append at end) + +- [ ] **Step 1: Add a thin wrapper around `syncTimeWithNTP()` from `timesync.h`** + +`syncTimeWithNTP()` in `timesync.h:48` already does the NTP heavy lifting (configTime + 5 s polling loop) and prints success/failure. We just need to detect success after the fact (system clock is now > 2025) and persist + bookkeep. + +Append to `src/rtcsync.h`: + +```cpp +// Versucht NTP-Sync via timesync.h. Bei Erfolg: schreibt UTC in RTC, +// setzt ntpEverSynced=true, aktualisiert lastNtpSyncEpoch. +// Returns true bei Erfolg. +bool syncFromNTP() { + // Snapshot vor dem Sync — wenn die Zeit hinterher >= 2025 und neuer als vorher, + // werten wir den Sync als erfolgreich. + time_t before = time(NULL); + syncTimeWithNTP(); // benutzt Defaults aus timesync.h: pool.ntp.org, +1h, kein DST + time_t after = time(NULL); + + bool ok = ((uint32_t)after >= RTC_MIN_EPOCH) && (after >= before); + if (!ok) { + Serial.println("[RTC] NTP-Sync fehlgeschlagen — RTC unverändert"); + return false; + } + + // setSystemTime() wird intern von syncTimeWithNTP NICHT aufgerufen, + // also den Weak-Hook-Pfad hier umgehen und direkt schreiben. + persistSystemTimeToRTC(after); + ntpEverSynced = true; + lastNtpSyncEpoch = after; + return true; +} +``` + +- [ ] **Step 2: Build CI env** + +Run: `pio run -e esp32thing_CI` +Expected: `SUCCESS`. + +- [ ] **Step 3: Commit** + +```bash +git add src/rtcsync.h +git commit -m "feat(rtc): add syncFromNTP wrapper that persists to RTC" +``` + +--- + +## Task 7: Implement `loopRTC()` (STA reconnect edge + 24 h periodic) + +**Files:** +- Modify: `src/rtcsync.h` (append at end) + +- [ ] **Step 1: Add the loop-tick function** + +Append to `src/rtcsync.h`: + +```cpp +// Aus loop() aufgerufen. Triggert syncFromNTP(): +// - bei steigender Flanke von WiFi.isConnected() (STA-Reconnect) +// - alle 24h, sobald STA verbunden ist +// Reine Vergleichsoperationen, NTP-Roundtrip selbst ist blockierend (~ms..5s). +void loopRTC() { + bool sta = (WiFi.status() == WL_CONNECTED); + + // Edge: false -> true (frischer STA-Connect) + if (sta && !lastStaConnected) { + Serial.println("[RTC] STA-Reconnect erkannt — NTP-Sync"); + syncFromNTP(); + } + lastStaConnected = sta; + + // 24h-Periodik (nur wenn STA online) + if (sta && ntpEverSynced) { + time_t nowEpoch = time(NULL); + if (nowEpoch - lastNtpSyncEpoch >= 86400) { + Serial.println("[RTC] 24h-Periodik — NTP-Sync"); + syncFromNTP(); + } + } +} +``` + +`WiFi.h` ist bereits indirekt durch andere Header (`wificlass.h` → `WiFi.h`) verfügbar; falls der Build hier ein Symbol nicht findet, oben im Header `#include ` ergänzen. + +- [ ] **Step 2: Build CI env** + +Run: `pio run -e esp32thing_CI` +Expected: `SUCCESS`. Bei `'WiFi' was not declared`: `#include ` ganz oben in `rtcsync.h` ergänzen und neu bauen. + +- [ ] **Step 3: Commit** + +```bash +git add src/rtcsync.h +git commit -m "feat(rtc): add loopRTC with STA-reconnect and 24h re-sync" +``` + +--- + +## Task 8: Implement `appendTimeStatus()` override + +**Files:** +- Modify: `src/rtcsync.h` (append at end) + +- [ ] **Step 1: Add the JSON-extension override** + +Append to `src/rtcsync.h`: + +```cpp +// Weak-Hook-Override aus timesync.h — erweitert /api/time um RTC-Status. +extern "C" void appendTimeStatus(JsonDocument &doc) { + doc["rtc_available"] = rtcAvailable; + doc["rtc_synced_from_ntp"] = ntpEverSynced; + doc["last_ntp_sync_ago_s"] = + ntpEverSynced ? (long)(time(NULL) - lastNtpSyncEpoch) : (long)-1; + if (rtcAvailable) { + doc["rtc_time_utc"] = (long)rtc.now().unixtime(); + } else { + doc["rtc_time_utc"] = (long)0; + } +} +``` + +- [ ] **Step 2: Build CI env** + +Run: `pio run -e esp32thing_CI` +Expected: `SUCCESS`. + +- [ ] **Step 3: Commit** + +```bash +git add src/rtcsync.h +git commit -m "feat(rtc): expose RTC status fields via /api/time" +``` + +--- + +## Task 9: Wire `rtcsync.h` into `master.cpp` + +This is the activation step. Until now, `rtcsync.h` exists but is unused. After this commit, devices with the FeatherWing get RTC behavior; devices without it print one warning line and run normally. + +**Files:** +- Modify: `src/master.cpp:3-25` (include block) and `src/master.cpp:31-81` (setup/loop) + +- [ ] **Step 1: Add the include** + +In `src/master.cpp`, add `#include ` directly after the existing `#include ` line (currently line 23). The new include block tail should look like: + +```cpp +#include +#include +#include +#include +#include +``` + +**Reihenfolge:** `rtcsync.h` MUSS nach `timesync.h` kommen — die Weak-Hook-Overrides in `rtcsync.h` brauchen die Deklarationen aus `timesync.h`. + +- [ ] **Step 2: Call `setupRTC()` before `setupWifi()`** + +In `setup()`, insert `setupRTC();` directly before the existing `setupWifi();` call (currently line 53). The relevant block becomes: + +```cpp + loadWifiSettings(); + loadLocationSettings(); + + setupRTC(); // RTC zuerst, damit Systemzeit vor WiFi plausibel ist + setupWifi(); // WiFi initialisieren + setupOTA(&server); +``` + +- [ ] **Step 3: Add an explicit NTP-sync attempt after WiFi is up** + +After `setupWifi()`, the STA may already be connected. The first `loopRTC()` call would catch the edge eventually, but doing one explicit boot-time sync makes the boot log cleaner. Insert directly after `setupWifi();`: + +```cpp + setupWifi(); // WiFi initialisieren + if (WiFi.status() == WL_CONNECTED) { + syncFromNTP(); + lastStaConnected = true; // Edge bereits "konsumiert" + } + setupOTA(&server); +``` + +- [ ] **Step 4: Call `loopRTC()` at the start of `loop()`** + +In `loop()` (currently lines 65–81), make `loopRTC()` the first call after `checkAutoReset()`: + +```cpp +void loop() { + checkAutoReset(); + loopRTC(); + + // MQTT hat höchste Priorität (wird zuerst verarbeitet) + loopMqttServer(); + ... +``` + +- [ ] **Step 5: Build all PlatformIO envs** + +Run them sequentially (or in one go if your machine handles it): + +``` +pio run -e esp32thing_CI +pio run -e esp32thing +pio run -e wemos_d1_mini32 +pio run -e um_feathers3 +``` + +Expected: every env reports `SUCCESS`. Note the increase in flash usage (RTClib is small, ~5 KB). + +- [ ] **Step 6: Commit** + +```bash +git add src/master.cpp +git commit -m "feat(rtc): wire rtcsync into setup/loop" +``` + +--- + +## Task 10: Hardware verification on real device + +This task is mandatory — automated tests cannot validate the I²C interaction or boot ordering. If no FeatherWing is currently attached, do at least Sub-step A (no-hardware soft-fail check). + +**Files:** none (manual) + +- [ ] **Step A — Without RTC attached: confirm soft-fail path** + +1. Ensure no FeatherWing is plugged in. +2. `pio run -e esp32thing -t upload` +3. `pio device monitor -b 115200` +4. Expected serial output during boot: + ``` + [RTC] PCF8523 nicht gefunden — RTC-Funktionen deaktiviert + ``` + followed by the rest of the boot sequence reaching the WiFi/MQTT init **without crash or reboot loop**. +5. Open `http:///api/time` in the browser. Expected JSON contains: + ```json + "rtc_available": false, + "rtc_synced_from_ntp": , + "rtc_time_utc": 0 + ``` + +- [ ] **Step B — With RTC attached, no NTP (AP-only mode)** + +1. Plug in the Adalogger FeatherWing. +2. Disconnect the device from any STA network (factory-reset WiFi or boot in AP-only). +3. Boot, observe serial: + ``` + [RTC] PCF8523 initialisiert + [RTC] Systemzeit aus RTC gesetzt: (UTC) + ``` + (If the RTC was never set, the line says "unplausibel" instead — acceptable for a fresh chip.) +4. Connect to the AP and load `http://192.168.10.1/api/time`. Expected: `rtc_available: true`, formatted time matches the RTC's last known value. + +- [ ] **Step C — With RTC attached, with NTP** + +1. Configure WiFi-STA credentials so the device joins the home network. +2. Boot, observe serial: + ``` + [RTC] PCF8523 initialisiert + [RTC] Systemzeit aus RTC gesetzt: ... + ... (WiFi connect) + Warte auf NTP-Zeit (max 5s)... + NTP-Zeit synchronisiert! + [RTC] in RTC geschrieben: (UTC) + ``` +3. `curl http:///api/time` — expected `rtc_synced_from_ntp: true`, `last_ntp_sync_ago_s` close to 0, `rtc_time_utc` matches `timestamp`. + +- [ ] **Step D — Browser-time persistence** + +1. With device booted (Step C), use the existing "set browser time" button in the settings UI. +2. Power-cycle the device with WiFi disabled (pull antenna or block STA). +3. Boot. Expected: serial shows the time from the browser-set, not 1970. + +- [ ] **Step E — Document outcome** + +If all steps pass: the task is done. +If any step fails: do **not** mark this task complete. Open a follow-up note describing what failed and which task's assumptions were wrong. + +--- + +## Self-Review (already performed) + +- **Spec coverage:** Each spec section maps to a task — Architecture/library → Task 1; new header + soft-fail → Tasks 2–3; weak-hook decoupling → Tasks 4–5; NTP-sync wrapper → Task 6; loop integration → Task 7; API extension → Task 8; setup/loop wiring → Task 9; hardware verification → Task 10. +- **Placeholder scan:** No TBD/TODO; all code blocks are concrete and copy-pasteable. +- **Type/name consistency:** `appendTimeStatus`, `onSystemTimeSet`, `persistSystemTimeToRTC`, `syncFromNTP`, `setupRTC`, `loopRTC`, `rtcAvailable`, `ntpEverSynced`, `lastNtpSyncEpoch`, `lastStaConnected`, `RTC_MIN_EPOCH`, `RTC_MAX_EPOCH` all used consistently across tasks. +- **Bounds check:** `RTC_MIN_EPOCH = 1735689600` (2025-01-01) and `RTC_MAX_EPOCH = 4102444800` (2100-01-01) — match the spec's "Year 2025–2099" range.