Skip to content

Configuration & Development

Everything you need to build, flash, debug, and configure the firmware. The user-tunable runtime parameters themselves are documented next to the subsystem that consumes them — see Safeties, Field Control, Battery Management, Advanced Features.

Development Environment

Tool Version / notes
Arduino IDE 2.x with ESP32 board package
Board (FQBN) esp32:esp32:esp32s3:FlashSize=16M,PartitionScheme=custom,CPUFreq=240,PSRAMMode=enabled
Custom partition scheme partitions.csv in the sketch folder (System Overview → Partition Map)
PSRAM Required. All large allocations use ps_malloc(). The board must report Found <N> MB OPI PSRAM at boot
Loop task stack SET_LOOP_TASK_STACK_SIZE(20 * 1024) — 20 KB, bumped from default 8 KB for TLS handshakes

Required libraries

  • ADS1115_lite — 4-channel ADC
  • INA228 — battery voltage / current monitor
  • VeDirectFrameHandler — Victron VE.Direct
  • NMEA2000_esp32, N2kMessages, N2kMessagesEnumToStr — Marine CAN
  • OneWire, DallasTemperature — DS18B20
  • BMP388_DEV — barometric pressure + ambient temp. Do not upgrade — local patched version with better error handling
  • LSM6DSOXSensor — 6-axis IMU
  • PID_v1_xeng — fork of Brett Beauregard's PID library with tracking anti-windup additions
  • AsyncTCP, ESPAsyncWebServer — do not upgrade past the mathieucarbou fork specified in the project README. The upstream ESPAsyncWebServer is incompatible with this codebase
  • ArduinoJson, HTTPClient, WiFiClientSecure — networking
  • TinyGPSPlus — NMEA0183 (currently included but parser not wired up)
  • ESP-IDF native: esp_heap_caps, esp_task_wdt, mbedtls, nvs_flash, esp_ota_ops, esp_partition, esp_psram, esp_system

Build / Flash Workflow

Shell functions in ~/.zshrc provide the canonical build pipeline.

Alias Steps
flashFactory ./compress_web.sh → mklittlefs → esptool.py to factory web partition (0x510000)
flashOTA Same → esptool.py to production web partition (0x610000)
build_and_deploy.sh <version> "<notes>" Full OTA pipeline: compress_web.sh → arduino-cli compile → tar bundle → SHA-256 hash → mbedTLS RSA sign → upload to the public Supabase Storage bucket (ota) via the Supabase CLI → versions.json update

Firmware (.ino → .bin) is uploaded separately via the Arduino IDE upload button. Always flash firmware before LittleFS — the firmware tells ensureLittleFS() whether to mount factory or production web partition, and the layout has to be consistent.

compress_web.sh

web_src/         (edit here)
    ├── index.html
    ├── script.js
    ├── styles.css
    ├── uPlot.iife.min.js
    └── uPlot.min.css
        ↓
    compress_web.sh
        ↓
data/            (auto-generated; gzipped; never edit directly)
    ├── index.html.gz
    ├── script.js.gz
    └── …

The AsyncWebServer onNotFound handler checks for path.gz before falling back to plain serving. Browser receives Content-Encoding: gzip and decompresses transparently. HTML references like <script src="script.js"> work because the server rewrites them.

Never edit data/ directly — compress_web.sh will overwrite it on the next flash.

Settings Storage — When to Use What

Two storage backends. The choice is determined by write cadence, not value type.

Backend Write trigger Wear pattern Block time
LittleFS (<setting>.txt, one file each) User form submission Single user-rate write per setting ~150 ms file write at field-off-edge gate
NVS (storage, learning, update_req, others) Loop-driven (extrema, counters, accumulators) Hundreds of writes per session — needs wear-leveling ~300 ms sector erase (commit cost)

Rule of thumb: if the variable changes only when the user touches a form, store it in LittleFS. If it can change every tick (peak loop time, totalPowerCycles, IMU stats, voltage extrema, CoulombCount), it belongs in NVS.

LittleFS settings — file-per-parameter pattern

void InitSystemSettings() {
    // …
    if (!fsExists("/BulkVoltage.txt")) {
        writeFile(LittleFS, "/BulkVoltage.txt", String(BulkVoltage).c_str());
    } else {
        BulkVoltage = readFile(LittleFS, "/BulkVoltage.txt").toFloat();
    }
    // … one block per setting
}

Pros: trivially inspectable via mklittlefs or the regulator's diagnostic dump. A user with serial console can cat /BulkVoltage.txt. No serialization layer — text files all the way down.

Cons: Each setting takes ~1 KB minimum on LittleFS (sector size). Doesn't matter — the userdata partition has 8.88 MB and there are only ~150 settings.

NVS — change-detection + field-off commit

// initNVSCache() seeds prev_* shadows at boot from current NVS values.
// saveNVSDataFull() commits only changed values:
if (CoulombCount_Ah_scaled != prev_CoulombCount_Ah_scaled) {
    nvs_set_i32(handle, "CoulombCount", CoulombCount_Ah_scaled);
    prev_CoulombCount_Ah_scaled = CoulombCount_Ah_scaled;
    dirty = true;
}
// … one block per variable
if (dirty) nvs_commit(handle);

The commit is the expensive part. The change-detection shadow pattern means a saveNVSDataFull() call where nothing actually changed costs effectively nothing — the commit doesn't fire.

saveNVSDataFull() runs only at:

  • Field-off edge + 5 s — fieldOffFlushDone in loop().
  • Shutdown phase 2 — once, immediately after the field cut on ignition-off.
  • A few one-shot moments — user-pressed factory reset, post-OTA boot validation.

Periodic in-loop NVS commits were removed because they collided with the 100 ms CV control loop. See plan1_storage_migration.md for the full migration history.

Critical zone — inCriticalZone() / safeToFlushIO()

bool inCriticalZone() {
    return (millis() - lastElectricalRecordMs) < 5000;
}
bool safeToFlushIO() {
    static uint32_t safeSince = 0;
    if (inCriticalZone()) { safeSince = 0; return false; }
    if (safeSince == 0) safeSince = millis();
    return (millis() - safeSince) >= 5000;  // 5 consecutive seconds safe
}

lastElectricalRecordMs is bumped every time a new extreme is recorded (voltage record, current record, etc.). The 5-second-clean requirement before safeToFlushIO() returns true keeps NVS and LittleFS writes from landing during electrically interesting moments — even if the field is technically off in steady state.

The deferred-I/O drain block in loop() is the only consumer of safeToFlushIO(). It empties any pendingSave* / pendingClear* flags set by Core-0 SSE button handlers.

Adding a New Variable

Full recipe: Important Functions → Three Variable Patterns. Quick summary:

Pattern What's needed
A1 — High-rate live telemetry (CSV1) global decl + payload1 snprintf + JS index map
A2 — Slower telemetry / diagnostic (CSV2) global decl + payload2 snprintf + JS index map
B — User-configurable setting (CSV3 + LittleFS) global decl + /get handler + boot init + payload3 + JS echo + HTML form row

Adding a setting that auto-fires every loop (extrema, accumulator)? Use NVS, not LittleFS — and add the change-detection shadow. See saveNVSData*() for examples.

Recovery Modes — GPIO-Triggered Boot

Three special boot states triggered by grounding specific GPIO pins during power-on.

Pin Effect
GPIO41 LOW at boot Boot from factory partition (factory) instead of ota_0. Use to recover from a bad OTA
GPIO45 LOW at boot Force configuration mode — AP up with default credentials (ALTERNATOR_WIFI / alternator123), captive portal active. Alternator operation disabled. Used for password recovery
GPIO46 LOW at boot Operational AP mode — full alternator interface served from the device's own WiFi. CloudFeatures = 0. Used when ship WiFi is down

GPIO41 acts at the bootloader level (factory bootloader checks the strap). GPIO45 and GPIO46 are checked in firmware inside setupWiFi(). Pull-ups are internal — strap to ground for "active."

Factory reset (web button)

POST /factoryReset with password → performDeepFactoryReset(). Unmounts LittleFS, reformats the user partition, erases the entire NVS namespace, reinitializes both, then calls ESP.restart(). Total wipe — every saved setting is gone after this.

There is no "factory reset via GPIO" — recovery from a non-bootable image goes through GPIO41 (factory partition) instead.

Backtrace Decoding

When the watchdog or panic handler reboots the device, ESP-IDF prints a backtrace like:

Guru Meditation Error: Core 1 panic'ed (LoadProhibited).
Backtrace: 0x420aabcc:0x3fcdf800 0x420ab0e8:0x3fcdf820 …

To map addresses back to source lines:

# Find the current ELF (cached by Arduino IDE)
find ~/Library/Caches/arduino -name "*.elf" -newer ~/Documents/Arduino/Xregulator/Xregulator.ino | head -3

# Decode the application-code addresses (0x42xxxxxx range — ignore 0x403xxxxx ROM/SDK addresses)
/Users/joeceo/Library/Arduino15/packages/esp32/tools/esp-x32/2601/bin/xtensa-esp32s3-elf-addr2line \
    -pfiaC \
    -e /Users/joeceo/Library/Caches/arduino/sketches/<hash>/Xregulator.ino.elf \
    0x420aabcc 0x420ab0e8 …

Strip everything after the colon in each backtrace pair (the :0x3fcdxxxx is the stack pointer, not an address). Common decode results:

  • __sfp → fopen → VFSImpl::exists at the top of the stack means FILE* pool exhaustion in newlib — check the fs mutex isn't being held across long operations.
  • A 4_functions.ino:<line> near the top with mbedtls_* below it means a TLS-related stack overflow — verify the loop stack is at 20 KB.

Console Debug Endpoints

Endpoint What
GET /debug Live snapshot of internal state (varies per build)
GET /debugToken Current Supabase JWT (for cloud-side debugging)
GET /getAuthToken Same
printPartitionInfo() (boot serial only) Dump of every partition's type, subtype, offset, size
printTempDebugStatus() (every 5 min serial only) DS18B20 success / failure counter dump
printEffDiagnostics() (every 30 s — currently commented out, easy to re-enable) Efficiency tracker rejection breakdown
Serial console at 230400 baud All Serial.print*() output. Mostly silent in steady state; verbose during boot, OTA, faults

Reset Reasons

Captured at boot by captureResetReason(). Visible on the Stats panel.

esp_reset_reason_t What it means
ESP_RST_POWERON Cold start / power applied
ESP_RST_EXT External reset pin asserted
ESP_RST_SW ESP.restart() called from firmware
ESP_RST_PANIC Crash / abort() — check coredump partition
ESP_RST_INT_WDT, ESP_RST_TASK_WDT Watchdog timeout
ESP_RST_BROWNOUT Supply voltage below brownout detector threshold
ESP_RST_DEEPSLEEP Wakeup from deep sleep (not currently used)

Crash dumps land in the coredump partition (64 KB). Use esp_core_dump_image_get() paths from ESP-IDF tooling to extract them.

CSV Payload Integrity Verification

Mandatory after any CSV change. Format-spec count must equal <FIELD_COUNT> + 1 (the +1 is the count field itself). Browser silently drops the entire packet on mismatch.

# Replace STREAM and EXPECTED_COUNT for the channel changed.
STREAM=payload2Len; EXPECTED_COUNT='CSV2_FIELD_COUNT'
grep -A 200 "int ${STREAM} = snprintf" /Users/joeceo/Documents/Arduino/Xregulator/3_functions.ino | \
  awk '/snprintf|"%/ {for(i=1;i<=NF;i++){n=split($i,a,"%d"); count+=(n-1)}} /\);/{exit} END{print "Format specifiers:", count, "(should be " ENVIRON["EXPECTED_COUNT"] " + 1)"}' EXPECTED_COUNT=$EXPECTED_COUNT

The browser side: script.js arrays CSV1_FIELDS, CSV2_FIELDS, CSV3_FIELDS, TS_FIELDS — their .length must equal the declared count.

Function Timing — How Performance Data is Collected

Every long-running function in loop() is wrapped in TIMED_CALL(ft_xxx, fn()):

#define TIMED_CALL(ft, call) \
    do { \
        uint32_t _t0 = (uint32_t)esp_timer_get_time(); \
        call; \
        uint32_t _dt = (uint32_t)esp_timer_get_time() - _t0; \
        (ft).lastCall = _dt; \
        if (_dt > (ft).worstWindow) (ft).worstWindow = _dt; \
        if (_dt > (ft).worstSession) (ft).worstSession = _dt; \
    } while (0)

FuncTiming structs (ft_*) live as globals. The Function Timing table in the Stats panel reads them via CSV2. Window-worst resets every 5 s; session-worst resets only when the user presses "Reset Peak Values." See Important Functions → Timed Function Pattern for the full recipe.

TIMED_CALL is a do-while macro because it needs to execute the wrapped call in the surrounding scope — the function it wraps may need access to surrounding variables.

Top-Level Constants You Probably Want to Find

Constant Definition site Meaning
FIRMWARE_VERSION Xregulator.ino:167 Version string parsed at boot into firmwareVersionInt
webgaugesinterval Xregulator.ino:3213 CSV1 cadence (ms; default 100)
WeatherUpdateInterval LittleFS-persisted Weather fetch interval (typically 1 h)
SENSOR_UPLOAD_INTERVAL Xregulator.ino:619 Local sensor snapshot cadence (30 s for testing; 5 min production)
BUFFER_UPLOAD_INTERVAL Xregulator.ino:620 Cloud upload retry cadence (13 s)
CONFIG_SNAPSHOT_INTERVAL Xregulator.ino:617 Cloud config-snapshot cadence (40 min)
FIELD_COLLAPSE_DELAY Xregulator.ino:1712 Lockout cooldown (30 s)
TEMP_TASK_TIMEOUT Xregulator.ino:763 TempTask heartbeat deadline (20 s)
WIFI_WAKE_DURATION Xregulator.ino:3207 Wake-button window (5 min)
INA_FAST_INTERVAL_MS / INA_SLOW_INTERVAL_MS Xregulator.ino:1606-1607 INA228 cadence in fast/slow mode
MAX_DATA_INDICES Xregulator.ino:3104 Sentinel for DataIndex enum (35)
RPM_TABLE_SIZE Xregulator.ino:2500 10
PAYLOAD_BUFFER_SIZE, CONFIG_PAYLOAD_SIZE Xregulator.ino:1138-1139 4096, 8192 — PSRAM payload caps
SENSOR_RING_SIZE Xregulator.ino:1324 1000 (≈ 83 h at 5 min/sample)

Cross-references