Troubleshooting¶

Common failure modes, what causes them, and how to recover. For hardware-level diagnostics see the Hardware section.

Boot Issues¶

Symptom: No serial output at boot¶

Check baud rate — 230 400, 8N1. Not 115 200.
USB-C cable — must be data-capable (not power-only).
Try a different USB port — some hubs flake.
Unplug the main power cable — on V9, connect main power or USB-C, not both. See USB-C.

Symptom: Boot loops (repeated `=== SYSTEM STARTUP ===`)¶

Most common causes:

Cause	Indicator	Fix
`FATAL: ... ps_malloc failed`	Boot log shows the failed allocation	PSRAM is missing or mis-detected. Verify the WROOM-1 part has the `-N16R8` suffix (16 MB flash, 8 MB PSRAM). Rebuild with `PSRAMMode=enabled`
Watchdog reboot during setup	`Guru Meditation Error` or `rst:0xc` reset reason	A specific setup step is hanging. Comment out subsystems one at a time to bisect — start with `initializeHardware()`
Brownout	`rst:0xf (BROWNOUT_RST)`	Power supply can't deliver enough current during boot. Add bulk capacitance at the input
Stuck on `Cannot continue without filesystem`	User-data storage failed to mount and the automatic reformat also failed	The firmware already attempts a reformat on its own (`formatOnFail`), so reaching this halt means storage is not recoverable in place. Power-cycle once more; if it persists, contact joe@xengineering.net

Symptom: Booted but no WiFi network appears¶

First boot detection — the firmware enters CONFIG mode automatically on first boot. The AP should be at ALTERNATOR_WIFI / alternator123.
GPIO46 stuck low — also forces AP mode. Check the boot serial log; "GPIO46 LOW: OPERATIONAL AP MODE" means the pin is being held low.
GPIO45 stuck low — forces CONFIG mode regardless of saved credentials. Check for "GPIO45 LOW: FORCED CONFIGURATION MODE" in the boot log.
WiFi hardware fault — rare. Look for "WiFi connection failed" repeats in the serial log.

Forced recovery (any state)¶

This is the master table for the recovery straps. All are active-low: ground the wire (to Cable 4 pin 16, brown / GND) and hold it grounded at boot.

Function	Cable and wire color	Connector pin	ESP32 GPIO	Effect at boot
Factory firmware	Cable 4 Orange/White	RJ3 pin 9	GPIO41	Boots from the factory partition (recovers from a bad OTA image)
Wake from deep sleep	Cable 4 Orange	RJ3 pin 10	GPIO5	Wakes WiFi and the dashboard with the engine off (this strap works at any time, not only at boot)
WiFi reset / CONFIG mode	Cable 4 Green/White	RJ3 pin 11	GPIO45	Forces CONFIG mode with default AP credentials. Alternator disabled while grounded
AP/hotspot mode	Cable 4 Blue	RJ3 pin 12	GPIO46	Forces AP mode with custom credentials. Full alternator operation

Each pin also has its own dedicated solder pad / pin header on the board — see the schematic for the exact location. Full cable pinout: Data Cables & Pinout → Cable 4. User-facing procedures for the pin 11 and pin 9 straps: Connecting → Override wires.

WiFi / Dashboard Issues¶

Symptom: Dashboard won't load¶

Wrong URL — Client mode uses http://alternator.local. The regulator's own hotspot uses http://192.168.4.1. Use HTTP, not HTTPS.
mDNS not resolving — some networks block mDNS, and it is unreliable over a phone hotspot. Find the IP directly: from your router's connected-devices list, or your phone's hotspot Connected Devices list, then browse to http://<that-IP>.
Over a phone hotspot the address changes — the regulator can come back at a different 172.20.10.X after the hotspot restarts; see Reaching the dashboard over a hotspot.
Loading from the hosted site over https — a secure page can't reach a plain-http local device. Go directly to http://alternator.local or the regulator's IP.
Browser cache — hard-reload (Cmd-Shift-R / Ctrl-F5) once. The dashboard is five files served together (index.html, styles.css, script.js, and the two plotting-library files), and they are a matched set: every one is sent with Cache-Control: no-cache and an ETag, so the browser re-checks each on every load and normally gets a 304 with no re-download. A hard reload clears the case where one of the five was held back from an older bundle.
Capacitor app — see Dashboard Architecture → Connection Lifecycle.

Symptom: Regulator connects to a phone hotspot, then immediately drops¶

Budget prepaid carrier blocking it — a few low-cost prepaid plans (Visible is a confirmed one) restrict their Personal Hotspot and deauthenticate the regulator within a second or two of it associating, even when it is the only device connected. On the serial console you'll see a WiFi connect followed right away by WiFi STA disconnect reason: 2 (AUTH_EXPIRE), then CLIENT MODE FAILED — the network is kicking the regulator off, not the regulator failing to join. The workarounds are covered in WiFi Setup → Using your phone's hotspot.

Symptom: Dashboard loads but values don't update¶

Check the connection dot (left of the XEngineering wordmark) — teal means the live stream is arriving; red or orange pulsing means it is not. The colors, the nine-second silence rule, the auto-retry behavior, and the Connection Lost dialog are all covered in The connection dot.
CSV schema mismatch — open the browser console (F12). A warning like [CSV1] schema mismatch: ESP32=34, UI=33 means the firmware and dashboard versions are out of sync. Re-flash both web files (flashFactory or flashOTA) and firmware together.

Symptom: A feature described in the release notes is missing, or an export has fewer columns than expected¶

Firmware and the dashboard files live in separate flash partitions and can be updated independently, so the two can end up disagreeing about what exists.

Publish OTA updates both together. That is the path to use whenever the versions need to match.
A plain Arduino IDE or arduino-cli upload writes firmware only — the dashboard files stay at whatever was last flashed. The flashFactory / flashOTA USB aliases are the mirror image: dashboard files only, no firmware. Either one on its own leaves a mismatch.
How the mismatch shows up — a control or plot is absent, or a downloaded CSV is missing columns. In a mixed export the sections written by the regulator look current while the sections written by the page look older, because the two halves are produced by different software.
Browser cache is rarely the cause. Each dashboard file is sent with Cache-Control: no-cache and a content-hashed ETag, so browsers revalidate on every load. A hard reload (Cmd-Shift-R / Ctrl-F5) is still worth one attempt before re-flashing.
The phone app carries its own copy of the interface. It does not download the screens from the regulator, so flashing the regulator cannot change them, and force-quitting, clearing app data, or reinstalling will not either. Updating the app is the only route on that path.

Symptom: Settings save fails¶

Settings unlocked? — press Unlock Settings in the header first: the device's unlock window lasts 30 minutes and each rejected write is logged to the Console tab (see the settings arm gate). Unlock again and re-submit.
WiFi password forgotten — the two WiFi passwords are recovered differently. See Forgotten passwords below.

Forgotten passwords¶

The regulator has two passwords, both WiFi-related. (Changing dashboard settings needs no password — just the Unlock Settings button.) Recovery:

Password	What it protects	If forgotten
Ship's WiFi password (Client mode)	Joining your boat's network	Boot with GPIO45 LOW → setup page appears → enter new ship-WiFi credentials
Hotspot password (AP mode, default `alternator123`)	Joining the regulator's own WiFi	Boot with GPIO45 LOW → the setup hotspot comes up with default credentials (`ALTERNATOR_WIFI` / `alternator123`) regardless of what you customized → set a new hotspot password

Note the live dashboard stays fully readable while settings are locked, and turning the alternator off never requires unlocking — only changing settings is gated.

Charging Issues¶

Symptom: Field never engages (duty stays at 0)¶

Ignition not detected — verify wiring to the ignition input (GPIO1, optocoupler). The dashboard shows IGN status in the header.
OnOff toggle off — the master alternator-enable toggle in the header.
BMS gate — if bmsLogic == 1, the BMS input (GPIO42) must be in the active state. Check bmsLogicLevelOff polarity setting.
In Idle stage (UseFloat=0 post-absorption) — by design. Will rebulk on voltage sag or discharge current.
In Lockout — a recent safety event triggered the cooldown (FIELD_COLLAPSE_DELAY, default 30 s). Wait it out; the dashboard shows remaining seconds.
In Limp Home — LimpHome == 1 is set. Console will show "LIMP HOME MODE" every 30 s.
RPM < MinRPMForField — engine not running, or RPM sensor not reading. Check Channel 2 raw voltage on the dashboard.
Defer to Solar paused charging — currentWeatherMode == 1 means the forecast says solar is sufficient (the two-of-three-day vote is explained in Defer to Solar). The dashboard names this one directly: resting — solar forecast is strong (Defer to Solar). Switch it off in Setup → Solar to override. A forecast that can no longer be refreshed releases the pause on its own once it ages out, so this never strands the alternator off.

Symptom: Field engages but voltage doesn't reach bulk¶

uTargetAmps capped by RPM table — at low RPM the cap table is low by design. Increase the cap-table entry for your operating RPM if your alternator can sustain it.
Thermal derate — thermalPenaltyAmps > 0. The temperature loop is reducing current to protect the alternator. Look at AlternatorTemperatureF and projectedTempF.
MaxTableValue ceiling — sanity cap on the cap table. Increase only if you understand the consequences.

Symptom: Voltage overshoots target¶

Group 1/2/3 overvoltage supervisors not tuned — if the dashboard shows frequent fastOvClampCount increments and cv_I collapses, the CV loop is over-correcting. Run the Waveform Generator test to find better VoltageKp / VoltageKi values.
Load dump — g_loadDumpCount incrementing means loads are switching off and the current spike is being caught. Working as intended.
System voltage class mismatch — verify BulkVoltage is in the right range for your battery (e.g. 14.4 V on a 12 V system, not 14.4 V on a 24 V system).

Symptom: Repeated lockouts¶

Sensor disagreement — BatteryV (ADS) vs IBV (INA) differ by more than VoltageDisagreeThreshold for VoltageDisagreeTimeout. Check wiring, calibration, and noise on both inputs.
Voltage spike — AlternatorHardShutdownV exceeded. This is by design; it means the system saw a dangerous voltage. Look at the cv-log for what happened.
Temperature warning sustained — TempToUse > TemperatureLimitF + TempWarnExcess continuously for TempSustainedTimeout (default 2 min). Reduce alternator load, improve cooling, or raise TemperatureLimitF if your alternator is rated for higher.

Sensor Issues¶

Symptom: Temperature stale / 20 s cut¶

TempTask hung — Console will show "CRITICAL: TempTask hung up". Indicates Core 0 has stalled. Most often caused by I²C bus issues affecting other Core 0 work.
DS18B20 read failures — open Live Data → ESP32, and in the Alternator Temperature Sensor Health (DS18B20) panel look at tempReadFailCount, tempCrcFailCount, tempCrcRecoveredCount. Climbing CRC failures = noisy bus, marginal pull-up, or corroded crimp.
Re-enumeration retry — after an enumeration failure the task keeps retrying: every 1 s while the engine is turning (RPM ≥ 200), where the 20 s staleness cut is armed and a fast recovery matters, and every 5 s with the engine stopped, to save standby power. A hot-unplugged sensor is picked back up on the first retry after it is reconnected. (The reading cadence once the sensor is enumerated is throttled separately, and that one does depend on whether anyone is watching — every 5 s with a dashboard connected, every 60 s without.)

Symptom: ADS1115 errors¶

adsI2CErrorCount climbing — bus contention or wiring fault. Inspect cabling.
adsSlowReadCount non-zero — I2C operation took > 5 ms. Indicates bus stall. Bench-test with a logic analyzer.
ADS1115Disconnected = 1 (auto) — a failed init at boot, or 5 consecutive I²C failures at run time. There is no settings parameter that clears it: fix the wiring and reboot.

Symptom: INA228 errors¶

INA228 read failed in console — the I²C transaction threw an exception. Throttled to one per 10 s.
INADisconnected = 1 — set at boot when INA.begin() gets no answer from the chip; there is no run-time auto-disable for the INA228 and no settings parameter to clear the flag, so recovery is fix-the-wiring-and-reboot. Voltage/current freeze at last value.

Symptom: IMU disabled¶

imuEnabled = false with "IMU disabled: N I2C errors in 60s" — 10+ I²C failures in any rolling minute. Check IMU wiring and I²C bus integrity. The bus already runs at 400 kHz (in-spec Fast-mode for every chip on it), so speed is not the cause — an 800 kHz experiment was tried and reverted after the ADS1115 dropped off, and Wire.setClock(400000) in initializeHardware() carries a comment saying not to raise it again.

Shutdown and Data Retention¶

When learned data is written to flash¶

Learned data — settings, the alternator-health matrix, the boat-performance matrix, and the pending sensor-history window — is held in RAM while the engine runs and written to flash at shutdown. Flash writes stall the processor for a moment, so keeping them out of a running control loop is deliberate.

After the ignition signal drops:

The field ramps down.
As soon as the field is confirmed off, the flush runs: settings first, then the learned matrices.
If the field goes off without the ignition signal dropping — a protection cut, or the engine simply stopping — the same flush runs on a timer about half a minute later.

The regulator needs to still be powered when the flush runs. That is a matter of seconds, not minutes.

Symptom: Learned data resets after every trip¶

Main power switched off along with the engine. The regulator is designed with Power connected permanently and Ignition as the switching input — that is what lets the flush complete after the key is turned off, and what keeps the dashboard reachable at anchor (about 20 mA at 12 V idle — see How much power it actually uses). Putting the regulator's supply on a switch that gets killed at the same moment as the ignition, or opening a battery isolator immediately after shutdown, cuts power before step 2 above finishes. Settings and everything learned since the last successful flush are lost.
What it looks like — the alternator-health matrix never accumulates points across trips, the engine-hours trend stays empty, and settings revert to their previous values.
The fix is wiring, not configuration. Feed the regulator from a permanently live supply and use the ignition input to tell it when the engine is running. If a disconnect switch has to stay in the supply, leave it closed for several seconds after shutdown.
A hard power cut mid-run is not fatal to the record — the matrices revert to the last completed flush rather than emptying. Only what was learned since then is lost.

The This Session plot starts fresh each run¶

The This Session graph covers the current engine run and is not carried across shutdowns by design. Within a run it is recoverable: the regulator keeps the last two hours of session points itself, so opening the dashboard part-way through a run, reloading the page, or reconnecting after a WiFi dropout refills the graph rather than starting from blank. Gaps longer than two hours come back partially, with a break drawn where the missing stretch was.

The alternator-health matrix, the engine-hours trend, and the boat-performance data are separate from this and do persist across shutdowns, subject to the flush above.

Crash / Reset Investigation¶

The regulator restarts itself on purpose¶

The regulator restarts itself opportunistically, picking the quietest moment it can find. This is normal maintenance, not a fault: like any long-running computer, restarting periodically keeps its memory tidy and is the cheapest insurance against slow degradation over weeks of continuous running.

It gets less picky the longer it has been running:

Uptime	Restarts when	You see
After 1 day	Everything is idle: engine off, not charging, no one connected, battery healthy, cloud uploads finished	Nothing
After 2 days	Engine off and not charging. Being connected no longer holds it off	10-minute countdown banner
After 3 days	No matter what — even under way and charging. Charging is switched off first	10-minute countdown banner

The restart itself takes about 30 seconds. Even in the last case, a low battery, a running guided test, or a firmware update in progress will still postpone it.

Nothing is lost across a maintenance restart: settings, history, statistics, and any telemetry not yet uploaded are all saved to flash first, and the regulator returns to whatever operating mode it was in. Under Live Data → ESP32 this shows up as reset reason "Scheduled maintenance restart" — it is the most common reset reason on a healthy system.

Reading the reset reason¶

Live Data → ESP32 shows LastResetReason in its Session Info panel — and, beside it, ancientResetReason, the reset before that one. Both are stored as small integer codes and rendered by the dashboard as plain-English labels. The ones that matter:

Label shown	Meaning
Power-on (plugged in)	Normal cold boot
Software reset (unscheduled)	Firmware called `ESP.restart()` (factory reset, post-OTA)
Scheduled maintenance restart	A restart the firmware planned — distinguished from the unscheduled case by a flag file written before rebooting
Task watchdog (loop blocked) / Interrupt watchdog / Other watchdog	Watchdog timeout — the main loop (Core 1) hung
Panic/Exception (crash)	Crash — see `coredump` partition; this is the case that prints a backtrace
CPU lockup (double exception)	Fault while already handling a fault
Brownout (power issue) / Power glitch (supply transient)	Supply voltage dipped or glitched
External reset (button) / USB reset / JTAG reset	Reset asserted from outside the firmware

An unrecognized code displays as "Unknown reset". The full code-to-label map is resetReasonLookup in web_src/script.js; the firmware side is captureResetReason() in 5_functions.ino, which also records the raw ESP-level and per-CPU ROM reset codes for cases the plain-English label collapses.

Decoding a crash backtrace¶

When the device prints Guru Meditation Error followed by a backtrace, decode it with the ESP32 toolchain's address-to-source tool (xtensa-esp32s3-elf-addr2line, installed with the board package) against the .elf file from your build. Feed it the 0x42...... application addresses from the backtrace; it prints the source file and function for each.

Cloud / OTA Issues¶

Symptom: Cloud upload says "queued" but never completes¶

WiFi RSSI — uploads gate on RSSI ≥ −80 dBm. Weak signal blocks uploads. (OTA firmware downloads use a deliberately looser bar, −90 dBm / OTA_MIN_RSSI_DBM: signal strength was never the thing that made a download fail, so the gate was widened and the timeouts lengthened instead.)
fieldOffSettled(10 s) gate — the field must be off for 70 s before uploads run; cycling the engine on/off resets the timer. See Advanced Features → Upload path.
Backoff active — after 5 consecutive failures, uploads suspend for 30 s. Console shows "Cloud sync paused, too many failures."

Symptom: OTA update fails¶

Signature verification fail — wrong bundle for this device, or the public key in firmware doesn't match the signing private key. Check OTA_PUBLIC_KEY in Xregulator.ino against the build pipeline.
Heap integrity fail — system is in a degraded state. Boot from factory partition (GPIO41 LOW) to clear, then retry.
Mid-download disconnect — automatic rollback. Factory partition keeps running.

Recovery from a bad OTA image¶

Power off the regulator.
Ground GPIO41 (Cable 4 Orange/White, RJ3 pin 9 — see the strap table).
Power on. Boot log will show "We are in factory partition."
Connect to the dashboard normally and install a known-good version from the Cloud Features → Software Update tab.

Last-Resort Recovery¶

Factory reset (preserves only the cloud registration token)¶

Via dashboard — Setup → System → Security & Maintenance → Erase All Memory (the button reads "Restore Defaults"; settings must be unlocked, plus a confirm dialog). Reformats the LittleFS data partition (logs, history rings, buffered uploads), erases every NVS namespace — which is where user settings, the vessel description, and lifetime counters actually live — and restarts. Defaults are re-created on the next boot, not during the reset itself, so nothing carries the old values back into flash.
The one deliberate exception — the cloud registration token is deliberately preserved, so your cloud account, history, and leaderboard entries survive; the token lifecycle, including the power-loss caveat during the wipe, is in Cloud Account Management. To delete the cloud account too, press Delete My Account first.
Via GPIO45 boot — enters CONFIG mode with default credentials so you can at least reach the device, but does NOT wipe settings. Use to recover from a lost WiFi password without losing data.

Re-flashing firmware over USB¶

If OTA is broken and GPIO41 boot doesn't help, fall back to USB-C:

Connect USB-C.
Hold GPIO0 LOW, press Reset, release GPIO0. The chip enters bootloader mode.
Upload the sketch from the Arduino IDE — see Building the Firmware. (flashFactory / flashOTA do not re-flash firmware: they only rebuild the compressed web bundle and write it to the factory_fs / prod_fs filesystem partitions.)

An Arduino IDE upload writes the factory application slot only — user settings survive because the LittleFS and NVS partitions are left alone. Since a device that has taken an over-the-air update boots from ota_0, USB-flashed code does not run until you force the factory partition with the GPIO41 strap (see the strap table and OTA Updates → note for contributors).

Getting more help¶

GitHub Issues — github.com/markliquid1/Regulator2026-public/issues for bug reports and feature requests.
Console output — the dashboard's Console tab is the live serial-equivalent log. Save it before reporting an issue.
ESP32 panel — Live Data → ESP32 shows error counters and runtime diagnostics. Include them in bug reports.