Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions README-repeated-sending.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Repeated Sending (branch `feature/repeated-sending-2`)

Reliable delivery of direct-routed packets via **cancellable retransmissions**. The feature augments DIRECT/TRACE routing with an automatic, self-limiting retransmission that switches itself off one hop downstream as soon as successful forwarding is detected.

## Problem

A direct-routed packet (DIRECT / TRACE) that travels to its destination via one or more relays is considered "sent" by the originator as soon as it is on air. If it is lost on the first hop (collision, interference, fading), there is no retry at the MeshCore layer — the packet is gone, and the application layer must re-schedule the entire message. Flood traffic does not have this problem because it is widely broadcast anyway; DIRECT packets, however, are the cost-conscious path for targeted messages.

## Solution

After every DIRECT packet is sent, the Dispatcher schedules a retransmission. The key trick: **the originator listens for a downstream relay forwarding the packet**. As soon as it receives that forward (the "forwarding echo"), it knows its packet survived at least the next hop and cancels the pending retransmission. The result is an error-correction mechanism that produces **no** extra packet in the success case, and in the failure case retransmits precisely where the packet actually got stuck.

Flow in detail:

1. **Schedule the resend** – `Dispatcher::resendPacket()` runs after TX only for DIRECT packets with at least one relay hash in the path (`getPathHashCount() > 0`). It does not wait for a timeout; instead it re-queues the packet after a computed **cancel window** `W = C0 + K·airtime + margin + (attempt−1)·jitter`, capped at an airtime-proportional ceiling (`CAP_BASE + CAP_AIRTIME_X·airtime`). The window always outlasts the downstream forward for any packet length or spreading factor (a flat ceiling under-covered SF10 long messages, so every long packet fired a redundant, colliding resend), while staying as short as the airtime allows so the echo arrives before the resend fires.
2. **Detect the forward and cancel** – In `Mesh::onRecvPacket()`, every received DIRECT packet is checked: is it a downstream forward of one of our own pending resends (`Packet::isRetryMatch()`)? If so, the resend is removed from the outbound queue (`removeOutboundByIdx`) — the current `outbound` packet and every queued entry alike. A deterministic per-hop busy-recheck bucket (`pathHashCount % 3`) keeps deferred resenders of the same chain from re-checking the channel on the same loop tick.
3. **Non-invasive LBT** – Resends use *no* hardware CAD, but `isResendChannelActive()`: a pure IRQ/register read (preamble detection + RSSI margin above the noise floor). RX stays open so the forwarding echo is not missed. First sends still use normal CAD (collision avoidance is worth the momentary deafness).

### TRACE handling

TRACE packets append an SNR byte per hop, which changes their hash. `isRetryMatch()` therefore compares payload + SNR-path prefix instead of the hash for TRACE. Because the TRACE branch in `onRecvPacket()` always returns, the generic echo-cancel block is never reached for TRACE — the TRACE branch therefore performs its own cancel scan: when a later hop's forward of a TRACE we already sent is overheard, our queued TRACE resend is dropped (for well-formed, non-terminal legs only). Without this, every intermediate hop would re-send each TRACE up to `max_resend_attempts` times even though the next hop had already forwarded it.

At the final forwarding hop (destination does not forward) the packet is flagged `no_retry` and sent once through the normal TX gate — a retry here could never be cancelled by overhearing a forward, and pre-setting the attempt counter (the earlier approach) made the first send look like a resend: it went through the resend LBT gate instead of the normal channel check and was counted in the resend statistics.

### Final-hop handling

On the final relay hop there is no downstream forward to overhear — the destination does not forward. For **TXT_MSG with ACK** the destination acknowledges receipt with an ACK. This hop is allowed exactly one ACK-cancellable resend (flag `final_hop_ack_resend`). `Mesh::cancelPendingFinalHopResend()` cancels the oldest final-hop resend still sitting in the queue as soon as the ACK returns (FIFO, since ACKs are produced in delivery order). A resend already on air is deliberately not aborted (the destination dedups the duplicate via `wasSeen`).

## Configuration

| Location | Value | Meaning |
|---|---|---|
| CLI `set max.resend <0–3>` / `get max.resend` | Default `2` | Maximum resend attempts for DIRECT packets. `0` disables the feature entirely. `get` also reports the resend ratio, e.g. `2, resends 64/173 (37%)`. Available on CommonCLI nodes (repeater/chat/room-server) **and on companion radios** via their USB/BLE CLI (`CMD_RUN_CLI_COMMAND` passthrough) — identical commands and reply format everywhere. |
| `NodePrefs` (persisted) | `max_resend_attempts` | Saved/loaded via the ConfigSerializer prefs file; sanitised to 0–3. (The legacy binary migration path reads it at byte offset 295.) |
| Companion-radio `CMD_SET_*` frame | Byte 6 in the path block | App-protocol path to set `max_resend_attempts` from the companion. |
| `RESEND_INTERFERENCE_MARGIN` (compile-time) | Default `12` dB | Margin above the noise floor at which a resend is blocked (non-invasive LBT). |
| `RESEND_DEFERRAL_*` (compile-time) | `C0 1500`, `K 2`, `margin 40`, `cap 1500 + 3·airtime`, `jitter 100` | Cancel-window model, in milliseconds/airtime multiples (see `src/Dispatcher.h`). |

## Implementation (core files)

- `src/Dispatcher.cpp` / `.h` – `resendPacket()`, `resendWindowFor()`, `isResendChannelActive()`, `getMaxResendAttempts()`, pool-shedding skip, resend statistics.
- `src/Mesh.cpp` / `.h` – forwarding detection and resend cancellation in `onRecvPacket()` (generic DIRECT block + TRACE branch scan), `cancelPendingFinalHopResend()`, final-hop marking (`final_hop_ack_resend`, `no_retry`).
- `src/Packet.cpp` / `.h` – cached packet hash (`hash`, `hash_hex`), `isRetryMatch()` (TRACE-specific vs. hash-based), fields `sending_attempts` / `no_retry` / `final_hop_ack_resend`.
- `src/helpers/SimpleMeshTables.h` – dedicated ACK dedup table (`_acks[]`, via `ack_crc`), so multi-ACK/resend duplicates do not evict the flood-dedup entries.
- `src/helpers/StaticPoolPacketManager.*` – `peek()` / `peekNextOutbound()` (non-consuming) and reset of `sending_attempts` / `no_retry` / `final_hop_ack_resend` in `free()` (see below).
- `src/helpers/CommonCLI.*`, `docs/cli_commands.md`, `examples/*/MyMesh.*`, `NodePrefs.h` – config knobs and persistence.

## Note: pool-reuse fix

`sending_attempts` and `final_hop_ack_resend` were originally zeroed only in the `Packet` constructor, **not** when the packet was returned to the pool (`free()`). A recycled pool slot thus kept a stale `sending_attempts >= getMaxResendAttempts()`, causing `resendPacket()` to skip the resend for every subsequent DIRECT packet routed through that slot. Symptom on hardware: resends work right after boot (fresh pool) but degrade to zero once the pool has turned over (~`pool_size` sends) — forwards kept working because they do not check `sending_attempts`. `StaticPoolPacketManager::free()` now resets the cached hash and all resend fields in that chokepoint.

## Status & scope

- Rebased on `upstream/dev`; three commits: simulator hooks, the resend feature, and the TRACE resend fixes (found by @Kernel-Error, measured by @m0urs — see m0urs/MeshCore#25, m0urs/MeshCore#26).
- Applies exclusively to **DIRECT/TRACE-routed** traffic. Flood traffic is unaffected.
- Only active when `max.resend > 0` (default `2`).
12 changes: 12 additions & 0 deletions docs/cli_commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -515,6 +515,18 @@ This document provides an overview of CLI commands that can be sent to MeshCore

---

#### View or change the maximum direct-route resend attempts
**Usage:**
- `get max.resend`
- `set max.resend <value>`

**Parameters:**
- `value`: Maximum number of resend attempts for direct-routed packets (0–3). `0` disables resending entirely.

**Default:** `2`

---

#### View or change the retransmit delay factor for flood traffic
**Usage:**
- `get txdelay`
Expand Down
1 change: 1 addition & 0 deletions examples/companion_radio/DataStore.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,7 @@ void DataStore::loadPrefsInt(const char *filename, NodePrefs& _prefs) {
file.read((uint8_t *)&_prefs.rx_boosted_gain, sizeof(_prefs.rx_boosted_gain)); // 89
file.read((uint8_t *)_prefs.default_scope_name, sizeof(_prefs.default_scope_name)); // 90
file.read((uint8_t *)_prefs.default_scope_key, sizeof(_prefs.default_scope_key)); // 121
file.read((uint8_t *)&_prefs.max_resend_attempts, sizeof(_prefs.max_resend_attempts)); // 137

// migrate old fields
_prefs.setRepeatEn(_prefs._client_repeat != 0);
Expand Down
53 changes: 53 additions & 0 deletions examples/companion_radio/MyMesh.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
#include <WiFi.h>
#endif
#include <Mesh.h>
#include <helpers/StatsFormatHelper.h>

#define CMD_APP_START 1
#define CMD_SEND_TXT_MSG 2
Expand Down Expand Up @@ -111,6 +112,22 @@
#define DIRECT_SEND_PERHOP_EXTRA_MILLIS 250
#define LAZY_CONTACTS_WRITE_DELAY 5000

// Fixed RSSI margin (dB above the noise floor) at which a DIRECT resend treats the channel as busy
// (isResendChannelActive(): busy when isReceivingPacket() || currentRSSI - noiseFloor >= this). It is
// the resend-only counterpart of the general-LBT _prefs.interference_threshold, which flows through
// getInterferenceThreshold() -> triggerNoiseFloorCalibrate() -> RadioLibWrapper::isChannelActive().
// They are separate today only because interference_threshold is blanket-initialised to 0 (disabled;
// see getInterferenceThreshold(), cited there as a currentRSSI() reliability concern). The margin
// currentRSSI-noiseFloor is only as trustworthy as both terms: the median noise-floor estimator
// (PR #2933) stabilises getNoiseFloor() and removes the dominant drift source (the old one-way
// ratchet to -120 that inflated every margin), substantially rehabilitating the threshold path.
// Once interference_threshold is enabled across the fleet, this constant can be superseded by it.
// NB: here getInterferenceThreshold() is hardcoded to 0 (not prefs-driven), so superseding this
// constant via interference_threshold additionally requires dropping that override.
#ifndef RESEND_INTERFERENCE_MARGIN
#define RESEND_INTERFERENCE_MARGIN 12 // dB above noise floor that blocks a resend (non-invasive LBT)
#endif

#define PUBLIC_GROUP_PSK "izOH6cXN6mrJ5e26oRXNcg=="

// these are _pushed_ to client app at any time
Expand Down Expand Up @@ -270,6 +287,15 @@ bool MyMesh::getCADEnabled() const {
return _prefs.cad_enabled;
}

bool MyMesh::isResendChannelActive() {
// Non-invasive resend LBT (NO CAD, so RX stays open to overhear the downstream forward
// and cancel the resend). Block the resend when a LoRa preamble/header is being received
// (often that forward itself) OR the live channel energy is well above the noise floor
// (foreign interference). isReceivingPacket()/getCurrentRSSI() are IRQ/register reads.
int margin = (int)radio_driver.getCurrentRSSI() - _radio->getNoiseFloor();
return radio_driver.isReceivingPacket() || (margin >= RESEND_INTERFERENCE_MARGIN);
}

int MyMesh::calcRxDelay(float score, uint32_t air_time) const {
if (_prefs.rx_delay_base <= 0.0f) return 0;
return (int)((pow(_prefs.rx_delay_base, 0.85f - score) - 1.0) * air_time);
Expand All @@ -279,6 +305,7 @@ uint32_t MyMesh::getRetransmitDelay(const mesh::Packet *packet) {
uint32_t t = (_radio->getEstAirtimeFor(packet->getPathByteLen() + packet->payload_len + 2) * _prefs.tx_delay_factor);
return getRNG()->nextInt(0, 5*t + 1);
}

uint32_t MyMesh::getDirectRetransmitDelay(const mesh::Packet *packet) {
uint32_t t = (_radio->getEstAirtimeFor(packet->getPathByteLen() + packet->payload_len + 2) * _prefs.direct_tx_delay_factor);
return getRNG()->nextInt(0, 5*t + 1);
Expand Down Expand Up @@ -925,6 +952,7 @@ MyMesh::MyMesh(mesh::Radio &radio, mesh::RNG &rng, mesh::RTCClock &rtc, SimpleMe
_prefs.tx_power_dbm = LORA_TX_POWER;
_prefs.gps_enabled = 0; // GPS disabled by default
_prefs.gps_interval = 0; // No automatic GPS updates by default
_prefs.max_resend_attempts = 2;
_prefs.radio_fem_rxgain = 1;
#ifdef RADIO_FEM_TXGAIN
_prefs.radio_fem_txgain = RADIO_FEM_TXGAIN; // board-specific default for PA
Expand Down Expand Up @@ -995,6 +1023,7 @@ void MyMesh::begin() {
_prefs.gps_interval = constrain(_prefs.gps_interval, 0, 86400); // Max 24 hours
_prefs.adc_multiplier = constrain(_prefs.adc_multiplier, 0.0f, 10.0f);
if (!board.setAdcMultiplier(_prefs.adc_multiplier)) _prefs.adc_multiplier = 0.0f;
_prefs.max_resend_attempts = constrain(_prefs.max_resend_attempts, 0, 3);

_active_ble_pin = _prefs.ble_pin;

Expand Down Expand Up @@ -1500,6 +1529,9 @@ void MyMesh::handleCmdFrame(size_t len) {
_prefs.advert_loc_policy = cmd_frame[3];
if (len >= 5) {
_prefs.multi_acks = cmd_frame[4];
if (len >= 6) {
_prefs.max_resend_attempts = constrain(cmd_frame[5], 0, 3);
}
}
}
}
Expand Down Expand Up @@ -2131,6 +2163,27 @@ bool MyMesh::handleCommand(const char* command, uint32_t sender_timestamp, char*
return true;
}

// repeated sending (feature): same command and reply format as the CommonCLI nodes
// (repeaters/chat), so the app's CLI passthrough behaves identically everywhere.
if (memcmp(command, "set max.resend ", 15) == 0) {
int v = atoi(&command[15]);
if (v < 0 || v > 3) {
strcpy(reply, "ERROR: max.resend must be 0-3");
} else {
_prefs.max_resend_attempts = (uint8_t)v;
savePrefs();
strcpy(reply, "OK");
}
return true;
}
if (strcmp(command, "get max.resend") == 0) {
sprintf(reply, "> %d", (uint32_t)_prefs.max_resend_attempts);
if (_prefs.max_resend_attempts) { // plain "> 0" when resends are disabled
StatsFormatHelper::formatResendRatio(reply, getNumResentDirect(), getNumSentDirect());
}
return true;
}

#ifdef PIN_BUZZER
if (strcmp(command, "get buzzer") == 0) {
sprintf(reply, "> %s", _prefs.buzzer_quiet ? "off" : "on");
Expand Down
2 changes: 2 additions & 0 deletions examples/companion_radio/MyMesh.h
Original file line number Diff line number Diff line change
Expand Up @@ -130,13 +130,15 @@ class MyMesh : public BaseChatMesh, public DataStoreHost {
float getAirtimeBudgetFactor() const override;
int getInterferenceThreshold() const override;
bool getCADEnabled() const override;
bool isResendChannelActive() override;
int getAGCResetInterval() const override {
return ((int)_prefs.agc_reset_interval) * 4000; // milliseconds
}
int calcRxDelay(float score, uint32_t air_time) const override;
uint32_t getRetransmitDelay(const mesh::Packet *packet) override;
uint32_t getDirectRetransmitDelay(const mesh::Packet *packet) override;
uint8_t getExtraAckTransmitCount() const override;
uint8_t getMaxResendAttempts() const override { return _prefs.max_resend_attempts; }
bool filterRecvFloodPacket(mesh::Packet* packet) override;
bool allowPacketForward(const mesh::Packet* packet) override;

Expand Down
2 changes: 2 additions & 0 deletions examples/companion_radio/NodePrefs.h
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ class NodePrefs : public ConfigSerializer { // persisted to file
uint8_t default_scope_key[16];
float tz_offset = 0;
float adc_multiplier = 0; // battery ADC multiplier, 0 = board default
uint8_t max_resend_attempts; // 0 = disabled, 1-3, default 2 (repeated sending)
#ifdef ENABLE_WIFI_INTERFACE
#ifndef WIFI_SSID
#define WIFI_SSID ""
Expand Down Expand Up @@ -287,6 +288,7 @@ class NodePrefs : public ConfigSerializer { // persisted to file
//def("f_adv_int", flood_advert_interval);
def("lat", node_lat);
def("lon", node_lon);
def("max_resend", max_resend_attempts); // repeated sending (feature)
def("radio", radio);
def("gps", gps);
def("repeat", repeat);
Expand Down
28 changes: 28 additions & 0 deletions examples/simple_repeater/MyMesh.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,20 @@

#define LAZY_CONTACTS_WRITE_DELAY 5000

// Fixed RSSI margin (dB above the noise floor) at which a DIRECT resend treats the channel as busy
// (isResendChannelActive(): busy when isReceivingPacket() || currentRSSI - noiseFloor >= this). It is
// the resend-only counterpart of the general-LBT _prefs.interference_threshold, which flows through
// getInterferenceThreshold() -> triggerNoiseFloorCalibrate() -> RadioLibWrapper::isChannelActive().
// They are separate today only because interference_threshold is blanket-initialised to 0 (disabled;
// see getInterferenceThreshold(), cited there as a currentRSSI() reliability concern). The margin
// currentRSSI-noiseFloor is only as trustworthy as both terms: the median noise-floor estimator
// (PR #2933) stabilises getNoiseFloor() and removes the dominant drift source (the old one-way
// ratchet to -120 that inflated every margin), substantially rehabilitating the threshold path.
// Once interference_threshold is enabled across the fleet, this constant can be superseded by it.
#ifndef RESEND_INTERFERENCE_MARGIN
#define RESEND_INTERFERENCE_MARGIN 12 // dB above noise floor that blocks a resend (non-invasive LBT)
#endif

void MyMesh::putNeighbour(const mesh::Identity &id, uint32_t timestamp, float snr) {
#if MAX_NEIGHBOURS // check if neighbours enabled
// find existing neighbour, else use least recently updated
Expand Down Expand Up @@ -553,6 +567,14 @@ uint32_t MyMesh::getDirectRetransmitDelay(const mesh::Packet *packet) {
return getRNG()->nextInt(0, 5*t + 1);
}

bool MyMesh::isResendChannelActive() {
// Non-invasive resend LBT (NO CAD, so RX stays open to overhear the downstream forward
// and cancel the resend). Block when a LoRa preamble/header is being received (often that
// forward) OR the live channel energy is well above the noise floor (foreign interference).
int margin = (int)radio_driver.getCurrentRSSI() - _radio->getNoiseFloor();
return radio_driver.isReceivingPacket() || (margin >= RESEND_INTERFERENCE_MARGIN);
}

mesh::DispatcherAction MyMesh::onRecvPacket(mesh::Packet* pkt) {
if (pkt->getRouteType() == ROUTE_TYPE_TRANSPORT_FLOOD) {
recv_pkt_region = region_map.findMatch(pkt, REGION_DENY_FLOOD);
Expand Down Expand Up @@ -893,6 +915,7 @@ MyMesh::MyMesh(mesh::MainBoard &board, mesh::Radio &radio, mesh::MillisecondCloc
_prefs.rx_delay_base = 0.0f; // turn off by default, was 10.0;
_prefs.tx_delay_factor = 0.5f; // was 0.25f
_prefs.direct_tx_delay_factor = 0.3f; // was 0.2
_prefs.max_resend_attempts = 2;
StrHelper::strncpy(_prefs.node_name, ADVERT_NAME, sizeof(_prefs.node_name));
_prefs.node_lat = ADVERT_LAT;
_prefs.node_lon = ADVERT_LON;
Expand Down Expand Up @@ -1177,6 +1200,11 @@ void MyMesh::formatPacketStatsReply(char *reply) {
getNumRecvFlood(), getNumRecvDirect());
}

void MyMesh::formatResendRatioReply(char *reply) {
if (!_prefs.max_resend_attempts) return; // plain "> 0" when resends are disabled
StatsFormatHelper::formatResendRatio(reply, getNumResentDirect(), getNumSentDirect());
}

void MyMesh::saveIdentity(const mesh::LocalIdentity &new_id) {
#if defined(NRF52_PLATFORM) || defined(STM32_PLATFORM)
IdentityStore store(*_fs, "");
Expand Down
5 changes: 5 additions & 0 deletions examples/simple_repeater/MyMesh.h
Original file line number Diff line number Diff line change
Expand Up @@ -150,12 +150,16 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks {
bool getCADEnabled() const override {
return _prefs.cad_enabled;
}
bool isResendChannelActive() override; // non-invasive resend LBT (preamble/RSSI, no CAD)
int getAGCResetInterval() const override {
return ((int)_prefs.agc_reset_interval) * 4000; // milliseconds
}
uint8_t getExtraAckTransmitCount() const override {
return _prefs.multi_acks;
}
uint8_t getMaxResendAttempts() const override {
return _prefs.max_resend_attempts;
}

#if ENV_INCLUDE_GPS == 1
void applyGpsPrefs() {
Expand Down Expand Up @@ -224,6 +228,7 @@ class MyMesh : public mesh::Mesh, public CommonCLICallbacks {
void formatStatsReply(char *reply) override;
void formatRadioStatsReply(char *reply) override;
void formatPacketStatsReply(char *reply) override;
void formatResendRatioReply(char *reply) override;
void startRegionsLoad() override;
bool saveRegions() override;
void onDefaultRegionChanged(const RegionEntry* r) override;
Expand Down
Loading