Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| dbb1aeb62e |
@@ -1,56 +1,8 @@
|
||||
---
|
||||
- name: Configure WAN connection marking
|
||||
community.routeros.api_modify:
|
||||
path: ip firewall mangle
|
||||
data:
|
||||
- action: mark-connection
|
||||
chain: forward
|
||||
connection-state: new
|
||||
new-connection-mark: wan-gpon
|
||||
out-interface: pppoe-gpon
|
||||
passthrough: true
|
||||
comment: Mark connections going out GPON
|
||||
- action: mark-connection
|
||||
chain: forward
|
||||
connection-state: new
|
||||
new-connection-mark: wan-lte
|
||||
out-interface: vlan6
|
||||
passthrough: true
|
||||
comment: Mark connections going out LTE
|
||||
handle_absent_entries: remove
|
||||
handle_entries_content: remove_as_much_as_possible
|
||||
ensure_order: true
|
||||
|
||||
- name: Configure IPv4 firewall filter rules
|
||||
community.routeros.api_modify:
|
||||
path: ip firewall filter
|
||||
data:
|
||||
- action: reject
|
||||
chain: forward
|
||||
connection-mark: wan-gpon
|
||||
out-interface: vlan6
|
||||
protocol: tcp
|
||||
reject-with: tcp-reset
|
||||
comment: Fast-fail TCP connections that shifted from GPON to LTE
|
||||
- action: reject
|
||||
chain: forward
|
||||
connection-mark: wan-gpon
|
||||
out-interface: vlan6
|
||||
reject-with: icmp-network-unreachable
|
||||
comment: Fast-fail non-TCP connections that shifted from GPON to LTE
|
||||
- action: reject
|
||||
chain: forward
|
||||
connection-mark: wan-lte
|
||||
out-interface: pppoe-gpon
|
||||
protocol: tcp
|
||||
reject-with: tcp-reset
|
||||
comment: Fast-fail TCP connections that shifted from LTE to GPON
|
||||
- action: reject
|
||||
chain: forward
|
||||
connection-mark: wan-lte
|
||||
out-interface: pppoe-gpon
|
||||
reject-with: icmp-network-unreachable
|
||||
comment: Fast-fail non-TCP connections that shifted from LTE to GPON
|
||||
- action: fasttrack-connection
|
||||
chain: forward
|
||||
connection-state: established,related
|
||||
|
||||
@@ -12,44 +12,15 @@
|
||||
scope: 30
|
||||
suppress-hw-offload: false
|
||||
target-scope: 10
|
||||
- comment: GPON Monitor 1
|
||||
disabled: false
|
||||
distance: 1
|
||||
dst-address: 1.0.0.1/32
|
||||
gateway: pppoe-gpon
|
||||
routing-table: main
|
||||
scope: 10
|
||||
suppress-hw-offload: false
|
||||
target-scope: 10
|
||||
- comment: GPON Monitor 2
|
||||
disabled: false
|
||||
distance: 1
|
||||
dst-address: 8.8.4.4/32
|
||||
gateway: pppoe-gpon
|
||||
routing-table: main
|
||||
scope: 10
|
||||
suppress-hw-offload: false
|
||||
target-scope: 10
|
||||
- comment: GPON Default 1
|
||||
disabled: false
|
||||
- disabled: false
|
||||
distance: 1
|
||||
dst-address: 0.0.0.0/0
|
||||
gateway: 1.0.0.1
|
||||
check-gateway: ping
|
||||
gateway: pppoe-gpon
|
||||
routing-table: main
|
||||
scope: 30
|
||||
suppress-hw-offload: false
|
||||
target-scope: 11
|
||||
- comment: GPON Default 2
|
||||
disabled: false
|
||||
distance: 2
|
||||
dst-address: 0.0.0.0/0
|
||||
gateway: 8.8.4.4
|
||||
check-gateway: ping
|
||||
routing-table: main
|
||||
scope: 30
|
||||
suppress-hw-offload: false
|
||||
target-scope: 11
|
||||
target-scope: 10
|
||||
vrf-interface: pppoe-gpon
|
||||
handle_absent_entries: remove
|
||||
handle_entries_content: remove_as_much_as_possible
|
||||
|
||||
@@ -61,7 +32,6 @@
|
||||
distance: 1
|
||||
dst-address: 2000::/3
|
||||
gateway: 2001:470:70:dd::1
|
||||
check-gateway: ping
|
||||
scope: 30
|
||||
target-scope: 10
|
||||
- comment: Tailnet
|
||||
|
||||
@@ -10,7 +10,6 @@
|
||||
password: "{{ routeros_pppoe_password }}"
|
||||
# Using CoreDNS container with DNS64
|
||||
use-peer-dns: false
|
||||
add-default-route: false
|
||||
user: "{{ routeros_pppoe_username }}"
|
||||
handle_absent_entries: remove
|
||||
handle_entries_content: remove_as_much_as_possible
|
||||
|
||||
@@ -18,7 +18,7 @@ spec:
|
||||
chart:
|
||||
spec:
|
||||
chart: valkey
|
||||
version: 0.9.4
|
||||
version: 0.10.0
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: valkey
|
||||
|
||||
@@ -84,10 +84,9 @@ subnets would fail routing lookup with "net unreachable" without it.
|
||||
|
||||
| Destination | Source | Distance | Active when |
|
||||
|---|---|---|---|
|
||||
| `1.0.0.1/32`, `8.8.4.4/32` | static via `pppoe-gpon` | 1 | always |
|
||||
| `0.0.0.0/0` | static via `1.0.0.1`, `8.8.4.4` (recursive) | 1, 2 | GPON ping check succeeds |
|
||||
| `0.0.0.0/0` | static via `pppoe-gpon` | 1 | GPON up |
|
||||
| `0.0.0.0/0` | BGP from D-Link via `192.168.6.2` | 200 | wwan up on D-Link |
|
||||
| `2000::/3` | static via `2001:470:70:dd::1` (HE tunnel) | 1 | HE tunnel ping check succeeds |
|
||||
| `2000::/3` | static via `sit1` (HE tunnel) | 1 | sit1 active (HE tunnel works) |
|
||||
| `2000::/3` | BGP from D-Link via `2001:470:61a3:600::2` | 200 | wwan up on D-Link |
|
||||
|
||||
RouterOS distance comparison is straightforward: distance 1 always wins
|
||||
@@ -137,12 +136,11 @@ preferred route for D-Link's own traffic.
|
||||
- **wwan modem goes down** → BIRD2 device protocol detects wwan0 down →
|
||||
static `lte_default` / `lte_default6` become unreachable → BGP withdraws
|
||||
announcements → CRS removes BGP-learned default
|
||||
- **GPON drops or blackholes** → recursive ping checks (1.0.0.1, 8.8.4.4) over `pppoe-gpon`
|
||||
fail (takes ~20s: 10s ping interval + 10s timeout) → CRS distance-1/2 default routes inactive → distance-200 BGP route
|
||||
activates → CRS withdraws its default-originate announcement to D-Link (loop
|
||||
prevention prevents reflecting D-Link's own route) → D-Link's kernel
|
||||
default-via-CRS is removed → D-Link uses wwan kernel default → traffic flows
|
||||
from CRS via vlan6 → D-Link → wwan
|
||||
- **GPON drops** → `pppoe-gpon` interface down → CRS distance-1 default
|
||||
route inactive → distance-200 BGP route activates → CRS withdraws its
|
||||
default-originate announcement to D-Link (since no default is installed
|
||||
any more) → D-Link's kernel default-via-CRS is removed → D-Link uses
|
||||
wwan kernel default → traffic flows from CRS via vlan6 → D-Link → wwan
|
||||
|
||||
All transitions are automatic and driven by interface state. No active
|
||||
probing (Netwatch / mwan3), no scripts toggling routes.
|
||||
@@ -243,16 +241,6 @@ QMI initialization within ~1 second.
|
||||
|
||||
Full investigation: see [wwan-bm806c-qmi-workaround.md](./wwan-bm806c-qmi-workaround.md).
|
||||
|
||||
## Multi-WAN Stale Connection Tracking
|
||||
|
||||
When the routing table fails over from GPON to LTE (or vice versa), RouterOS does not automatically clear existing connection tracking entries. If an established TCP/UDP connection is routed out the new WAN interface, it retains the NAT translation state (source IP) of the old WAN interface. The packet is sent to the ISP with the wrong source IP and is silently dropped, causing clients (like Tailscale) to hang for minutes until their internal sockets time out.
|
||||
|
||||
To solve this purely declaratively without scripts or blanket connection flushes, the `forward` chain is configured to "fast-fail" these shifted connections:
|
||||
|
||||
1. Connections are marked with their egress WAN upon establishment (`wan-gpon` or `wan-lte`) via the `mangle` table.
|
||||
2. If an established connection with a `wan-gpon` mark attempts to route out `vlan6` (LTE), or a `wan-lte` mark routes out `pppoe-gpon`, it is explicitly rejected (`tcp-reset` for TCP, `icmp-network-unreachable` for UDP) before reaching the NAT table.
|
||||
3. This rejection immediately signals the client OS that the route is dead, forcing the application (Tailscale, SIP clients, etc.) to instantly close the socket and establish a new one, which successfully binds to the new WAN interface and NAT state.
|
||||
|
||||
## Implementation files
|
||||
|
||||
| File | Role |
|
||||
|
||||
@@ -110,7 +110,7 @@ spec:
|
||||
kubernetes.io/os: linux
|
||||
containers:
|
||||
- name: coredns
|
||||
image: registry.k8s.io/coredns/coredns:v1.14.4
|
||||
image: registry.k8s.io/coredns/coredns:v1.14.3
|
||||
imagePullPolicy: IfNotPresent
|
||||
args: ["-conf", "/etc/coredns/Corefile"]
|
||||
ports:
|
||||
|
||||
@@ -97,7 +97,7 @@ spec:
|
||||
env:
|
||||
- name: GOMEMLIMIT
|
||||
value: 161MiB
|
||||
image: registry.k8s.io/coredns/coredns:v1.14.4
|
||||
image: registry.k8s.io/coredns/coredns:v1.14.3
|
||||
imagePullPolicy: IfNotPresent
|
||||
livenessProbe:
|
||||
failureThreshold: 5
|
||||
|
||||
@@ -23,7 +23,7 @@ spec:
|
||||
chart:
|
||||
spec:
|
||||
chart: openebs
|
||||
version: 4.5.1
|
||||
version: 4.5.0
|
||||
sourceRef:
|
||||
kind: HelmRepository
|
||||
name: openebs
|
||||
|
||||
Reference in New Issue
Block a user