Doku: OPNsense↔Horus WG produktiv (Route, Firewall, LAN/NAT).
Issue-Vorfall 2026-06-28, opnsense-step-a-nat und README mit Gateway 10.1.1.1 und wg_horus-Regeln. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,127 @@
|
|||||||
|
# OPNsense ↔ Horus WireGuard — Tunnel + LAN (2026-06-28)
|
||||||
|
|
||||||
|
**Datum:** 2026-06-28
|
||||||
|
**Status:** gelöst
|
||||||
|
**Betroffen:** WireGuard OPNsense (`wg_horus` / opt10, `10.1.1.22`) ↔ Horus (`10.1.1.1`), LAN/VLAN → Horus
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
| Test | Ergebnis |
|
||||||
|
|------|----------|
|
||||||
|
| WireGuard-Handshake OPNsense ↔ Horus | ✅ (trotzdem wirkte Tunnel „tot“) |
|
||||||
|
| OPNsense → Horus (`ping 10.1.1.1`, Source **wg_horus**) | ❌ → später ✅ |
|
||||||
|
| Horus → OPNsense (`ping 10.1.1.22`) | ❌ → später ✅ |
|
||||||
|
| LAN/PC → `10.1.1.1` | ❌ → ✅ nach Gateway-Fix + NAT |
|
||||||
|
|
||||||
|
Typisch: **Bytes gesendet, Handshake da**, aber **Ping/Daten innen tot** — sieht aus wie „Tunnel kaputt“, ist aber oft **Routing + Firewall auf dem zugewiesenen WG-Interface**.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Fehldiagnosen (Zeitfresser)
|
||||||
|
|
||||||
|
| These | Wahrheit |
|
||||||
|
|-------|----------|
|
||||||
|
| Keys/PSK auf Horus falsch (VM-Tunnel) | **Nein** — siehe [2026-06-28-vm101-horus-wireguard-nat.md](2026-06-28-vm101-horus-wireguard-nat.md) |
|
||||||
|
| Horus-Route-Skript `wg0-opnsense-routes.sh` blockiert Ping | **Nein** — betrifft nur Horus-Routing **zu** OPNsense/LAN, nicht OPNsense→Horus |
|
||||||
|
| Fritzbox / fehlende WAN-Zuweisung | **Nein** — OPNsense steht in **Fritzbox-DMZ**; Handshake beweist UDP hin+her |
|
||||||
|
| Nur LAN-Firewall fehlt | **Teilweise** — ohne funktionierenden **Tunnel innen** nützt LAN nichts |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Echte Ursachen (drei Stellen)
|
||||||
|
|
||||||
|
### 1. Keine Route zu Horus auf OPNsense (`Disable routes` = an)
|
||||||
|
|
||||||
|
Mit **Disable routes** am Peer legt OPNsense **keine** Routen aus Allowed IPs an → **manuelle Static Route** nötig.
|
||||||
|
|
||||||
|
**Falsch:**
|
||||||
|
|
||||||
|
- Gateway `10.1.1.21` (existiert nicht — Tunnel-IP ist **`10.1.1.22`**, Horus **`10.1.1.1`**)
|
||||||
|
- Route `10.1.1.0/24 → 192.168.178.1` (Fritzbox statt Tunnel)
|
||||||
|
|
||||||
|
**Richtig:**
|
||||||
|
|
||||||
|
```
|
||||||
|
Network: 10.1.1.0/24
|
||||||
|
Gateway: 10.1.1.1 ← Horus-Tunnel-IP (Peer-Gegenstelle)
|
||||||
|
Interface: wg_horus (opt10)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Allowed IPs** am Peer trotzdem setzen (`10.1.1.0/24, …`) — WireGuard-Crypto, unabhängig von Disable routes.
|
||||||
|
|
||||||
|
### 2. Firewall auf **`wg_horus` (opt10)** — nicht nur LAN/WAN
|
||||||
|
|
||||||
|
Handshake läuft im Kernel; **innerer Traffic** (ICMP/IP) braucht Regeln auf dem **zugewiesenen Interface**:
|
||||||
|
|
||||||
|
| Regel | Source | Destination | Zweck |
|
||||||
|
|-------|--------|-------------|--------|
|
||||||
|
| Pass | `10.1.1.0/24` | **This Firewall** | Horus → OPNsense (`ping 10.1.1.22`) |
|
||||||
|
| Pass | **This Firewall** / `10.1.1.22` | `10.1.1.0/24` | OPNsense → Horus (Diagnostics-Ping mit Source wg_horus) |
|
||||||
|
|
||||||
|
Ohne Inbound-Regel: Horus pingt, OPNsense antwortet nicht → 100 % Loss trotz Handshake.
|
||||||
|
|
||||||
|
### 3. LAN → Horus (Step A)
|
||||||
|
|
||||||
|
- **Firewall** auf LAN/VLAN: Pass → `10.1.1.0/24`
|
||||||
|
- **Outbound NAT** auf **`wg_horus`**: VLAN-Quellen → `10.1.1.0/24` → SNAT **`10.1.1.22`**
|
||||||
|
|
||||||
|
Horus-Peer kennt nur `10.1.1.22/32` + `10.100.2.0/24` — ohne NAT sieht Horus `192.168.x` und antwortet nicht sinnvoll.
|
||||||
|
|
||||||
|
**Gateway-Fix** (`10.1.1.1` statt `.21`) war der letzte Blocker für LAN.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Lösung (Kurz)
|
||||||
|
|
||||||
|
1. **Interfaces → Assignments:** `wg_horus` (opt10) zugewiesen
|
||||||
|
2. **Peer horus:** Allowed IPs `10.1.1.0/24, …`, **Disable routes = an** (wegen mehrerer WG-Tunnel)
|
||||||
|
3. **Static Route:** `10.1.1.0/24 → 10.1.1.1` via **wg_horus**
|
||||||
|
4. **Firewall wg_horus:** Pass ICMP (bzw. any) von `10.1.1.0/24` → This Firewall **und** Richtung raus
|
||||||
|
5. **Firewall LAN/VLAN:** Pass → `10.1.1.0/24`
|
||||||
|
6. **Outbound NAT (Hybrid):** Interface `wg_horus`, SNAT `10.1.1.22`
|
||||||
|
7. **Listen Port** Instanz z. B. `51821` (nach NAT-State-Problemen sinnvoll; VM 101 nutzt weiter eigenen Port)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Verifikation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Auf Horus (via VM 101):
|
||||||
|
ping -c3 10.1.1.22 # ~16 ms
|
||||||
|
ping -c3 10.1.1.5 # VM-Peer Vergleich
|
||||||
|
|
||||||
|
# OPNsense GUI:
|
||||||
|
# Diagnostics → Ping, Source = wg_horus, Dest = 10.1.1.1 → ~17 ms
|
||||||
|
|
||||||
|
# LAN (pve1 / PC):
|
||||||
|
ping -c3 10.1.1.1
|
||||||
|
|
||||||
|
# Horus während LAN-Ping:
|
||||||
|
tcpdump -ni wg0 icmp and host 10.1.1.1
|
||||||
|
# Quelle muss 10.1.1.22 sein (NAT), nicht 192.168.x
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Breakpoint-Matrix (für nächstes Mal)
|
||||||
|
|
||||||
|
| Symptom | Breakpoint |
|
||||||
|
|---------|------------|
|
||||||
|
| Handshake ok, OPNsense ping 10.1.1.1 fail (ohne Source) | Route fehlt / falsches Gateway |
|
||||||
|
| OPNsense ping ok **nur** mit Source `wg_horus` | Static Route + Outbound-FW wg_horus |
|
||||||
|
| Horus ping 10.1.1.22 fail | **Inbound-FW** wg_horus (10.1.1.0/24 → This Firewall) |
|
||||||
|
| Tunnel ok, LAN fail | LAN-FW + Outbound NAT + Gateway `10.1.1.1` |
|
||||||
|
| tcpdump Horus: kein LAN-Traffic | OPNsense vor WG (Route/FW LAN) |
|
||||||
|
| tcpdump Horus: 192.168.x statt 10.1.1.22 | Outbound NAT fehlt |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Referenzen
|
||||||
|
|
||||||
|
- Setup: [../shared/horus-opnsense-wireguard/README.md](../shared/horus-opnsense-wireguard/README.md)
|
||||||
|
- Step A NAT: [../shared/horus-opnsense-wireguard/opnsense-step-a-nat.md](../shared/horus-opnsense-wireguard/opnsense-step-a-nat.md)
|
||||||
|
- VM-101-NAT-Vorfall: [2026-06-28-vm101-horus-wireguard-nat.md](2026-06-28-vm101-horus-wireguard-nat.md)
|
||||||
|
|
||||||
|
**⛔ VM 104 (OPNsense) nie per Agent `qm stop/start` — siehe [../pve1/04_fallback_aktivierung.md](../pve1/04_fallback_aktivierung.md).**
|
||||||
@@ -23,3 +23,4 @@ Das Handover beschreibt den *aktuellen Übergabestand*. Hier dagegen landen **ab
|
|||||||
| Datum | Vorfall | Status |
|
| Datum | Vorfall | Status |
|
||||||
|-------|---------|--------|
|
|-------|---------|--------|
|
||||||
| 2026-06-28 | [VM 101 ↔ Horus WireGuard tot (NAT, nicht Keys)](2026-06-28-vm101-horus-wireguard-nat.md) | gelöst |
|
| 2026-06-28 | [VM 101 ↔ Horus WireGuard tot (NAT, nicht Keys)](2026-06-28-vm101-horus-wireguard-nat.md) | gelöst |
|
||||||
|
| 2026-06-28 | [OPNsense ↔ Horus Tunnel + LAN (Route, Firewall wg_horus, NAT)](2026-06-28-opnsense-horus-wireguard-lan.md) | gelöst |
|
||||||
|
|||||||
@@ -5,163 +5,111 @@ Direkter WireGuard-Tunnel zwischen **OPNsense** (lokales Netz) und **Horus** (VP
|
|||||||
| | |
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
| **Horus** | `horus.jeanavril.com` / `207.180.222.207`, WG-Port **61951**, Tunnel-IP **10.1.1.1** |
|
| **Horus** | `horus.jeanavril.com` / `207.180.222.207`, WG-Port **61951**, Tunnel-IP **10.1.1.1** |
|
||||||
| **OPNsense** | Tunnel-IP **10.1.1.22** (Peer `opnsense-jeanavril`) |
|
| **OPNsense** | Tunnel-IP **10.1.1.22**, Interface **`wg_horus` (opt10)** |
|
||||||
| **VM 101** (legacy) | Tunnel-IP **10.1.1.5** (Peer `server5`) — **eigener Tunnel**, bleibt |
|
| **VM 101** | Tunnel-IP **10.1.1.5** — **eigener Tunnel**, bleibt (Cert-Rsync, SSH) |
|
||||||
|
|
||||||
Configs inkl. Private Keys: **privates Repo** — siehe Dateien in diesem Ordner.
|
**Produktiv seit 2026-06-28.** Vorfälle & Fehldiagnosen: [../../issues/2026-06-28-opnsense-horus-wireguard-lan.md](../../issues/2026-06-28-opnsense-horus-wireguard-lan.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Keys nicht verwechseln (VM ≠ OPNsense)
|
## Keys nicht verwechseln (VM ≠ OPNsense)
|
||||||
|
|
||||||
Zwei getrennte Tunnel = **zwei getrennte Keypairs**. Horus kennt jeden Peer nur über den **Public Key**.
|
| Tunnel | Private Key (Datei) | Public Key |
|
||||||
|
|--------|-------------------|------------|
|
||||||
| Tunnel | Private Key (Datei) | Public Key (muss in GUI stehen) |
|
|
||||||
|--------|-------------------|----------------------------------|
|
|
||||||
| **OPNsense** `10.1.1.22` | `opnsense-client.conf` → `AGEam06B9…` | `walbWTYXAGOD1mOxPK+NwKT6qUhLyY0qieWBeTIbdXU=` |
|
| **OPNsense** `10.1.1.22` | `opnsense-client.conf` → `AGEam06B9…` | `walbWTYXAGOD1mOxPK+NwKT6qUhLyY0qieWBeTIbdXU=` |
|
||||||
| **VM 101** `10.1.1.5` | `vm101-client.conf` → `SKMnLpkj…` | `VB3Cf8kDxpzO+FyMrLxPyJ0vUjm8yJ/qIKmhY2KeeyI=` |
|
| **VM 101** `10.1.1.5` | `vm101-client.conf` → `SKMnLpkj…` | `VB3Cf8kDxpzO+FyMrLxPyJ0vUjm8yJ/qIKmhY2KeeyI=` |
|
||||||
|
|
||||||
**Falscher Private Key auf OPNsense** → GUI zeigt evtl. trotzdem einen Public Key, Horus antwortet aber nicht zum richtigen Peer → Symptom: **Bytes gesendet, 0 empfangen, kein Handshake**.
|
**Niemals** den VM-Key auf OPNsense verwenden.
|
||||||
|
|
||||||
OPNsense-Instanz: Private Key **nur** aus `opnsense-client.conf`, **nicht** aus `vm101-client.conf`.
|
---
|
||||||
|
|
||||||
## Subnetz-Aufteilung (kein Teilen zwischen Peers)
|
## Subnetz-Aufteilung
|
||||||
|
|
||||||
| Netz | Horus-Peer | Wer routet |
|
| Netz | Horus-Peer | Routing |
|
||||||
|------|------------|------------|
|
|------|------------|---------|
|
||||||
| `10.1.1.5/32` | VM | VM-WG-Tunnel |
|
| `10.1.1.5/32`, `10.2.2.0/24` | VM | VM-WG |
|
||||||
| **`10.2.2.0/24`** (docbr0) | **VM** | VM → docbr0; OPNsense nur `→ 192.168.10.10` im LAN |
|
| `10.1.1.22/32`, `10.100.2.0/24` | OPNsense | OPNsense-WG |
|
||||||
| `10.1.1.22/32` | OPNsense | OPNsense-WG-Tunnel |
|
| `192.168.10–60.0/24` | — | OPNsense **NAT** → Horus sieht `10.1.1.22` |
|
||||||
| `10.100.2.0/24` (Services pve2) | OPNsense | OPNsense opt7 |
|
|
||||||
| `192.168.10–60.0/24` (VLANs) | — (nicht auf Horus) | OPNsense **NAT** → Horus sieht nur `10.1.1.22` |
|
|
||||||
|
|
||||||
**Schritt A (NAT):** Horus ✅ · OPNsense → [opnsense-step-a-nat.md](opnsense-step-a-nat.md)
|
---
|
||||||
|
|
||||||
**Nicht:** `10.2.0.0/16` auf VM-Peer — war zu groß; nur **`10.2.2.0/24`**.
|
## Dateien
|
||||||
|
|
||||||
| Datei | Inhalt |
|
| Datei | Inhalt |
|
||||||
|-------|--------|
|
|-------|--------|
|
||||||
| [opnsense-client.conf](opnsense-client.conf) | OPNsense Client (Private Key, PSK, Endpoint) |
|
| [opnsense-client.conf](opnsense-client.conf) | Keys, PSK, Endpoint (OPNsense) |
|
||||||
| [horus-server-peer-opnsense.conf](horus-server-peer-opnsense.conf) | Gegenstück auf Horus |
|
| [opnsense-step-a-nat.md](opnsense-step-a-nat.md) | **OPNsense-Setup** (Route, Firewall, NAT) |
|
||||||
| [vm101-client.conf](vm101-client.conf) | VM 101 Client (Referenz / Migration) |
|
| [horus-server-peer-opnsense.conf](horus-server-peer-opnsense.conf) | Horus Peer-Block |
|
||||||
| [horus-server-peer-vm101.conf](horus-server-peer-vm101.conf) | VM-Peer auf Horus |
|
| [wg0-opnsense-routes.sh](wg0-opnsense-routes.sh) | Horus Route-Script |
|
||||||
|
| [vm101-client.conf](vm101-client.conf) / [horus-server-peer-vm101.conf](horus-server-peer-vm101.conf) | VM-Tunnel (getrennt) |
|
||||||
|
|
||||||
## Topologie
|
---
|
||||||
|
|
||||||
```
|
## OPNsense — Kurz-Checkliste
|
||||||
LAN/VLANs (192.168.x.0/24, 10.2.2.0/24)
|
|
||||||
↓
|
|
||||||
OPNsense (10.1.1.22) ←——WireGuard——→ Horus (10.1.1.1)
|
|
||||||
↓ ↓
|
|
||||||
10.2.2.0/24 → 192.168.10.10 (docbr0) Services / SSH / …
|
|
||||||
```
|
|
||||||
|
|
||||||
**10.2.2.0/24 (docbr0):** Horus schickt Traffic dorthin an OPNsense; OPNsense leitet weiter an **192.168.10.10** (bestehende Route/Gateway `VM101_DOCKER` — siehe [opnsense-docker-subnet-routing.md](../opnsense-docker-subnet-routing.md)).
|
Details: [opnsense-step-a-nat.md](opnsense-step-a-nat.md)
|
||||||
|
|
||||||
## Public Keys
|
1. **WireGuard Local + Peer** aus `opnsense-client.conf`
|
||||||
|
2. **Interfaces → Assignments:** `wg_horus` (opt10)
|
||||||
|
3. Peer **Disable routes = an** → **Static Route** `10.1.1.0/24` Gateway **`10.1.1.1`**, Interface **wg_horus**
|
||||||
|
4. **Firewall wg_horus:** Pass von `10.1.1.0/24` → This Firewall **und** raus Richtung Horus
|
||||||
|
5. **Firewall LAN/VLAN:** Pass → `10.1.1.0/24`
|
||||||
|
6. **Outbound NAT** auf `wg_horus` → SNAT `10.1.1.22`
|
||||||
|
7. Alte Route `10.1.1.0/24 → 192.168.178.1` **löschen**
|
||||||
|
|
||||||
| Rolle | Public Key |
|
Fritzbox: OPNsense in **DMZ** — kein extra WAN-Forwarding für WG.
|
||||||
|-------|------------|
|
|
||||||
| Horus (Server) | `qXxhgerS2ORypVadhKCBuxgIX5Pu4J75nSWazdtd+Qk=` |
|
|
||||||
| OPNsense | `walbWTYXAGOD1mOxPK+NwKT6qUhLyY0qieWBeTIbdXU=` |
|
|
||||||
| VM 101 | `VB3Cf8kDxpzO+FyMrLxPyJ0vUjm8yJ/qIKmhY2KeeyI=` |
|
|
||||||
|
|
||||||
## OPNsense einrichten
|
---
|
||||||
|
|
||||||
Werte aus [opnsense-client.conf](opnsense-client.conf) in die GUI übernehmen.
|
## Horus (erledigt)
|
||||||
|
|
||||||
**VPN → WireGuard → Local → +**
|
- Peer `walbWTYX…`: AllowedIPs `10.1.1.22/32`, `10.100.2.0/24`
|
||||||
|
- `wg0-opnsense-routes.service` für Kernel-Routen zu OPNsense/Services
|
||||||
|
- VM-Peer: `10.1.1.5/32`, `10.2.2.0/24` only
|
||||||
|
|
||||||
| Feld | Wert |
|
---
|
||||||
|------|------|
|
|
||||||
| Enabled | ✓ |
|
|
||||||
| Name | `wg_horus` |
|
|
||||||
| Listen port | leer oder `51820` (nur ausgehend nötig) |
|
|
||||||
| Tunnel Address | `10.1.1.22/32` |
|
|
||||||
| MTU | `1250` |
|
|
||||||
| Private key | `[Interface] PrivateKey` aus `opnsense-client.conf` |
|
|
||||||
|
|
||||||
**VPN → WireGuard → Endpoints → +** (Peer Horus)
|
## Tests
|
||||||
|
|
||||||
| Feld | Wert |
|
|
||||||
|------|------|
|
|
||||||
| Enabled | ✓ |
|
|
||||||
| Name | `horus` |
|
|
||||||
| Public Key | `qXxhgerS2ORypVadhKCBuxgIX5Pu4J75nSWazdtd+Qk=` |
|
|
||||||
| Shared Secret | `[Peer] PresharedKey` aus `opnsense-client.conf` |
|
|
||||||
| Allowed IPs | `10.1.1.0/24, 10.1.2.0/24, 10.1.3.0/24, 10.1.4.0/24, 10.8.0.0/24` |
|
|
||||||
| Endpoint Address | `horus.jeanavril.com` |
|
|
||||||
| Endpoint Port | `61951` |
|
|
||||||
| Persistent Keepalive | `25` |
|
|
||||||
|
|
||||||
Instance und Endpoint verknüpfen (Peer der Instance zuweisen — je nach OPNsense-Version unter **Local → Peers** oder Endpoint an Instance binden).
|
|
||||||
|
|
||||||
### Interface zuweisen
|
|
||||||
|
|
||||||
**Interfaces → Assignments → New → `wg_horus` (optX) → Save**
|
|
||||||
|
|
||||||
Interface aktivieren, ggf. **Block private networks** auf WG-Interface deaktivieren (Site-to-Site).
|
|
||||||
|
|
||||||
### Firewall
|
|
||||||
|
|
||||||
| Regel | Interface | Direction | Source | Destination | Beschreibung |
|
|
||||||
|-------|-----------|-----------|--------|-------------|--------------|
|
|
||||||
| Pass | LAN / VLAN10 / … | in | Netz(e) | `10.1.1.0/24`, `10.8.0.0/24`, … | LAN → Horus-Netze |
|
|
||||||
| Pass | `optX` (WG) | in | `10.1.1.0/24` | LAN / VLANs | Horus → Heimnetz (Rückweg) |
|
|
||||||
|
|
||||||
### Routing auf Horus
|
|
||||||
|
|
||||||
Siehe [horus-server-peer-opnsense.conf](horus-server-peer-opnsense.conf) — bereits auf Horus aktiv.
|
|
||||||
|
|
||||||
## Test
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Handshake auf Horus:
|
# Horus → OPNsense
|
||||||
ssh jean@192.168.10.10 'ssh root@10.1.1.1 wg show wg0 | grep -A6 walbWTYX'
|
ssh jean@192.168.10.10 'ssh root@10.1.1.1 ping -c3 10.1.1.22'
|
||||||
|
|
||||||
# Von LAN-PC:
|
# OPNsense GUI: Diagnostics → Ping, Source wg_horus, Dest 10.1.1.1
|
||||||
|
|
||||||
|
# LAN
|
||||||
ping 10.1.1.1
|
ping 10.1.1.1
|
||||||
ssh root@10.1.1.1
|
|
||||||
|
# Horus: LAN-Traffic muss als 10.1.1.22 erscheinen
|
||||||
|
ssh root@10.1.1.1 'tcpdump -ni wg0 icmp and host 10.1.1.1'
|
||||||
```
|
```
|
||||||
|
|
||||||
## Horus: fehlende WG-Routen (wichtig)
|
---
|
||||||
|
|
||||||
`wg syncconf` auf Horus legt **keine Kernel-Routen** für die OPNsense-`AllowedIPs` an. Ohne Fix gehen Antworten von Horus zu euren LANs über **eth0/Internet** statt **wg0**.
|
## Wenn „Handshake ok, aber Ping tot“
|
||||||
|
|
||||||
Symptom auf Horus: `ip route get 192.168.20.2` zeigt `via 207.180.222.1 dev eth0` statt `dev wg0`.
|
| Richtung | Erste Prüfung |
|
||||||
|
|----------|----------------|
|
||||||
|
| OPNsense → Horus | Static Route Gateway **`10.1.1.1`**, Ping mit Source **wg_horus** |
|
||||||
|
| Horus → OPNsense | Firewall **wg_horus** Inbound |
|
||||||
|
| LAN → Horus | LAN-Pass + Outbound NAT + Gateway |
|
||||||
|
|
||||||
**Fix (auf Horus, persistent):**
|
Breakpoint-Matrix: [Issue-Doku](../../issues/2026-06-28-opnsense-horus-wireguard-lan.md)
|
||||||
|
|
||||||
```bash
|
---
|
||||||
/usr/local/sbin/wg0-opnsense-routes.sh # siehe wg0-opnsense-routes.sh in diesem Ordner
|
|
||||||
systemctl enable --now wg0-opnsense-routes.service
|
|
||||||
```
|
|
||||||
|
|
||||||
**iptables/ufw:** wg0 ist offen (61951/udp, `Anywhere on wg0` IN + FWD). Kein IP-spezifischer Block für 10.1.1.22.
|
## VM 101 — eigener Tunnel
|
||||||
|
|
||||||
## OPNsense: falsche Route (LAN → Horus)
|
Separater WG-Tunnel; bei NAT-Problemen siehe [../../issues/2026-06-28-vm101-horus-wireguard-nat.md](../../issues/2026-06-28-vm101-horus-wireguard-nat.md).
|
||||||
|
|
||||||
Wenn der Tunnel steht, aber vom LAN nichts ankommt: oft eine **alte Static Route** auf OPNsense.
|
**⛔ VM 104 (OPNsense) nie per Agent stoppen** — [../../pve1/04_fallback_aktivierung.md](../../pve1/04_fallback_aktivierung.md)
|
||||||
|
|
||||||
Traceroute von intern sollte sein: `… → OPNsense → 10.1.1.1` (via WG).
|
---
|
||||||
|
|
||||||
Falsch (beobachtet): Hop 2 = `192.168.178.1` — Traffic geht zur alten Fritz/VM-Route statt WireGuard.
|
|
||||||
|
|
||||||
**Prüfen:** System → Routes → Einträge für `10.1.1.0/24` / `10.8.0.0/24` — muss über **WireGuard-Gateway** (`horusopnsense`), nicht `192.168.178.1`.
|
|
||||||
|
|
||||||
**Firewall:** VLAN20 → Destination `10.1.1.0/24` → Pass.
|
|
||||||
|
|
||||||
## VM 101 — eigener Tunnel (bleibt)
|
|
||||||
|
|
||||||
VM 101 behält **eigenes** WireGuard zu Horus (`10.1.1.5`) — für Cert-Rsync, SSH, Automation.
|
|
||||||
|
|
||||||
Horus `AllowedIPs` VM-Peer: **`10.1.1.5/32`, `10.2.2.0/24` only** — siehe [horus-server-peer-vm101.conf](horus-server-peer-vm101.conf).
|
|
||||||
|
|
||||||
Wenn OPNsense stabil läuft: **kein** Abschalten der VM-WG nötig. Nur **keine überlappenden Subnetze** zwischen VM- und OPNsense-Peer auf Horus.
|
|
||||||
|
|
||||||
## Referenzen
|
## Referenzen
|
||||||
|
|
||||||
| Thema | Doc |
|
| Thema | Doc |
|
||||||
|-------|-----|
|
|-------|-----|
|
||||||
| Horus SSH-Keys | [../ssh/README.md](../ssh/README.md#horus-vps-wireguard) |
|
| Horus SSH | [../ssh/README.md](../ssh/README.md#horus-vps-wireguard) |
|
||||||
| Route-Skript | [wg0-opnsense-routes.sh](wg0-opnsense-routes.sh) |
|
| docbr0 | [../../pve1/guests/vm101-ubuntu/docbr0-opnsense-routing.md](../../pve1/guests/vm101-ubuntu/docbr0-opnsense-routing.md) |
|
||||||
| docbr0 / 10.2.2.0/24 | [../../pve1/guests/vm101-ubuntu/docbr0-opnsense-routing.md](../../pve1/guests/vm101-ubuntu/docbr0-opnsense-routing.md) |
|
| Issues-Index | [../../issues/README.md](../../issues/README.md) |
|
||||||
| VLAN-Übersicht | [../infrastruktur-netzwerk.md](../infrastruktur-netzwerk.md) |
|
|
||||||
|
|||||||
@@ -1,64 +1,139 @@
|
|||||||
# Schritt A — LAN/VLAN → Horus per NAT (OPNsense)
|
# OPNsense — Horus WireGuard (Routing, Firewall, LAN/NAT)
|
||||||
|
|
||||||
**Ziel:** Interne Hosts (LAN/VLANs) erreichen Horus (`10.1.1.1` + dort gehostete Dienste) über den OPNsense↔Horus-WireGuard-Tunnel. Horus kennt **keine** `192.168.x`-Netze → Rückrouting ins Heimnetz ist nicht möglich (gewollt).
|
**Stand:** 2026-06-28 (produktiv getestet)
|
||||||
|
**Interface:** `wg_horus` (opt10) · Tunnel-IP **`10.1.1.22`** · Horus **`10.1.1.1`**
|
||||||
|
|
||||||
| Richtung | Verhalten |
|
Vollständiger Vorfall: [../../issues/2026-06-28-opnsense-horus-wireguard-lan.md](../../issues/2026-06-28-opnsense-horus-wireguard-lan.md)
|
||||||
|----------|-----------|
|
|
||||||
| LAN/VLAN → Horus | ✅ via NAT (Quelle wird `10.1.1.22`) |
|
|
||||||
| Horus → LAN/VLAN | ❌ nicht möglich (Horus kennt die Netze nicht) |
|
|
||||||
| Horus ↔ `10.100.2.0/24` | ✅ bidirektional (Services, OPNsense-Peer) |
|
|
||||||
| Horus ↔ `10.2.2.0/24` | ✅ über **VM-101-Peer** (`10.1.1.5`), nicht OPNsense |
|
|
||||||
|
|
||||||
## Warum NAT zwingend ist
|
|
||||||
|
|
||||||
Horus' Peer-Eintrag für OPNsense hat `AllowedIPs = 10.1.1.22/32, 10.100.2.0/24`. WireGuard verwirft jedes Antwortpaket an eine Quelle, die **nicht** in diesen AllowedIPs steht. LAN-Traffic (`192.168.x`) muss daher **vor** dem Tunnel auf `10.1.1.22` ge-SNAT-et werden — sonst kommt von Horus nie eine Antwort zurück.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 0. Voraussetzung
|
## Topologie
|
||||||
|
|
||||||
WireGuard-Instanz „horus" als **Interface zuweisen** (*Interfaces → Assignments*), Tunnel-Adresse `10.1.1.22/32`, z. B. Name `horusopnsense`. Nötig für NAT-Interface + Firewallregeln.
|
```
|
||||||
|
LAN/VLAN (192.168.x) → OPNsense (NAT → 10.1.1.22) → wg_horus → Horus 10.1.1.1
|
||||||
|
Services (10.100.2.0/24) → direkt über wg_horus (bidirektional, kein NAT nötig)
|
||||||
|
VM docbr0 (10.2.2.0/24) → über VM-101-Tunnel (10.1.1.5), nicht OPNsense-WG
|
||||||
|
```
|
||||||
|
|
||||||
## 1. Aliase (*Firewall → Aliases*)
|
OPNsense hängt in der **Fritzbox-DMZ** — kein extra WAN-Port-Forwarding für WireGuard nötig (Handshake beweist UDP-Rückweg).
|
||||||
|
|
||||||
- **`HORUS_WG`** (Network): `10.1.1.0/24` *(optional zusätzlich `10.1.2.0/24, 10.1.3.0/24, 10.1.4.0/24, 10.8.0.0/24`)*
|
---
|
||||||
- **`LAN_VLANS`** (Network): `192.168.10.0/24, 192.168.20.0/24, 192.168.30.0/24, 192.168.40.0/24, 192.168.50.0/24, 192.168.60.0/24`
|
|
||||||
|
|
||||||
## 2. Falsche Static Route löschen (*System → Routes → Configuration*)
|
## 0. WireGuard-Basis
|
||||||
|
|
||||||
- **Löschen:** jede Route `10.1.1.0/24` (bzw. `10.1.2–4.0/24`, `10.8.0.0/24`) **→ Gateway `192.168.178.1`** — die kapert den Traffic am Tunnel vorbei.
|
Werte aus [opnsense-client.conf](opnsense-client.conf).
|
||||||
- **Behalten:** `10.2.2.0/24 → 192.168.10.10` (VM) und `10.100.2.0/24` (Services).
|
|
||||||
|
|
||||||
## 3. Outbound NAT (*Firewall → NAT → Outbound*, Modus **Hybrid**)
|
| Wo | Feld | Wert |
|
||||||
|
|----|------|------|
|
||||||
|
| **Local** | Tunnel Address | `10.1.1.22/32` |
|
||||||
|
| **Local** | Listen Port | z. B. **`51821`** (fester Port; nach NAT-Glitches neu wählen) |
|
||||||
|
| **Local** | Private Key | `AGEam06B9…` (**nicht** VM-Key `SKMnLpkj…`) |
|
||||||
|
| **Peer horus** | Public Key | `qXxhgerS2ORypVadhKCBuxgIX5Pu4J75nSWazdtd+Qk=` |
|
||||||
|
| **Peer horus** | PSK | aus `opnsense-client.conf` |
|
||||||
|
| **Peer horus** | Endpoint | `horus.jeanavril.com:61951` |
|
||||||
|
| **Peer horus** | Allowed IPs | `10.1.1.0/24, 10.1.2.0/24, 10.1.3.0/24, 10.1.4.0/24, 10.8.0.0/24` |
|
||||||
|
| **Peer horus** | Persistent Keepalive | `25` |
|
||||||
|
| **Peer horus** | **Disable routes** | **✓ an** (mehrere WG-Instanzen: Surfshark, WGServer, wg_horus) |
|
||||||
|
|
||||||
Neue Regel (oben):
|
**Interfaces → Assignments:** Instanz als **`wg_horus` (opt10)** zuweisen — sonst keine Firewall/NAT auf dem Tunnel.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Static Route (Pflicht bei Disable routes)
|
||||||
|
|
||||||
|
**System → Routing → Configuration → Routes → +**
|
||||||
|
|
||||||
| Feld | Wert |
|
| Feld | Wert |
|
||||||
|------|------|
|
|------|------|
|
||||||
| Interface | `horusopnsense` (WG) |
|
| Network | `10.1.1.0/24` *(weitere Horus-Netze bei Bedarf)* |
|
||||||
| TCP/IP Version | IPv4 |
|
| Gateway | **`10.1.1.1`** ← Horus-Tunnel-IP (**nicht** `.21`, **nicht** `192.168.178.1`) |
|
||||||
| Protocol | any |
|
| Interface | **`wg_horus`** |
|
||||||
| Source | `LAN_VLANS` |
|
|
||||||
| Destination | `HORUS_WG` |
|
|
||||||
| Translation / target | **Interface address** (= `10.1.1.22`); falls Interface nicht zuweisbar: *Single host* `10.1.1.22` |
|
|
||||||
|
|
||||||
Nicht NATen: Ziele `10.2.2.0/24`, `10.100.2.0/24` (bleiben geroutet).
|
**Diagnostics → Routes:** Suche `10.1.1.1` → muss über **wg_horus** gehen.
|
||||||
|
|
||||||
## 4. Firewall-Pass (*Firewall → Rules → LAN bzw. je VLAN*)
|
Alte Routen **löschen:** `10.1.1.0/24 → 192.168.178.1` (Fritzbox-Umweg).
|
||||||
|
|
||||||
| Action | Source | Destination |
|
**Behalten:** `10.2.2.0/24 → 192.168.10.10` (VM), `10.100.2.0/24` (Services).
|
||||||
|--------|--------|-------------|
|
|
||||||
| Pass | LAN-/VLAN-net | `HORUS_WG` |
|
|
||||||
|
|
||||||
Optional (defense in depth, eingehend auf `horusopnsense`): Source `HORUS_WG` → Dest `LAN_VLANS` = **Block**.
|
|
||||||
|
|
||||||
## 5. Test
|
|
||||||
|
|
||||||
- LAN-PC: `tracert 10.1.1.1` → Hop 1 OPNsense, dann Horus (**kein** `192.168.178.x`); dann `ping 10.1.1.1`.
|
|
||||||
- Auf Horus: `tcpdump -ni wg0 icmp` muss als Quelle **`10.1.1.22`** zeigen (NAT greift), nicht `192.168.x`.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ⚠️ Falls es trotz korrekter Config nicht geht
|
## 2. Firewall — Interface `wg_horus` (opt10)
|
||||||
|
|
||||||
Denk an OPNsenses fragilen CARP/State-Zustand (war Ursache des VM-101-Tunnel-Problems, siehe [../../issues/2026-06-28-vm101-horus-wireguard-nat.md](../../issues/2026-06-28-vm101-horus-wireguard-nat.md)). Auf beiden OPNsense-Knoten Sync prüfen, ggf. State-Tabelle zurücksetzen.
|
**Ohne diese Regeln: Handshake ok, Ping/Daten innen tot.**
|
||||||
|
|
||||||
|
| # | Action | Source | Destination | Proto | Zweck |
|
||||||
|
|---|--------|--------|-------------|-------|--------|
|
||||||
|
| 1 | Pass | `10.1.1.0/24` | **This Firewall** | ICMP (oder any) | Horus → OPNsense (`ping 10.1.1.22`) |
|
||||||
|
| 2 | Pass | **This Firewall** / `10.1.1.22/32` | `10.1.1.0/24` | any | OPNsense → Horus (Diagnostics-Ping mit Source wg_horus) |
|
||||||
|
|
||||||
|
→ **Save → Apply Changes**
|
||||||
|
|
||||||
|
**Test OPNsense:** Diagnostics → Ping, Source = **`wg_horus`**, Destination = **`10.1.1.1`** (~17 ms).
|
||||||
|
|
||||||
|
**Test Horus:** `ping 10.1.1.22` (~16 ms).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Firewall — LAN / VLANs (LAN → Horus)
|
||||||
|
|
||||||
|
**Firewall → Rules → LAN** (und **PRIVAT**, **GAESTE**, … je nach Netz)
|
||||||
|
|
||||||
|
| Action | Source | Destination |
|
||||||
|
|--------|--------|-------------|
|
||||||
|
| Pass | LAN-/VLAN-net | `10.1.1.0/24` (Alias `HORUS_WG`) |
|
||||||
|
|
||||||
|
Optional Aliase (*Firewall → Aliases*):
|
||||||
|
|
||||||
|
- **`HORUS_WG`:** `10.1.1.0/24` (+ optional `10.1.2–4.0/24`, `10.8.0.0/24`)
|
||||||
|
- **`LAN_VLANS`:** `192.168.10.0/24`, `.20`, `.30`, `.40`, `.50`, `.60`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Outbound NAT (LAN → Horus, Step A)
|
||||||
|
|
||||||
|
Horus-Peer erlaubt nur **`10.1.1.22/32`** als Quelle von „Heimnetz“-Traffic — VLANs müssen maskiert werden.
|
||||||
|
|
||||||
|
**Firewall → NAT → Outbound** — Modus **Hybrid**, Regel **oben**:
|
||||||
|
|
||||||
|
| Feld | Wert |
|
||||||
|
|------|------|
|
||||||
|
| Interface | **`wg_horus`** |
|
||||||
|
| Source | `LAN_VLANS` (oder einzelne VLANs) |
|
||||||
|
| Destination | `HORUS_WG` |
|
||||||
|
| Translation | **Interface address** (`10.1.1.22`) |
|
||||||
|
|
||||||
|
**Nicht NATen:** `10.2.2.0/24` (VM), `10.100.2.0/24` (Services — direkt geroutet).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Tests (Reihenfolge)
|
||||||
|
|
||||||
|
| Schritt | Befehl / Ort | Erwartung |
|
||||||
|
|---------|--------------|-----------|
|
||||||
|
| 1 | OPNsense Ping, Source **wg_horus** → `10.1.1.1` | 0 % Loss |
|
||||||
|
| 2 | Horus `ping 10.1.1.22` | 0 % Loss |
|
||||||
|
| 3 | LAN-PC / pve1 `ping 10.1.1.1` | 0 % Loss |
|
||||||
|
| 4 | Horus `tcpdump -ni wg0 icmp` während LAN-Ping | Quelle **`10.1.1.22`**, nicht `192.168.x` |
|
||||||
|
| 5 | `tracert 10.1.1.1` vom LAN | kein Hop `192.168.178.x` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
| Symptom | Prüfen |
|
||||||
|
|---------|--------|
|
||||||
|
| Handshake ok, kein Ping innen | Static Route Gateway **`10.1.1.1`**, Firewall **wg_horus** |
|
||||||
|
| OPNsense-Ping nur mit Source wg_horus | Route + Outbound-Regel wg_horus |
|
||||||
|
| Horus → 10.1.1.22 fail | Inbound-Regel wg_horus (10.1.1.0/24 → This Firewall) |
|
||||||
|
| Tunnel ok, LAN fail | LAN-Pass + Outbound NAT; Gateway nochmal prüfen |
|
||||||
|
| Live View leer beim LAN-Ping | Route — Traffic verlässt LAN nicht Richtung wg_horus |
|
||||||
|
|
||||||
|
**Firewall → Log Files → Live View** während Ping — schnellster Beweis, welches Interface blockt.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Horus (Referenz, bereits erledigt)
|
||||||
|
|
||||||
|
- Peer `walbWTYX…`: AllowedIPs **`10.1.1.22/32`, `10.100.2.0/24`** only
|
||||||
|
- Route-Script: [wg0-opnsense-routes.sh](wg0-opnsense-routes.sh) + `wg0-opnsense-routes.service`
|
||||||
|
- VM-Peer getrennt: [horus-server-peer-vm101.conf](horus-server-peer-vm101.conf)
|
||||||
|
|||||||
Reference in New Issue
Block a user