Merge pull request 'Update registry.k8s.io/coredns/coredns Docker tag to v1.14.4' (#341 ) from renovate/registry.k8s.io-coredns-coredns-1.x into fresh-start

Update registry.k8s.io/coredns/coredns Docker tag to v1.14.4
Fast fail connection when WAN failover
2026-06-21 02:01:27 +00:00 · 2026-06-21 02:01:26 +00:00 · 2026-06-21 02:38:24 +02:00 · 2026-06-21 02:00:32 +02:00 · 2026-06-19 02:01:13 +00:00 · 2026-06-19 02:01:09 +00:00
49 changed files with 1805 additions and 198 deletions
@@ -0,0 +1,46 @@
+when:
+  - event: push
+    path:
+      include:
+        - mikrotik/coredns/**
+
+steps:
+  - name: Get registry creds from OpenBao
+    image: quay.io/openbao/openbao:2.5.5
+    environment:
+      VAULT_ADDR: https://openbao.lumpiasty.xyz:8200
+      ROLE_ID:
+        from_secret: renovate_role_id
+      SECRET_ID:
+        from_secret: renovate_secret_id
+    commands:
+      - bao write -field token auth/approle/login
+          role_id=$ROLE_ID
+          secret_id=$SECRET_ID > /woodpecker/.vault_id
+      - export VAULT_TOKEN=$(cat /woodpecker/.vault_id)
+      - 'printf "PLUGIN_USERNAME=%s\n" "$(bao kv get -mount secret -field REGISTRY_USERNAME container-registry)" > /woodpecker/registry.env'
+      - 'printf "PLUGIN_PASSWORD=%s\n" "$(bao kv get -mount secret -field REGISTRY_PASSWORD container-registry)" >> /woodpecker/registry.env'
+
+  - name: Build and push
+    image: woodpeckerci/plugin-docker-buildx:6.1.0
+    privileged: true
+    settings:
+      registry: gitea.lumpiasty.xyz
+      repo: gitea.lumpiasty.xyz/lumpiasty/coredns-mikrotik
+      platforms: linux/arm64
+      tags:
+        - latest
+        - ${CI_COMMIT_SHA:0:8}
+      dockerfile: mikrotik/coredns/Dockerfile
+      context: mikrotik/coredns/
+      env_file: /woodpecker/registry.env
+
+  - name: Invalidate OpenBao token
+    image: quay.io/openbao/openbao:2.5.5
+    environment:
+      VAULT_ADDR: https://openbao.lumpiasty.xyz:8200
+    commands:
+      - export VAULT_TOKEN=$(cat /woodpecker/.vault_id)
+      - bao write -f auth/token/revoke-self
+    when:
+      - status: [success, failure]
@@ -6,7 +6,7 @@ skip_clone: true

 steps:
  - name: Get kubernetes access from OpenBao
-    image: quay.io/openbao/openbao:2.5.4
+    image: quay.io/openbao/openbao:2.5.5
    environment:
      VAULT_ADDR: https://openbao.lumpiasty.xyz:8200
      ROLE_ID:
@@ -20,7 +20,7 @@ steps:
      - export VAULT_TOKEN=$(cat /woodpecker/.vault_id)
      - bao write -format json -f /kubernetes/creds/flux-reconcile > /woodpecker/kube_credentials
  - name: Construct Kubeconfig
-    image: alpine/k8s:1.36.1
+    image: alpine/k8s:1.36.2
    environment:
      KUBECONFIG: /woodpecker/kubeconfig
    commands:
@@ -41,7 +41,7 @@ steps:
    commands:
      - flux reconcile source git flux-system
  - name: Invalidate OpenBao token
-    image: quay.io/openbao/openbao:2.5.4
+    image: quay.io/openbao/openbao:2.5.5
    environment:
      VAULT_ADDR: https://openbao.lumpiasty.xyz:8200
    commands:
@@ -6,7 +6,7 @@ skip_clone: true

 steps:
  - name: Get renovate token from OpenBao
-    image: quay.io/openbao/openbao:2.5.4
+    image: quay.io/openbao/openbao:2.5.5
    environment:
      VAULT_ADDR: https://openbao.lumpiasty.xyz:8200
      ROLE_ID:
@@ -21,7 +21,7 @@ steps:
      - bao kv get -mount secret -field RENOVATE_TOKEN renovate > /woodpecker/renovate_token
      - bao kv get -mount secret -field GITHUB_COM_TOKEN renovate > /woodpecker/github_com_token
  - name: Run Renovate
-    image: renovate/renovate:43.197.0
+    image: renovate/renovate:43.222.1
    environment:
      RENOVATE_AUTODISCOVER: "true"
      RENOVATE_ENDPOINT: https://gitea.lumpiasty.xyz/api/v1
@@ -34,7 +34,7 @@ steps:
      - export GITHUB_COM_TOKEN=$(cat /woodpecker/github_com_token)
      - /usr/local/sbin/renovate-entrypoint.sh renovate
  - name: Invalidate OpenBao token
-    image: quay.io/openbao/openbao:2.5.4
+    image: quay.io/openbao/openbao:2.5.5
    environment:
      VAULT_ADDR: https://openbao.lumpiasty.xyz:8200
    commands:
@@ -15,6 +15,7 @@ gen-talos-config:
 		--config-patch @talos/patches/ollama.patch \
 		--config-patch @talos/patches/llama.patch \
 		--config-patch @talos/patches/frigate.patch \
+		--config-patch @talos/patches/woodpecker.patch \
 		--config-patch @talos/patches/anapistula-delrosalae.patch \
 		--output-types controlplane -o talos/generated/anapistula-delrosalae.yaml \
 		homelab https://kube-api.homelab.lumpiasty.xyz:6443
@@ -139,6 +139,8 @@ flowchart TD
   cluster -- "Routes exported via BGP" ----- k8s
 ```

+More information on network are available in [Network documentation](docs/network.md)
+
 Currently the k8s cluster consists of single node (hostname anapistula-delrosalae), which is a PC with Ryzen 5 3600, 64GB RAM, RX 580 8GB (for accelerating LLMs), 1TB NVMe SSD, 2TB and 3TB HDDs and serves both as control plane and worker node.

 ## Software stack
@@ -248,6 +250,8 @@ flowchart TD
 | <img src="docs/assets/llama-cpp.svg" alt="LLaMA.cpp" height="50" width="50"> | LLaMA.cpp | LLM inference server running local models with GPU acceleration |
 | <img src="docs/assets/llama-swap.svg" alt="llama-swap" height="50" width="50"> | llama-swap | Model swapping for LLaMA.cpp |
 | <img src="docs/assets/meridian.svg" alt="meridian" height="50" width="50"> | Meridian | Proxy that bridges Anthropic's official SDK to third-party tools |
+| | whisper.cpp | High-performance Whisper Automatic Speech Recognition inference server |
+| | Kokoro-FastAPI | Kokoro-82M text-to-speech inference server |

 ### Applications/Services

@@ -258,6 +262,7 @@ flowchart TD
 | <img src="docs/assets/teamspeak.svg" alt="iSpeak3" height="50" width="50"> | iSpeak3.pl | [ts3server://ispeak3.pl](ts3server://ispeak3.pl) | Public TeamSpeak 3 voice communication server |
 | <img src="docs/assets/immich.svg" alt="Immich" height="50" width="50"> | Immich | https://immich.lumpiasty.xyz/ | Self-hosted photo and video backup and streaming service |
 | <img src="docs/assets/frigate.svg" alt="Frigate" height="50" width="50"> | Frigate | https://frigate.lumpiasty.xyz/ | NVR for camera system with AI object detection and classification |
+| <img src="docs/assets/kaneo.svg" alt="Kaneo" height="50" width="50"> | Kaneo | https://kaneo.lumpiasty.xyz | Project management software |


 ## Development
@@ -266,7 +271,7 @@ This repo leverages [devenv](https://devenv.sh/) for easy setup of a development

 ### App deployment

-This repo is being watched by Flux running on cluster. To change config/add new app, simply commit to this repo and wait a while for cluster to reconcile changes. You can speed up this process by "notifying" Flux using `flux reconcile source git flux-system`.
+This repo is being watched by Flux running on cluster. To change config/add new app, simply commit to this repo and wait a while for cluster to reconcile changes. There is a Woodpecker job pushing Flux to reconcile cluster on push to this repository.

 Flux watches 3 kustomizations in this repo:

@@ -39,7 +39,6 @@ Secret layout expected in OpenBao (KVv2, mount `secret`):
 |---|---|
 | `routeros_api` | `username`, `password` |
 | `wan_pppoe` | `username`, `password` |
-| `router_tailscale` | `container_password` |

 ## OpenWrt dlink AP

@@ -39,15 +39,10 @@
              engine_mount_point=openbao_kv_mount
            ).secret[openbao_fields.wan_pppoe.password_key]
          }}
-        routeros_tailscale_container_password: >-
-          {{
-            lookup(
-              'community.hashi_vault.vault_kv2_get',
-              openbao_fields.routeros_tailscale_container.path,
-              engine_mount_point=openbao_kv_mount
-            ).secret[openbao_fields.routeros_tailscale_container.container_password_key]
-          }}
+
      no_log: true
+      tags:
+        - tailscale-script

  module_defaults:
    group/community.routeros.api:
@@ -3,9 +3,9 @@
  community.routeros.api_modify:
    path: ip address
    data:
-      - address: 172.17.0.1/16
-        interface: dockers
-        network: 172.17.0.0
+      - address: 172.20.0.1/24
+        interface: containers
+        network: 172.20.0.0
      - address: 192.168.4.1/24
        interface: lo
        network: 192.168.4.0
@@ -40,16 +40,18 @@
      - address: 2001:470:70:dd::2/64
        advertise: false
        interface: sit1
-      - address: ::ffff:ffff:ffff:ffff/64
-        from-pool: pool1
+      # Static instead of from-pool: pool allocation is dynamic (first free /64,
+      # e.g. ...:0::/64) which made the RDNSS address advertised in ND config
+      # point at a nonexistent router address. HE prefix is static, so static
+      # per-VLAN addressing is deterministic and matches docs/network.md.
+      - address: 2001:470:61a3:9:ffff:ffff:ffff:ffff/64
        interface: vlan2
      - address: 2001:470:61a3:500:ffff:ffff:ffff:ffff/64
-        interface: dockers
+        interface: containers
      - address: 2001:470:61a3:100::1/64
        advertise: false
        interface: vlan4
-      - address: ::ffff:ffff:ffff:ffff/64
-        from-pool: pool1
+      - address: 2001:470:61a3:a:ffff:ffff:ffff:ffff/64
        interface: vlan5
      - address: 2001:470:61a3:600::1/64
        advertise: false
@@ -5,7 +5,7 @@
    data:
      - name: bridge1
        vlan-filtering: true
-      - name: dockers
+      - name: containers
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible

@@ -62,9 +62,12 @@
  community.routeros.api_modify:
    path: interface bridge port
    data:
-      - bridge: dockers
-        interface: veth1
+      - bridge: containers
+        interface: veth-tailscale
        comment: Tailscale container interface
+      - bridge: containers
+        interface: veth-coredns
+        comment: CoreDNS container interface
      - bridge: bridge1
        interface: ether1
        pvid: 2
@@ -152,24 +155,9 @@
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible

- name: Configure DHCP networks
-  community.routeros.api_modify:
-    path: ip dhcp-server network
-    data:
-      - address: 192.168.0.0/24
-        dns-server: 192.168.0.1
-        gateway: 192.168.0.1
-      - address: 192.168.255.0/24
-        dns-none: true
-        gateway: 192.168.255.10
-      - address: 192.168.5.0/24
-        dns-server: 192.168.5.1
-        gateway: 192.168.5.1
-    handle_absent_entries: remove
-    handle_entries_content: remove_as_much_as_possible
-
-# TODO: IPv6 pools are useful when we have dynamic prefix, but we don't
-# We can remove it now
+# Pool is no longer referenced — vlan2/vlan5 now use static addresses
+# (addressing.yml) so the RDNSS addresses in ND config are deterministic.
+# Kept defined for one run after migration; safe to delete afterwards.
 - name: Configure IPv6 pools
  community.routeros.api_modify:
    path: ipv6 pool
@@ -188,7 +176,27 @@
    values:
      allow-remote-requests: true
      cache-size: 20480
-      servers: 1.1.1.1,1.0.0.1,2606:4700:4700::1111,2606:4700:4700::1001
+      # CoreDNS container: plain forwarder with selective AAAA suppression.
+      # Forwards upstream to 1.1.1.1/8.8.8.8.
+      servers: 172.20.0.3
+
+- name: Configure DNS static entries
+  community.routeros.api_modify:
+    path: ip dns static
+    data:
+      - name: ts.net
+        type: FWD
+        forward-to: 100.100.100.100
+        match-subdomain: true
+        comment: Tailscale MagicDNS
+      # Do NOT add a lumpiasty.xyz FWD entry here. RouterOS FWD entries return
+      # NOERROR with an empty answer instead of relaying NXDOMAIN, which breaks
+      # getaddrinfo search-domain processing (ENOTFOUND for valid names in k8s
+      # pods). Our own zone is handled in the CoreDNS Corefile (lumpiasty.xyz
+      # server block, AAAA kept) which relays rcodes correctly.
+      # See docs/coredns.md.
+    handle_absent_entries: remove
+    handle_entries_content: remove_as_much_as_possible

 - name: Configure NAT-PMP global settings
  community.routeros.api_find_and_modify:
@@ -202,7 +210,7 @@
  community.routeros.api_modify:
    path: ip nat-pmp interfaces
    data:
-      - interface: dockers
+      - interface: containers
        type: internal
      - interface: pppoe-gpon
        type: external
@@ -223,7 +231,7 @@
  community.routeros.api_modify:
    path: ip upnp interfaces
    data:
-      - interface: dockers
+      - interface: containers
        type: internal
      - interface: pppoe-gpon
        type: external
@@ -232,6 +240,22 @@
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible

+- name: Configure DHCP networks
+  community.routeros.api_modify:
+    path: ip dhcp-server network
+    data:
+      - address: 192.168.0.0/24
+        dns-server: 192.168.0.1
+        gateway: 192.168.0.1
+      - address: 192.168.255.0/24
+        dns-none: true
+        gateway: 192.168.255.10
+      - address: 192.168.5.0/24
+        dns-server: 192.168.5.1
+        gateway: 192.168.5.1
+    handle_absent_entries: remove
+    handle_entries_content: remove_as_much_as_possible
+
 - name: Configure IPv6 ND defaults
  community.routeros.api_find_and_modify:
    ignore_dynamic: false
@@ -240,3 +264,21 @@
      default: true
    values:
      advertise-dns: true
+
+# RDNSS (RFC 8106): advertise an IPv6 DNS server in RAs so dual-stack clients
+# have an IPv6 resolver. Points at the router's per-VLAN IPv6 address; RouterOS
+# DNS forwards to CoreDNS. No pref64 — NAT64 has been removed (see docs/coredns.md);
+# AAAA suppression now happens in CoreDNS, no client-side translation needed.
+- name: Configure IPv6 ND per-interface (RDNSS)
+  community.routeros.api_modify:
+    path: ipv6 nd
+    data:
+      # advertise-dns must be explicitly enabled — RouterOS creates new ND
+      # entries with advertise-dns=no, which suppresses the RDNSS option
+      # entirely even when a static dns= list is configured.
+      - interface: vlan2
+        advertise-dns: true
+        dns: 2001:470:61a3:9:ffff:ffff:ffff:ffff
+      - interface: vlan5
+        advertise-dns: true
+        dns: 2001:470:61a3:a:ffff:ffff:ffff:ffff
@@ -5,28 +5,12 @@
    path: container config
    find: {}
    values:
-      registry-url: https://ghcr.io
-      tmpdir: /tmp1/pull
+      tmpdir: tmp

 - name: Configure container env lists
  community.routeros.api_modify:
    path: container envs
-    data:
-      - key: ADVERTISE_ROUTES
-        list: tailscale
-        value: 192.168.0.0/24,192.168.1.0/24,192.168.4.1/32,192.168.100.1/32,192.168.255.0/24,10.42.0.0/16,10.43.0.0/16,10.44.0.0/16,2001:470:61a3::/48
-      - key: CONTAINER_GATEWAY
-        list: tailscale
-        value: 172.17.0.1
-      - key: PASSWORD
-        list: tailscale
-        value: "{{ routeros_tailscale_container_password }}"
-      - key: TAILSCALE_ARGS
-        list: tailscale
-        value: --accept-routes --advertise-exit-node --snat-subnet-routes=false
-      - key: UPDATE_TAILSCALE
-        list: tailscale
-        value: y
+    data: []
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible

@@ -35,29 +19,29 @@
    path: container mounts
    data:
      - dst: /var/lib/tailscale
-        list: tailscale
-        src: /usb1/tailscale
-      - dst: /root
-        list: tailscale-root
-        src: /tmp1/tailscale-root
+        list: tailscale_state
+        src: /tailscale/state
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible

- name: Configure tailscale container
+- name: Configure containers
  community.routeros.api_modify:
    path: container
    data:
-      - dns: 172.17.0.1
-        envlists: tailscale
-        hostname: mikrotik
-        interface: veth1
-        layer-dir: ""
-        mountlists: tailscale
-        name: tailscale-mikrotik:latest
-        remote-image: fluent-networks/tailscale-mikrotik:latest
-        root-dir: /usb1/containers/tailscale
+      - dns: 172.20.0.1
+        interface: veth-tailscale
+        logging: true
+        mountlists: tailscale_state
+        name: tailscale
+        remote-image: gitea.lumpiasty.xyz/lumpiasty/mikrotik-tailscale:stable
+        root-dir: tailscale/root
+        start-on-boot: true
+      - dns: 172.20.0.1
+        interface: veth-coredns
+        logging: true
+        name: coredns
+        remote-image: gitea.lumpiasty.xyz/lumpiasty/coredns-mikrotik:latest
+        root-dir: coredns/root
        start-on-boot: true
-        tmpfs: /tmp:67108864:01777
-        workdir: /
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible
@@ -1,8 +1,56 @@
 ---
+- name: Configure WAN connection marking
+  community.routeros.api_modify:
+    path: ip firewall mangle
+    data:
+      - action: mark-connection
+        chain: forward
+        connection-state: new
+        new-connection-mark: wan-gpon
+        out-interface: pppoe-gpon
+        passthrough: true
+        comment: Mark connections going out GPON
+      - action: mark-connection
+        chain: forward
+        connection-state: new
+        new-connection-mark: wan-lte
+        out-interface: vlan6
+        passthrough: true
+        comment: Mark connections going out LTE
+    handle_absent_entries: remove
+    handle_entries_content: remove_as_much_as_possible
+    ensure_order: true
+
 - name: Configure IPv4 firewall filter rules
  community.routeros.api_modify:
    path: ip firewall filter
    data:
+      - action: reject
+        chain: forward
+        connection-mark: wan-gpon
+        out-interface: vlan6
+        protocol: tcp
+        reject-with: tcp-reset
+        comment: Fast-fail TCP connections that shifted from GPON to LTE
+      - action: reject
+        chain: forward
+        connection-mark: wan-gpon
+        out-interface: vlan6
+        reject-with: icmp-network-unreachable
+        comment: Fast-fail non-TCP connections that shifted from GPON to LTE
+      - action: reject
+        chain: forward
+        connection-mark: wan-lte
+        out-interface: pppoe-gpon
+        protocol: tcp
+        reject-with: tcp-reset
+        comment: Fast-fail TCP connections that shifted from LTE to GPON
+      - action: reject
+        chain: forward
+        connection-mark: wan-lte
+        out-interface: pppoe-gpon
+        reject-with: icmp-network-unreachable
+        comment: Fast-fail non-TCP connections that shifted from LTE to GPON
      - action: fasttrack-connection
        chain: forward
        connection-state: established,related
@@ -43,6 +91,11 @@
        comment: Allow from SRV to internet
        in-interface: vlan4
        out-interface-list: wan
+      - action: accept
+        chain: forward
+        comment: Allow from SRV to SRV
+        in-interface: vlan4
+        out-interface: vlan4
      - action: accept
        chain: forward
        comment: Allow from SRV to CAM
@@ -60,8 +113,8 @@
        out-interface-list: wan
      - action: accept
        chain: forward
-        comment: Allow from dockers to everywhere
-        in-interface: dockers
+        comment: Allow from containers to everywhere
+        in-interface: containers
      - action: jump
        chain: forward
        comment: Allow port forwards
@@ -132,14 +185,14 @@
        protocol: tcp
      - action: accept
        chain: input
-        comment: Allow DNS from dockers
+        comment: Allow DNS from containers
        dst-port: 53
-        in-interface: dockers
+        in-interface: containers
        protocol: udp
      - action: accept
        chain: input
        dst-port: 53
-        in-interface: dockers
+        in-interface: containers
        protocol: tcp
      - action: accept
        chain: input
@@ -183,9 +236,9 @@
        protocol: udp
      - action: accept
        chain: input
-        comment: NAT-PMP from dockers (for tailscale)
+        comment: NAT-PMP from containers (for tailscale)
        dst-port: 5351
-        in-interface: dockers
+        in-interface: containers
        protocol: udp
      - action: reject
        chain: input
@@ -224,8 +277,8 @@
      - action: accept
        chain: allow-ports
        comment: Allow anything udp to Tailscale
-        dst-address: 172.17.0.2
-        out-interface: dockers
+        dst-address: 172.20.0.2
+        out-interface: containers
        protocol: udp
      - action: accept
        chain: allow-ports
@@ -277,6 +330,11 @@
        in-interface: '!pppoe-gpon'
        protocol: tcp
        to-addresses: 128.0.70.5
+      - action: masquerade
+        chain: srcnat
+        comment: hairpin to LoadBalancer pool (vlan4 -> vlan4)
+        dst-address: 10.44.0.0/16
+        in-interface: vlan4
      - action: dst-nat
        chain: dstnat
        comment: HTTPS
@@ -409,14 +467,14 @@
        out-interface-list: wan
      - action: accept
        chain: forward
-        comment: Allow from dockers to everywhere
-        in-interface: dockers
+        comment: Allow from containers to everywhere
+        in-interface: containers
      - action: accept
        chain: forward
-        comment: Allow from internet to dockers
+        comment: Allow from internet to containers
        dst-address: 2001:470:61a3:500::/64
        in-interface-list: wan
-        out-interface: dockers
+        out-interface: containers
      - action: accept
        chain: forward
        comment: Allow tcp transmission port to LAN
@@ -475,14 +533,14 @@
        protocol: tcp
      - action: accept
        chain: input
-        comment: Allow DNS from dockers
+        comment: Allow DNS from containers
        dst-port: 53
-        in-interface: dockers
+        in-interface: containers
        protocol: udp
      - action: accept
        chain: input
        dst-port: 53
-        in-interface: dockers
+        in-interface: containers
        protocol: tcp
      - action: accept
        chain: input
@@ -39,19 +39,43 @@
  loop_control:
    label: "{{ item.default_name }}"

- name: Configure temporary disk for containers
-  community.routeros.api_modify:
+# community.routeros.api_modify can't remove hardware disks
+# but it tries to do so with handle_absent_entries: remove
+# Working around by manually deleting other ones
+
+- name: Read current disk entries
+  community.routeros.api_info:
    path: disk
-    data:
-      - slot: tmp1
-        type: tmpfs
-      # This is not ideal, there's no unique identifier for usb disk,
-      # after reinstall it might be assigned to another slot
-      # Just adding disk with slot usb1 and not specifying anything else
-      # so ansible doesn't touch it
-      - slot: usb1
-    handle_absent_entries: remove
-    handle_entries_content: remove_as_much_as_possible
+  register: routeros_disks
+  check_mode: false
+
+- name: Remove stale software-defined disk entries
+  community.routeros.api:
+    path: disk
+    remove: "{{ item['.id'] }}"
+  loop: >-
+    {{
+      routeros_disks.result
+      | rejectattr('type', 'in', ['hardware', 'partition'])
+      | rejectattr('slot', 'equalto', 'tmp')
+    }}
+  loop_control:
+    label: "{{ item.slot }}"
+
+- name: Create temporary disk for containers if absent
+  community.routeros.api:
+    path: disk
+    add: "slot=tmp type=tmpfs"
+  when: routeros_disks.result | selectattr('slot', 'equalto', 'tmp') | list | length == 0
+
+- name: Configure temporary disk for containers
+  community.routeros.api_find_and_modify:
+    ignore_dynamic: false
+    path: disk
+    find:
+      slot: tmp
+    values:
+      type: tmpfs

 - name: Configure switch settings
  community.routeros.api_find_and_modify:
@@ -2,12 +2,12 @@
 - name: Preflight checks
  ansible.builtin.import_tasks: preflight.yml

- name: Base network configuration
-  ansible.builtin.import_tasks: base.yml
-
 - name: WAN and tunnel interfaces
  ansible.builtin.import_tasks: wan.yml

+- name: Base network configuration
+  ansible.builtin.import_tasks: base.yml
+
 - name: Hardware and platform tuning
  ansible.builtin.import_tasks: hardware.yml

@@ -32,15 +32,4 @@
    fail_msg: "RouterOS device-mode does not report container as enabled. Payload: {{ routeros_device_mode | to_nice_json }}"
    success_msg: "RouterOS device-mode confirms container=yes"

- name: Read configured disks
-  community.routeros.api_info:
-    path: disk
-  register: routeros_disks
-  check_mode: false

- name: Assert usb1 disk is present
-  ansible.builtin.assert:
-    that:
-      - (routeros_disks.result | selectattr('slot', 'equalto', 'usb1') | list | length) > 0
-    fail_msg: "Required disk slot usb1 is not present on router."
-    success_msg: "Required disk usb1 is present"
@@ -7,20 +7,49 @@
        disabled: false
        distance: 1
        dst-address: 100.64.0.0/10
-        gateway: 172.17.0.2
+        gateway: 172.20.0.2
        routing-table: main
        scope: 30
        suppress-hw-offload: false
        target-scope: 10
-      - disabled: false
+      - comment: GPON Monitor 1
+        disabled: false
        distance: 1
-        dst-address: 0.0.0.0/0
+        dst-address: 1.0.0.1/32
        gateway: pppoe-gpon
        routing-table: main
-        scope: 30
+        scope: 10
        suppress-hw-offload: false
        target-scope: 10
-        vrf-interface: pppoe-gpon
+      - comment: GPON Monitor 2
+        disabled: false
+        distance: 1
+        dst-address: 8.8.4.4/32
+        gateway: pppoe-gpon
+        routing-table: main
+        scope: 10
+        suppress-hw-offload: false
+        target-scope: 10
+      - comment: GPON Default 1
+        disabled: false
+        distance: 1
+        dst-address: 0.0.0.0/0
+        gateway: 1.0.0.1
+        check-gateway: ping
+        routing-table: main
+        scope: 30
+        suppress-hw-offload: false
+        target-scope: 11
+      - comment: GPON Default 2
+        disabled: false
+        distance: 2
+        dst-address: 0.0.0.0/0
+        gateway: 8.8.4.4
+        check-gateway: ping
+        routing-table: main
+        scope: 30
+        suppress-hw-offload: false
+        target-scope: 11
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible

@@ -32,6 +61,7 @@
        distance: 1
        dst-address: 2000::/3
        gateway: 2001:470:70:dd::1
+        check-gateway: ping
        scope: 30
        target-scope: 10
      - comment: Tailnet
@@ -19,6 +19,101 @@
    handle_absent_entries: ignore
    handle_entries_content: remove_as_much_as_possible

+# The RouterOS API can neither store multi-line script source (newlines
+# collapse into one line) nor evaluate the [/file/get ...] expression itself.
+# So we fetch the update logic as a .rsc file onto the router's flash, then run
+# a single-line bootstrap script (which the API CAN store) whose body RouterOS
+# evaluates natively: it builds the real, browsable, multi-line named script
+# from the file via [/file get ... contents]. The scheduler then runs that
+# named script by name (the upstream-intended design). The update logic stays
+# out of this repo entirely.
+- name: Download tailscale auto-update script to router
+  community.routeros.api:
+    path: tool
+    cmd: >-
+      fetch
+      url=https://gitea.lumpiasty.xyz/Lumpiasty/mikrotik-tailscale/raw/branch/main/routeros/update-tailscale.rsc
+      dst-path=update-tailscale.rsc
+      mode=https
+  changed_when: true
+  tags:
+    - tailscale-script
+
+- name: Build the named auto-update script from the fetched file
+  community.routeros.api:
+    path: system script
+    cmd: >-
+      add name=update-tailscale-bootstrap
+      source=":do { /system script remove update-tailscale } on-error={};
+      /system script add name=update-tailscale
+      comment=\"Check for mikrotik-tailscale image updates\"
+      source=[/file get update-tailscale.rsc contents]"
+  changed_when: true
+  tags:
+    - tailscale-script
+
+- name: Find bootstrap script id
+  community.routeros.api:
+    path: system script
+    extended_query:
+      attributes: [.id, name]
+      where:
+        - attribute: name
+          is: "=="
+          value: update-tailscale-bootstrap
+  register: routeros_bootstrap
+  changed_when: false
+  tags:
+    - tailscale-script
+
+- name: Run bootstrap to create the named auto-update script
+  community.routeros.api:
+    path: system script
+    cmd: "run .id={{ routeros_bootstrap.msg[0]['.id'] }}"
+  register: routeros_bootstrap_run
+  failed_when:
+    - routeros_bootstrap_run is failed
+    - "'interrupted' not in (routeros_bootstrap_run.msg | string)"
+  changed_when: true
+  tags:
+    - tailscale-script
+
+- name: Verify named auto-update script exists
+  community.routeros.api:
+    path: system script
+    extended_query:
+      attributes: [.id, name]
+      where:
+        - attribute: name
+          is: "=="
+          value: update-tailscale
+  register: routeros_named_script
+  failed_when: (routeros_named_script.msg | length) == 0
+  changed_when: false
+  tags:
+    - tailscale-script
+
+- name: Remove bootstrap script
+  community.routeros.api:
+    path: system script
+    remove: "{{ routeros_bootstrap.msg[0]['.id'] }}"
+  changed_when: true
+  tags:
+    - tailscale-script
+
+- name: Configure tailscale auto-update scheduler
+  community.routeros.api_modify:
+    path: system scheduler
+    data:
+      - name: update-tailscale
+        interval: 1d
+        on-event: /system script run update-tailscale
+        comment: Check for mikrotik-tailscale image updates
+    handle_absent_entries: remove
+    handle_entries_content: remove_as_much_as_possible
+  tags:
+    - tailscale-script
+
 - name: Configure service ports and service enablement
  community.routeros.api_find_and_modify:
    ignore_dynamic: false
@@ -8,7 +8,9 @@
        keepalive-timeout: 2
        name: pppoe-gpon
        password: "{{ routeros_pppoe_password }}"
-        use-peer-dns: true
+        # Using CoreDNS container with DNS64
+        use-peer-dns: false
+        add-default-route: false
        user: "{{ routeros_pppoe_username }}"
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible
@@ -29,13 +31,18 @@
  community.routeros.api_modify:
    path: interface veth
    data:
-      - address: 172.17.0.2/16,2001:470:61a3:500::1/64
+      - address: 172.20.0.2/24,2001:470:61a3:500::1/64
        container-mac-address: 7E:7E:A1:B1:2A:7C
        dhcp: false
-        gateway: 172.17.0.1
+        gateway: 172.20.0.1
        gateway6: 2001:470:61a3:500:ffff:ffff:ffff:ffff
        mac-address: 7E:7E:A1:B1:2A:7B
-        name: veth1
+        name: veth-tailscale
        comment: Tailscale container
+      - address: 172.20.0.3/24
+        dhcp: false
+        gateway: 172.20.0.1
+        name: veth-coredns
+        comment: CoreDNS container
    handle_absent_entries: remove
    handle_entries_content: remove_as_much_as_possible
@@ -14,6 +14,4 @@ openbao_fields:
    path: wan_pppoe
    username_key: username
    password_key: password
-  routeros_tailscale_container:
-    path: router_tailscale
-    container_password_key: container_password
+
@@ -18,7 +18,7 @@ spec:
  chart:
    spec:
      chart: authentik
-      version: 2026.5.0
+      version: 2026.5.3
      sourceRef:
        kind: HelmRepository
        name: authentik
@@ -7,7 +7,7 @@ metadata:
  name: gitea-shared-storage-lvmhdd
  namespace: openebs
 spec:
-  capacity: 10Gi
+  capacity: "21474836480"
  ownerNodeID: anapistula-delrosalae
  shared: "yes"
  thinProvision: "no"
@@ -20,7 +20,7 @@ metadata:
  name: gitea-shared-storage-lvmhdd
 spec:
  capacity:
-    storage: 10Gi
+    storage: 20Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
@@ -41,6 +41,6 @@ spec:
    - ReadWriteOnce
  resources:
    requests:
-      storage: 10Gi
+      storage: 20Gi
  storageClassName: hdd-lvmpv
  volumeName: gitea-shared-storage-lvmhdd
@@ -8,7 +8,7 @@ spec:
  interval: 24h
  url: https://github.com/usekaneo/kaneo.git
  ref:
-    tag: v2.7.5
+    tag: v2.7.7
  ignore: |
    # exclude all
    /*
@@ -16,7 +16,7 @@ spec:
    spec:
      containers:
        - name: caddy
-          image: caddy:2.11.3-alpine
+          image: caddy:2.11.4-alpine
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - mountPath: /etc/caddy
@@ -4,22 +4,23 @@ logToStdout: "both" # proxy and upstream

 macros:
  base_args: "--no-warmup --port ${PORT} --mlock --no-mmap"
-  common_args: "--fit-target 1536 --no-warmup --port ${PORT} --no-mmap"
+  common_args: "--fit-target 256 --no-warmup --port ${PORT} --no-mmap -tb 12 -t 6"
  cpu_args: "--no-warmup --port ${PORT} -ngl 0"
  ctx_64k: "--ctx-size 65536"
  ctx_128k: "--ctx-size 131072"
-  ctx_256k: "--ctx-size 262144"
+  ctx_256k: "--ctx-size 131072"
  qwen35_think_args:   "--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 -ctk q4_0 -ctv q4_0 --presence_penalty 1.5 --reasoning on"
  qwen35_nothink_args: "--temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.00 -ctk q4_0 -ctv q4_0 --presence_penalty 1.5 --reasoning off"
  qwen35_35b_heretic_mmproj: "--mmproj-url https://huggingface.co/unsloth/Qwen3.5-35B-A3B-GGUF/resolve/main/mmproj-F16.gguf --mmproj /root/.cache/llama.cpp/unsloth_Qwen3.5-35B-A3B-GGUF_mmproj-F16.gguf"
  qwen35_4b_heretic_mmproj: "--mmproj-url https://huggingface.co/unsloth/Qwen3.5-4B-GGUF/resolve/main/mmproj-F16.gguf --mmproj /root/.cache/llama.cpp/unsloth_Qwen3.5-4B-GGUF_mmproj-F16.gguf"
  gemma4_sampling: "--temp 1.0 --top-p 0.95 --top-k 64 -ctk q4_0 -ctv q4_0"
+  gemma4_nothink_sampling: "--temp 1.0 --top-p 0.95 --top-k 64 -ctk q4_0 -ctv q4_0 --reasoning off"

 hooks:
  on_startup:
    preload:
      - "Qwen3.5-0.8B-GGUF-nothink:Q4_K_XL"
-      - "whisper-small"
+      - "parakeet-tdt_ctc-1.1b"

 # matrix replaces groups (they are mutually exclusive).
 # The small 0.8B model runs alongside any LLM.
@@ -27,7 +28,7 @@ hooks:
 matrix:
  vars:
    q8: "Qwen3.5-0.8B-GGUF-nothink:Q4_K_XL"
-    stt: "whisper-small"
+    stt: "parakeet-tdt_ctc-1.1b"
    flux: "flux2-klein-4b:Q4_K_M"
    coder: "Qwen3-Coder-Next-GGUF:Q4_K_M"
    q35t: "Qwen3.5-35B-A3B-GGUF:Q4_K_M"
@@ -38,10 +39,24 @@ matrix:
    q4nt: "Qwen3.5-4B-GGUF-nothink:Q4_K_M"
    q4ht: "Qwen3.5-4B-heretic-GGUF:Q4_K_M"
    q4hnt: "Qwen3.5-4B-heretic-GGUF-nothink:Q4_K_M"
-    g26xl: "gemma-4-26B-A4B-it:UD-Q4_K_XL"
-    g26q2: "gemma-4-26B-A4B-it:UD-Q2_K_XL"
-    ge4xl: "unsloth/gemma-4-E4B-it-GGUF:UD-Q4_K_XL"
-    ge2xl: "unsloth/gemma-4-E2B-it-GGUF:UD-Q4_K_XL"
+    g26xl: "gemma-4-26B-A4B-it-qat:UD-Q4_K_XL"
+    g26xlnt: "gemma-4-26B-A4B-it-qat-nothink:UD-Q4_K_XL"
+    g26mtp: "gemma-4-26B-A4B-it-qat-mtp:UD-Q4_K_XL"
+    g26mtpnt: "gemma-4-26B-A4B-it-qat-mtp-nothink:UD-Q4_K_XL"
+    g26ht: "SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF:UD-Q4_K_XL"
+    g26hnt: "SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF-nothink:UD-Q4_K_XL"
+    g26hmtp: "SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF-mtp:UD-Q4_K_XL"
+    g26hmnt: "SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF-mtp-nothink:UD-Q4_K_XL"
+    ge4qat: "unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL"
+    ge4qatnt: "unsloth/gemma-4-E4B-it-qat-GGUF-nothink:UD-Q4_K_XL"
+    ge2qat: "unsloth/gemma-4-E2B-it-qat-GGUF:UD-Q4_K_XL"
+    ge2qatnt: "unsloth/gemma-4-E2B-it-qat-GGUF-nothink:UD-Q4_K_XL"
+    ge4mtp: "unsloth/gemma-4-E4B-it-qat-GGUF-mtp:UD-Q4_K_XL"
+    ge4mtpnt: "unsloth/gemma-4-E4B-it-qat-GGUF-mtp-nothink:UD-Q4_K_XL"
+    ge4ht: "llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF:Q4_K_M"
+    ge4hnt: "llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF-nothink:Q4_K_M"
+    ge4hmtp: "llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF-mtp:Q4_K_M"
+    ge4hmnt: "llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF-mtp-nothink:Q4_K_M"
    q36t: "unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_XL"
    q36nt: "unsloth/Qwen3.6-35B-A3B-GGUF-nothink:UD-Q4_K_XL"
    haut: "HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive:Q4_K_M"
@@ -54,10 +69,11 @@ matrix:

  sets:
    # any LLM can run alongside the small always-on model + STT + TTS (all CPU, no VRAM cost)
-    with_q8: "(coder | q35t | q35nt | q35ht | q35hnt | q4t | q4nt | q4ht | q4hnt | g26xl | g26q2 | ge4xl | ge2xl | q36t | q36nt | haut | haunt | mtpt | mtpnt) & q8 & stt"
+    with_q8: "(coder | q35t | q35nt | q35ht | q35hnt | q4t | q4nt | q4ht | q4hnt | g26xl | g26xlnt | g26mtp | g26mtpnt | g26ht | g26hnt | g26hmtp | g26hmnt | ge4qat | ge4qatnt | ge2qat | ge2qatnt | ge4mtp | ge4mtpnt | ge4ht | ge4hnt | ge4hmtp | ge4hmnt | q36t | q36nt | haut | haunt | mtpt | mtpnt) & q8 & stt"
    # FLUX runs alone — evicts everything including q8, but keeps STT for voice during image gen
    image_gen: "flux & stt"

+
 models:
  "Qwen3-Coder-Next-GGUF:Q4_K_M":
    cmd: |
@@ -151,38 +167,200 @@ models:
        ${qwen35_nothink_args}
        ${common_args}

-  "gemma-4-26B-A4B-it:UD-Q4_K_XL":
+  "gemma-4-26B-A4B-it-qat:UD-Q4_K_XL":
    cmd: |
      llama-server
-        -hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_XL \
+        -hf unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL \
        ${ctx_256k}
        ${gemma4_sampling}
        ${common_args}

-  "gemma-4-26B-A4B-it:UD-Q2_K_XL":
+  "gemma-4-26B-A4B-it-qat-nothink:UD-Q4_K_XL":
    cmd: |
      llama-server
-        -hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q2_K_XL \
+        -hf unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL \
+        ${ctx_256k}
+        ${gemma4_nothink_sampling}
+        ${common_args}
+
+  "gemma-4-26B-A4B-it-qat-mtp:UD-Q4_K_XL":
+    cmd: |
+      llama-server
+        -hf unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL \
+        --spec-draft-hf unsloth/gemma-4-26B-A4B-it-qat-GGUF:Q8_0-MTP \
+        --spec-type draft-mtp
+        --spec-draft-n-max 1
+        --swa-full
+        --kv-unified
+        --parallel 1
        ${ctx_256k}
        ${gemma4_sampling}
        ${common_args}

-  "unsloth/gemma-4-E4B-it-GGUF:UD-Q4_K_XL":
+  "gemma-4-26B-A4B-it-qat-mtp-nothink:UD-Q4_K_XL":
    cmd: |
      llama-server
-        -hf unsloth/gemma-4-E4B-it-GGUF:UD-Q4_K_XL \
+        -hf unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL \
+        --spec-draft-hf unsloth/gemma-4-26B-A4B-it-qat-GGUF:Q8_0-MTP \
+        --spec-type draft-mtp
+        --spec-draft-n-max 1
+        --swa-full
+        --kv-unified
+        --parallel 1
+        ${ctx_256k}
+        ${gemma4_nothink_sampling}
+        ${common_args}
+
+  "SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF:UD-Q4_K_XL":
+    cmd: |
+      llama-server
+        -hf SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF:UD-Q4_K_XL \
+        ${ctx_256k}
+        ${gemma4_sampling}
+        ${common_args}
+
+  "SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF-nothink:UD-Q4_K_XL":
+    cmd: |
+      llama-server
+        -hf SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF:UD-Q4_K_XL \
+        ${ctx_256k}
+        ${gemma4_nothink_sampling}
+        ${common_args}
+
+  # The heretic QAT repo does not ship an MTP drafter,
+  # so borrow the one from the non-heretic unsloth QAT repo.
+  "SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF-mtp:UD-Q4_K_XL":
+    cmd: |
+      llama-server
+        -hf SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF:UD-Q4_K_XL \
+        --spec-draft-hf unsloth/gemma-4-26B-A4B-it-qat-GGUF:Q8_0-MTP \
+        --spec-type draft-mtp
+        --spec-draft-n-max 1
+        --swa-full
+        --kv-unified
+        --parallel 1
+        ${ctx_256k}
+        ${gemma4_sampling}
+        ${common_args}
+
+  "SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF-mtp-nothink:UD-Q4_K_XL":
+    cmd: |
+      llama-server
+        -hf SC117/gemma-4-26B-A4B-it-qat-heretic-GGUF:UD-Q4_K_XL \
+        --spec-draft-hf unsloth/gemma-4-26B-A4B-it-qat-GGUF:Q8_0-MTP \
+        --spec-type draft-mtp
+        --spec-draft-n-max 1
+        --swa-full
+        --kv-unified
+        --parallel 1
+        ${ctx_256k}
+        ${gemma4_nothink_sampling}
+        ${common_args}
+
+  "unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL":
+    cmd: |
+      llama-server
+        -hf unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL \
        ${ctx_128k}
        ${gemma4_sampling}
        ${common_args}

-  "unsloth/gemma-4-E2B-it-GGUF:UD-Q4_K_XL":
+  "unsloth/gemma-4-E4B-it-qat-GGUF-nothink:UD-Q4_K_XL":
    cmd: |
      llama-server
-        -hf unsloth/gemma-4-E2B-it-GGUF:UD-Q4_K_XL \
+        -hf unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL \
+        ${ctx_128k}
+        ${gemma4_nothink_sampling}
+        ${common_args}
+
+  "unsloth/gemma-4-E2B-it-qat-GGUF:UD-Q4_K_XL":
+    cmd: |
+      llama-server
+        -hf unsloth/gemma-4-E2B-it-qat-GGUF:UD-Q4_K_XL \
        ${ctx_128k}
        ${gemma4_sampling}
        ${common_args}

+  "unsloth/gemma-4-E2B-it-qat-GGUF-nothink:UD-Q4_K_XL":
+    cmd: |
+      llama-server
+        -hf unsloth/gemma-4-E2B-it-qat-GGUF:UD-Q4_K_XL \
+        ${ctx_128k}
+        ${gemma4_nothink_sampling}
+        ${common_args}
+
+  "unsloth/gemma-4-E4B-it-qat-GGUF-mtp:UD-Q4_K_XL":
+    cmd: |
+      llama-server
+        -hf unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL \
+        --spec-draft-hf unsloth/gemma-4-E4B-it-qat-GGUF:Q8_0-MTP \
+        --spec-type draft-mtp
+        --spec-draft-n-max 1
+        --swa-full
+        --kv-unified
+        --parallel 1
+        ${ctx_128k}
+        ${gemma4_sampling}
+        ${common_args}
+
+  "unsloth/gemma-4-E4B-it-qat-GGUF-mtp-nothink:UD-Q4_K_XL":
+    cmd: |
+      llama-server
+        -hf unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL \
+        --spec-draft-hf unsloth/gemma-4-E4B-it-qat-GGUF:Q8_0-MTP \
+        --spec-type draft-mtp
+        --spec-draft-n-max 1
+        --swa-full
+        --kv-unified
+        --parallel 1
+        ${ctx_128k}
+        ${gemma4_nothink_sampling}
+        ${common_args}
+
+  "llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF:Q4_K_M":
+    cmd: |
+      llama-server
+        -hf llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF:Q4_K_M \
+        ${ctx_128k}
+        ${gemma4_sampling}
+        ${common_args}
+
+  "llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF-nothink:Q4_K_M":
+    cmd: |
+      llama-server
+        -hf llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF:Q4_K_M \
+        ${ctx_128k}
+        ${gemma4_nothink_sampling}
+        ${common_args}
+
+  "llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF-mtp:Q4_K_M":
+    cmd: |
+      llama-server
+        -hf llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF:Q4_K_M \
+        --spec-draft-hf unsloth/gemma-4-E4B-it-qat-GGUF:Q8_0-MTP \
+        --spec-type draft-mtp
+        --spec-draft-n-max 1
+        --swa-full
+        --kv-unified
+        --parallel 1
+        ${ctx_128k}
+        ${gemma4_sampling}
+        ${common_args}
+
+  "llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF-mtp-nothink:Q4_K_M":
+    cmd: |
+      llama-server
+        -hf llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic-GGUF:Q4_K_M \
+        --spec-draft-hf unsloth/gemma-4-E4B-it-qat-GGUF:Q8_0-MTP \
+        --spec-type draft-mtp
+        --spec-draft-n-max 1
+        --swa-full
+        --kv-unified
+        --parallel 1
+        ${ctx_128k}
+        ${gemma4_nothink_sampling}
+        ${common_args}
+
  "unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_XL":
    cmd: |
      llama-server
@@ -235,20 +413,14 @@ models:
      --parallel 1
      ${common_args}

-  # STT via whisper.cpp (Vulkan GPU on RX 580, always loaded, ~600MB VRAM)
-  # Model auto-downloaded by init container, see deployment.yaml
-  # Note: Vulkan whisper on AMD GPUs has known quality issues on some cards;
-  # if transcriptions come out as garbage/gibberish, add --no-gpu to fall back.
-  "whisper-small":
+  # STT via parakeet-server (parakeet.cpp OpenAI-compatible server, CPU, always loaded)
+  # Model downloaded on first start and cached under /root/.cache/parakeet.cpp/models
+  # parakeet-proxy.py sits in front to convert any audio format to WAV via ffmpeg,
+  # since parakeet-server only accepts real WAV but browsers send Ogg/Opus.
+  "parakeet-tdt_ctc-1.1b":
    checkEndpoint: none
    cmd: |
-      whisper-server
-        --port ${PORT}
-        -m /root/.cache/whisper/ggml-small.bin
-        --request-path /v1/audio
-        --inference-path /transcriptions
-        --convert
-        --threads 6
+      env PROXY_PORT=${PORT} FFMPEG_BIN=/root/.cache/ffmpeg/ffmpeg python3 /config/parakeet-proxy.py


  # Image generation via stable-diffusion.cpp (sd-server)
@@ -0,0 +1,227 @@
+#!/usr/bin/env python3
+"""
+Thin reverse proxy for parakeet-server.
+
+Accepts POST /v1/audio/transcriptions with any audio format,
+converts the audio to 16 kHz mono WAV via ffmpeg, then forwards
+the converted file to the real parakeet-server running on PARAKEET_PORT.
+
+Also proxies GET /health straight through.
+
+Usage:
+    PROXY_PORT=<port>  PARAKEET_PORT=<upstream>  python3 parakeet-proxy.py
+"""
+
+import http.server
+import io
+import os
+import subprocess
+import sys
+import tempfile
+import urllib.request
+import urllib.error
+
+PROXY_PORT    = int(os.environ.get("PROXY_PORT", "8080"))
+PARAKEET_PORT = PROXY_PORT + 1
+FFMPEG        = os.environ.get("FFMPEG_BIN", "ffmpeg")
+MODEL         = os.environ.get("PARAKEET_MODEL", "tdt_ctc-1.1b-q4_k.gguf")
+CACHE_DIR     = os.environ.get("PARAKEET_CACHE_DIR", "/root/.cache/parakeet.cpp/models")
+
+
+def convert_to_wav(data: bytes) -> bytes:
+    """Convert any audio bytes to 16 kHz mono PCM WAV via ffmpeg."""
+    with tempfile.NamedTemporaryFile(suffix=".input", delete=False) as inf:
+        inf.write(data)
+        inf_path = inf.name
+    out_path = inf_path + ".wav"
+    try:
+        subprocess.run(
+            [
+                FFMPEG, "-y",
+                "-i", inf_path,
+                "-ar", "16000",
+                "-ac", "1",
+                "-f", "wav",
+                out_path,
+            ],
+            check=True,
+            stdout=subprocess.DEVNULL,
+            stderr=subprocess.DEVNULL,
+        )
+        with open(out_path, "rb") as f:
+            return f.read()
+    finally:
+        os.unlink(inf_path)
+        if os.path.exists(out_path):
+            os.unlink(out_path)
+
+
+def parse_multipart(content_type: str, body: bytes):
+    """
+    Parse a multipart/form-data body.
+    Returns a dict of field_name -> (filename_or_None, content_type, data).
+    """
+    import email
+    from email import policy as email_policy
+
+    # email.parser needs the full MIME headers to parse multipart
+    raw = b"Content-Type: " + content_type.encode() + b"\r\n\r\n" + body
+    msg = email.message_from_bytes(raw, policy=email_policy.compat32)
+    parts = {}
+    for part in msg.get_payload():
+        cd = part.get("Content-Disposition", "")
+        name = None
+        filename = None
+        for item in cd.split(";"):
+            item = item.strip()
+            if item.startswith('name='):
+                name = item[5:].strip('"')
+            elif item.startswith('filename='):
+                filename = item[9:].strip('"')
+        if name is not None:
+            parts[name] = (filename, part.get_content_type(), part.get_payload(decode=True))
+    return parts
+
+
+def build_multipart(fields: dict) -> tuple[bytes, str]:
+    """
+    Build a multipart/form-data body from fields dict:
+      field_name -> (filename_or_None, content_type, data_bytes)
+    Returns (body_bytes, content_type_header_value).
+    """
+    boundary = b"----ParakeetProxyBoundary0xDEADBEEF"
+    body = b""
+    for name, (filename, ct, data) in fields.items():
+        body += b"--" + boundary + b"\r\n"
+        if filename:
+            body += (
+                f'Content-Disposition: form-data; name="{name}"; filename="{filename}"\r\n'
+            ).encode()
+        else:
+            body += f'Content-Disposition: form-data; name="{name}"\r\n'.encode()
+        body += f"Content-Type: {ct}\r\n\r\n".encode()
+        body += data + b"\r\n"
+    body += b"--" + boundary + b"--\r\n"
+    return body, f"multipart/form-data; boundary={boundary.decode()}"
+
+
+class ProxyHandler(http.server.BaseHTTPRequestHandler):
+    def log_message(self, fmt, *args):
+        print(f"[parakeet-proxy] {self.address_string()} - {fmt % args}", flush=True)
+
+    def do_GET(self):
+        if self.path == "/health":
+            self._forward_get("/health")
+        else:
+            self.send_response(404)
+            self.end_headers()
+
+    def do_POST(self):
+        if self.path.rstrip("/") == "/v1/audio/transcriptions":
+            self._handle_transcription()
+        else:
+            self.send_response(404)
+            self.end_headers()
+
+    def _forward_get(self, path):
+        try:
+            url = f"http://127.0.0.1:{PARAKEET_PORT}{path}"
+            with urllib.request.urlopen(url, timeout=5) as resp:
+                body = resp.read()
+                self.send_response(resp.status)
+                self.send_header("Content-Type", resp.headers.get("Content-Type", "application/json"))
+                self.end_headers()
+                self.wfile.write(body)
+        except Exception as e:
+            self.send_response(502)
+            self.end_headers()
+            self.wfile.write(str(e).encode())
+
+    def _handle_transcription(self):
+        length = int(self.headers.get("Content-Length", 0))
+        body = self.rfile.read(length)
+        ct = self.headers.get("Content-Type", "")
+
+        try:
+            fields = parse_multipart(ct, body)
+        except Exception as e:
+            self._error(400, f"failed to parse multipart: {e}")
+            return
+
+        if "file" not in fields:
+            self._error(400, "missing required field 'file'")
+            return
+
+        filename, file_ct, audio_data = fields["file"]
+
+        # Convert to WAV regardless of what we received
+        try:
+            wav_data = convert_to_wav(audio_data)
+        except subprocess.CalledProcessError:
+            self._error(400, "ffmpeg could not decode audio")
+            return
+        except Exception as e:
+            self._error(500, f"conversion error: {e}")
+            return
+
+        # Rebuild multipart with converted WAV, preserve other fields
+        new_fields = {}
+        for name, (fn, fct, data) in fields.items():
+            if name == "file":
+                new_fields[name] = ("recording.wav", "audio/wav", wav_data)
+            else:
+                new_fields[name] = (fn, fct, data)
+
+        new_body, new_ct = build_multipart(new_fields)
+
+        # Forward to parakeet-server
+        try:
+            url = f"http://127.0.0.1:{PARAKEET_PORT}/v1/audio/transcriptions"
+            req = urllib.request.Request(
+                url,
+                data=new_body,
+                headers={"Content-Type": new_ct},
+                method="POST",
+            )
+            with urllib.request.urlopen(req, timeout=300) as resp:
+                resp_body = resp.read()
+                self.send_response(resp.status)
+                self.send_header("Content-Type", resp.headers.get("Content-Type", "application/json"))
+                self.end_headers()
+                self.wfile.write(resp_body)
+        except urllib.error.HTTPError as e:
+            resp_body = e.read()
+            self.send_response(e.code)
+            self.send_header("Content-Type", e.headers.get("Content-Type", "application/json"))
+            self.end_headers()
+            self.wfile.write(resp_body)
+        except Exception as e:
+            self._error(502, f"upstream error: {e}")
+
+    def _error(self, code: int, msg: str):
+        body = f'{{"error":{{"message":"{msg}","type":"proxy_error"}}}}'.encode()
+        self.send_response(code)
+        self.send_header("Content-Type", "application/json")
+        self.end_headers()
+        self.wfile.write(body)
+
+
+if __name__ == "__main__":
+    proc = subprocess.Popen([
+        "parakeet-server",
+        "--host", "127.0.0.1",
+        "--port", str(PARAKEET_PORT),
+        "--model", MODEL,
+        "--cache-dir", CACHE_DIR,
+    ])
+    print(f"[parakeet-proxy] started parakeet-server pid={proc.pid} on :{PARAKEET_PORT}", flush=True)
+
+    server = http.server.HTTPServer(("0.0.0.0", PROXY_PORT), ProxyHandler)
+    print(f"[parakeet-proxy] listening on :{PROXY_PORT}", flush=True)
+    try:
+        server.serve_forever()
+    except KeyboardInterrupt:
+        pass
+    finally:
+        proc.terminate()
+        proc.wait()
@@ -18,7 +18,7 @@ spec:
    spec:
      initContainers:
        - name: download-whisper
-          image: ghcr.io/mostlygeek/llama-swap:unified-vulkan-2026-05-25
+          image: gitea.lumpiasty.xyz/lumpiasty/llama-swap:unified-vulkan-parakeet-2026-06-12
          command:
            - sh
            - -c
@@ -48,13 +48,16 @@ spec:
              mountPath: /root/.cache
      containers:
        - name: llama-swap
-          image: ghcr.io/mostlygeek/llama-swap:unified-vulkan-2026-05-25
+          image: gitea.lumpiasty.xyz/lumpiasty/llama-swap:unified-vulkan-parakeet-2026-06-12
          imagePullPolicy: IfNotPresent
          command:
            - llama-swap
          args:
            - --config=/config/config.yaml
            - --watch-config
+          env:
+            - name: RADV_EXPERIMENTAL
+              value: transfer_queue
          ports:
            - containerPort: 8080
              name: http
@@ -21,7 +21,7 @@ spec:
          # OpenAI-compatible Kokoro-FastAPI TTS server, CPU PyTorch backend.
          # Models baked into the image (no PVC needed).
          # v0.3.0 includes fix for per-request voice tensor memory leak (#459).
-          image: ghcr.io/remsky/kokoro-fastapi-cpu:v0.4.0
+          image: ghcr.io/remsky/kokoro-fastapi-cpu:v0.5.0
          ports:
            - containerPort: 8880
              name: http
@@ -13,3 +13,4 @@ configMapGenerator:
    namespace: llama
    files:
      - config.yaml=configs/config.yaml
+      - parakeet-proxy.py=configs/parakeet-proxy.py
@@ -15,7 +15,7 @@ spec:
    spec:
      initContainers:
        - name: prepare-home
-          image: alpine:3.23.4
+          image: alpine:3.24.1
          imagePullPolicy: IfNotPresent
          command:
            - /bin/sh
@@ -18,7 +18,7 @@ spec:
  chart:
    spec:
      chart: open-webui
-      version: 14.6.0
+      version: 14.8.0
      sourceRef:
        kind: HelmRepository
        name: open-webui
@@ -74,7 +74,17 @@ spec:
        value: "false"
      - name: OAUTH_MERGE_ACCOUNTS_BY_EMAIL
        value: "true"
-      # STT via whisper-server (routed through llama-swap)
+      - name: ENABLE_OAUTH_ROLE_MANAGEMENT
+        value: "true"
+      - name: OAUTH_ROLES_CLAIM
+        value: "groups"
+      - name: OAUTH_ADMIN_ROLES
+        value: "Admins"
+      - name: OAUTH_ALLOWED_ROLES
+        value: "Users"
+      - name: OAUTH_AUTO_REDIRECT
+        value: "true"
+      # STT via parakeet-server (routed through llama-swap)
      - name: AUDIO_STT_ENGINE
        value: "openai"
      - name: AUDIO_STT_OPENAI_API_BASE_URL
@@ -82,9 +92,7 @@ spec:
      - name: AUDIO_STT_OPENAI_API_KEY
        value: "ignored"
      - name: AUDIO_STT_MODEL
-        value: "whisper-small"
-      - name: AUDIO_STT_SUPPORTED_CONTENT_TYPES
-        value: "audio/wav,audio/wave"
+        value: "parakeet-tdt_ctc-1.1b"
      # TTS via OuteTTS (routed through llama-swap)
      # TTS via dedicated Kokoro server (CPU-only, separate pod)
      - name: AUDIO_TTS_ENGINE
@@ -18,7 +18,7 @@ spec:
  chart:
    spec:
      chart: woodpecker
-      version: 3.6.3
+      version: 3.6.4
      sourceRef:
        kind: HelmRepository
        name: woodpecker
@@ -50,6 +50,7 @@ spec:
        WOODPECKER_OPEN: "true"
        # Make lumpiasty admin
        WOODPECKER_ADMIN: GiteaAdmin
+        WOODPECKER_PLUGINS_PRIVILEGED: woodpeckerci/plugin-docker-buildx

      createAgentSecret: true

@@ -0,0 +1,492 @@
+# App deployment guidelines
+
+This document summarizes current guidelines, requirements, common patterns, and standards that newly deployed apps should meet.
+
+## Structure
+
+Each app on cluster should be contained in its own kustomization living in subdirectory under [apps](/apps) and imported from main [apps kustomization](/apps/kustomization.yaml). Apps that provide infrastructural services belong to [infra](/infra). Few examples:
+
+- **Open WebUI**: Web app, belongs in [apps/openwebui](/apps/openwebui/) together with its direct and unique dependencies eg. database
+- **llama-swap** (llama.cpp + whisper + stablediffusion): Inference server, service used by other deployments on cluster but does not manages cluster, belongs in [apps/llama](/apps/llama/)
+- **kokoro**: Text to speech inference server, also service used by other deployments, I consider it closely related to llama-swap, so due to arbitrary decision, keeping it together with llama-swap under [apps/llama](/apps/llama/)
+- **crawl4ai**: Web scraper, another service used only by other apps, belongs in [apps/crawl4ai](/apps/crawl4ai/)
+- **Gitea**: Code forge, despite being essential for overall architecture (holding cluster's code) is not a core cluster software, belongs in [apps/gitea](/apps/gitea/)
+- **Woodpecker**: Continous Integration system, belongs in [apps/woodpecker](/apps/woodpecker/)
+- **Cilium**: Kubernetes CNI, core cluster functionality, belongs in [infra/controllers/cilium.yaml](/infra/controllers/cilium.yaml)
+- **Nginx Ingress Controller**: Provides ingress kubernetes functionality, belongs in [infra/controllers/nginx-ingress.yaml](/infra/controllers/nginx-ingress.yaml)
+- **CloudNativePG**: Kubernetes PostgreSQL operator, belongs in [infra/controllers/cloudnative-pg.yaml](/infra/controllers/cloudnative-pg.yaml)
+- **OpenBao** Secret storage and Kubernetes operator, belongs in [infra/controllers/openbao.yaml](/infra/controllers/openbao.yaml)
+
+Kustomizations are reconciled on `git push` by flux running on cluster, triggered by [Woodpecker job](/.woodpecker/flux-reconcile-source.yaml). App Kustomization should import all resources related to app in `kustomization.yaml`:
+
+```yaml
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - namespace.yaml
+  - pvc.yaml
+  - release.yaml
+```
+
+## Namespace
+
+Each app kustomization should have its own kubernetes namespace to contain all resources related to app in `namespace.yaml`:
+
+```yaml
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: immich
+```
+
+## Helm charts
+
+If app is distributed via Helm chart, you can deploy it using flux HelmRepository and HelmRelease resources like in following example:
+
+```yaml
+---
+apiVersion: source.toolkit.fluxcd.io/v1
+kind: HelmRepository
+metadata:
+  name: secustor
+  namespace: immich
+spec:
+  interval: 24h
+  url: https://secustor.dev/helm-charts
+---
+apiVersion: helm.toolkit.fluxcd.io/v2
+kind: HelmRelease
+metadata:
+  name: immich
+  namespace: immich
+spec:
+  interval: 30m
+  chart:
+    spec:
+      chart: immich
+      version: 1.2.6
+      sourceRef:
+        kind: HelmRepository
+        name: secustor
+  values:
+    <values>
+```
+
+If the app does not have a helm repository, but helm chart is available in git repository directly in repository, you can make use of it using GitRepository flux source:
+
+```yaml
+---
+apiVersion: source.toolkit.fluxcd.io/v1
+kind: GitRepository
+metadata:
+  name: kaneo
+  namespace: kaneo
+spec:
+  interval: 24h
+  url: https://github.com/usekaneo/kaneo.git
+  ref:
+    tag: v2.7.5
+  ignore: |
+    # exclude all
+    /*
+    # include charts directory
+    !/charts/
+```
+
+You can use third-party helm charts to deploy applications, consider this possibility if:
+
+- There is no official helm chart for the application
+- The official helm chart is unmaintained
+- The official helm chart is using glaring bad practices
+- The official helm chart is missing configuration options for what we need
+
+When deciding which helm chart to use, watch out for following things in particular:
+
+- Development activity, stability, maturity
+- Whether the app deployed by chart is up to date - automated updates are large bonus
+- Unresolved / breaking issues
+- Configurability, can we configure things we need, disable undesired features
+
+When configuring Helm chart, keep in mind:
+- Do not use bundled PVCs, bring our own one or at least configure chart to bind it to manually created `PersistentVolume` according to [Data / PVCs pattern](#data--pvcs-pattern)
+- Do not use bundled Postgres database unless the chart is using CloudNativePG's Cluster resource, bring our own one using [Postgres operator](#postgres-operator)
+- do not
+
+## Bare Kubernetes deployments
+
+If:
+
+- the app is not packaged as a helm chart or
+- it would be simpler to deploy it without package (for example custom privileged pod with access to gpu) or
+- the app is so simple it doesn't make sense to make helm package it (for example, simple http proxy that alters headers or stateless single-binary app) or
+- for any other reason it would make more sense to skip helm
+
+You can deploy app skipping helm chart and just create raw Kubernetes manifests like Deployment, StatefulSet and other supporting resources like ConfigMap, Service, Ingress directly.
+
+## Data / PVCs pattern
+
+Data are stored on local disk of node using OpenEBS LVM LocalPV. To create a persistent volume, use following example:
+
+```yaml
+---
+apiVersion: local.openebs.io/v1alpha1
+kind: LVMVolume
+metadata:
+  labels:
+    kubernetes.io/nodename: anapistula-delrosalae
+  name: immich-library-lvmhdd
+  namespace: openebs
+spec:
+  capacity: 150Gi
+  ownerNodeID: anapistula-delrosalae
+  shared: "yes"
+  thinProvision: "no"
+  vgPattern: ^openebs-hdd$
+  volGroup: openebs-hdd
+---
+kind: PersistentVolume
+apiVersion: v1
+metadata:
+  name: immich-library-lvmhdd
+spec:
+  capacity:
+    storage: 150Gi
+  accessModes:
+    - ReadWriteOnce
+  persistentVolumeReclaimPolicy: Retain
+  storageClassName: hdd-lvmpv
+  volumeMode: Filesystem
+  csi:
+    driver: local.csi.openebs.io
+    fsType: btrfs
+    volumeHandle: immich-library-lvmhdd
+---
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: library-lvmhdd
+  namespace: immich
+spec:
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: 150Gi
+  storageClassName: hdd-lvmpv
+  volumeName: immich-library-lvmhdd
+```
+
+Create LVMVolume and PersistentVolume resources manually and **do not** rely on automatic scheduling of PVCs because we want created LVM LVs on disk to have deterministic names and be reused if already exist on disk, which scheduler does not give us. There are two LVM storage classes:
+
+- **hdd-lvmpv**, volume group: openebs-hdd, use for bulk data, like media library
+- **ssd-lvmpv**, volume group: openebs-ssd, use for small datasets that benefit from quick storage access like databases, state data etc.
+
+When deciding the size of the volume, make minimal prediction, starting with 1GiB if you do not predict app to use much disk space.
+
+## Vault secrets
+
+There is OpenBao installed on cluster that manages access to secrets. The KV2 secret engine is mounted at `secret`, use it to store static secrets like API keys to external services, passwords and other entries you do not want to keep in plaintext in git repository.
+
+To access the KV secrets on cluster, use Vault Secrets Operator installed on cluster, which provides `VaultStaticSecret` custom resource that syncs a path from OpenBao to Kubernetes `Secret` object.
+
+```yaml
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: llama-proxy
+  namespace: llama
+---
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultAuth
+metadata:
+  name: llama
+  namespace: llama
+spec:
+  method: kubernetes
+  mount: kubernetes
+  kubernetes:
+    role: llama-proxy
+    serviceAccount: llama-proxy
+---
+apiVersion: secrets.hashicorp.com/v1beta1
+kind: VaultStaticSecret
+metadata:
+  name: llama-api-key
+  namespace: llama
+spec:
+  type: kv-v2
+
+  mount: secret
+  path: ollama
+
+  destination:
+    create: true
+    name: llama-api-key
+    type: Opaque
+    transformation:
+      excludeRaw: true
+
+  vaultAuthRef: llama
+```
+
+To give access to specified secret for given k8s ServiceAccount, you need to create kubernetes auth role and policy. Create a kubernetes auth role named `llama-proxy`, by creating file `vault/kubernetes-auth-roles/llama-proxy.yaml`:
+
+```yaml
+bound_service_account_names:
+  - llama-proxy
+bound_service_account_namespaces:
+  - llama
+token_policies:
+  - ollama
+```
+
+Create policy named `ollama` by creating file `vault/policy/ollama.hcl`:
+
+```hcl
+path "secret/data/ollama" {
+    capabilities = ["read"]
+}
+```
+
+Once these files are created, ask operator to reconcile OpenBao configuration and create required secret.
+
+## Postgres operator
+
+There is CloudNativePG operator installed on cluster that manages databases of applications running on cluster. You can create Postgres database by creating `Cluster` resource:
+
+```yaml
+---
+apiVersion: postgresql.cnpg.io/v1
+kind: Cluster
+metadata:
+  name: kaneo-db
+  namespace: kaneo
+spec:
+  instances: 1
+
+  storage:
+    pvcTemplate:
+      storageClassName: ssd-lvmpv
+      resources:
+        requests:
+          storage: 10Gi
+      volumeName: kaneo-db-1
+
+```
+
+Create a `PersistentVolume` and `LVMVol` resources manually but **do not** create `PersistentVolumeClaim`, CloudNativePG will create one on its own referencing `PersistentVolume` specified in `volumeName`. Do not replicate the database, there is only one node in the cluster currently. The `Cluster` resource will automatically create secret, use it to configure app:
+
+```
+Name:         kaneo-db-app
+Namespace:    kaneo
+Labels:       app.kubernetes.io/managed-by=cloudnative-pg
+              cnpg.io/cluster=kaneo-db
+              cnpg.io/reload=true
+              cnpg.io/userType=app
+Annotations:  cnpg.io/operatorVersion: 1.29.1
+
+Type:  kubernetes.io/basic-auth
+
+Data
+====
+dbname:         3 bytes
+fqdn-jdbc-uri:  145 bytes
+fqdn-uri:       126 bytes
+host:           11 bytes
+jdbc-uri:       127 bytes
+password:       64 bytes
+pgpass:         90 bytes
+port:           4 bytes
+uri:            108 bytes
+user:           3 bytes
+username:       3 bytes
+```
+
+## LoadBalancers
+
+You can expose installed app to the Internet using Cilium's LoadBalancer configured on cluster:
+
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+  name: teamspeak3
+  namespace: ispeak3
+spec:
+  selector:
+    app: teamspeak3
+  ports:
+  - name: voice
+    protocol: UDP
+    port: 9987
+    targetPort: 9987
+  - name: filetransfer
+    protocol: TCP
+    port: 30033
+    targetPort: 30033
+  type: LoadBalancer
+  externalTrafficPolicy: Local
+  ipFamilyPolicy: PreferDualStack
+```
+
+IPv6 will be directly reachable from the internet by its assigned address, for IPv4 currently you need to configure port forward on router in `ansible/roles/routeros/firewall.yml`, that step is not yet automated. The assigned internal IP will be known after manifests are applied on cluster. For this reason, there is no ExternalDNS configured yet, if you need a DNS name, ask the operator to configure DNS name for LoadBalancer. Assign names from lumpiasty.xyz subdomains (eg. kaneo.lumpiasty.xyz) unless explicitly requested. Do not use LoadBalancer for exposing HTTP applications, use Ingress instead.
+
+## Ingress
+
+You can expose HTTP applications using NGINX Ingress Controller:
+
+```yaml
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  namespace: llama
+  name: llama
+  annotations:
+    cert-manager.io/cluster-issuer: letsencrypt
+    acme.cert-manager.io/http01-edit-in-place: "true"
+    nginx.ingress.kubernetes.io/proxy-buffering: "false"
+    nginx.ingress.kubernetes.io/proxy-read-timeout: 30m
+    nginx.ingress.kubernetes.io/proxy-body-size: 8m
+spec:
+  ingressClassName: nginx-ingress
+  rules:
+    - host: llama.lumpiasty.xyz
+      http:
+        paths:
+          - backend:
+              service:
+                name: llama-proxy
+                port:
+                  number: 80
+            path: /
+            pathType: Prefix
+  tls:
+    - hosts:
+        - llama.lumpiasty.xyz
+      secretName: llama-ingress
+```
+
+TLS certificates are automatically issued for subdomains of lumpiasty.xyz using cert-manager. DNS name assignment is not automatic yet, ask operator to create DNS name for ingress resources.
+
+## Keeping app up to date
+
+There is a Renovate job configured for this repository as [Woodpecker job](/.woodpecker/renovate.yaml) to keep applications up to date. Renovate automatically keeps track of:
+
+- Docker images specified in Kubernetes manifests like Deployment, StatefulSet etc
+- HelmRelease versions
+- GitRepository tags
+
+To make Renovate automatically update applications, always specify full versions of docker images or helm chart release. If you use ambigous tags, renovate will not have chance to update and the cluster will never download new image because this tag already existed on node. **Do not** use:
+
+- latest (or its variants like stable, current, main, master current)
+- "Sliding" versions, like 1 or 1.2 that point at 1.2.1 currently and will change image it points at when version 1.2.2 is released
+
+As a last resort if the application does not publish stable image tags, pin digest of image.
+
+Renovate may require custom configuration if:
+
+- App is using non-standard versioning schema
+
+  Example app versioned by date (unified-vulkan-2026-01-01), renovate.json:
+  
+  ```json
+    {
+        "matchDatasources": ["docker"],
+        "matchPackageNames": ["ghcr.io/mostlygeek/llama-swap"],
+        "versioning": "regex:^unified-vulkan-(?<major>\\d{4})-(?<minor>\\d{2})-(?<patch>\\d{2})$",
+        "automerge": true,
+        "automergeType": "pr",
+        "platformAutomerge": true
+    }
+  ```
+
+- Docker image tag is specified in non-standard field that Renovate may not recognise automatically such as Helm values
+  
+  Example app with non-standard image selected in helm values instead of image's default (which is latest in this chart):
+  ```yaml
+    values:
+      kaneo:
+        image:
+          tag: "2.7.3"  # renovate: depName=ghcr.io/usekaneo/kaneo registryUrl=https://ghcr.io
+  ```
+
+Renovate is configured so it automatically merges patch versions, other updates are created as pull requests to be manually reviewed and merged unless explicitly desired on per case basis.
+
+## SSO / OIDC / Authentik
+
+There is an Authentik running on cluster providing SSO for applications. Configure user-facing apps to utilize it correctly.
+
+Authentik supports following protocols:
+
+- OAuth2 / OpenID Connect
+- SAML
+- Radius
+- LDAP
+- SCIM
+
+Currently, there is no Authentik configuration in code, ask operator to create application in the UI and save OAuth id and secret in OpenBao under `secret/authentik/<app>`. Authentik provides discovery URL for OAuth applications: `https://authentik.lumpiasty.xyz/application/o/<app slug>/.well-known/openid-configuration`.
+
+Configure the app to disable guest access, built-in registration and automatically register unprivileged users with `user` role and privileged users with `admin` role as the app allows.
+
+## Privileged apps
+
+Some apps require direct access to devices, like GPU. There are no specific operators yet, apps that require access to GPU are simply launched as privileged pods, example:
+
+```yaml
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: llama-swap
+  namespace: llama
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: llama-swap
+  template:
+    metadata:
+      labels:
+        app: llama-swap
+    spec:
+      containers:
+        - name: llama-swap
+          volumeMounts:
+            - mountPath: /dev/kfd
+              name: kfd
+            - mountPath: /dev/dri
+              name: dri
+          securityContext:
+            privileged: true
+      volumes:
+        - name: kfd
+          hostPath:
+            path: /dev/kfd
+            type: CharDevice
+        - name: dri
+          hostPath:
+            path: /dev/dri
+            type: Directory
+```
+
+Creating of such pods is forbidden unless explicitly allowed in Talos config:
+
+```yaml
+# CSI driver requirement
+cluster:
+  apiServer:
+    admissionControl:
+      - name: PodSecurity
+        configuration:
+          apiVersion: pod-security.admission.config.k8s.io/v1beta1
+          kind: PodSecurityConfiguration
+          exemptions:
+            namespaces:
+              - llama
+```
+
+Create the patch like this under `talos/patches/<app>.patch`, add it to `gen-talos-config` target in Makefile and ask operator to apply reconcile Talos config to allow privileged pods in specified namespace.
@@ -0,0 +1,13 @@
+<svg width="136" height="136" viewBox="0 0 136 136" fill="none" xmlns="http://www.w3.org/2000/svg">
+<g clip-path="url(#clip0_137_2)">
+<rect width="136" height="136" fill="#141414"/>
+<path fill-rule="evenodd" clip-rule="evenodd" d="M62.6855 103.724C58.5716 104.595 56.0001 103.265 56 101.66L56 70.0264C56 69.8606 56.0032 69.686 56.0088 69.5069C56.0039 69.3848 56.001 69.249 56.001 69.0977L56.001 37.9444C56.015 36.6524 56.2588 35.7449 59.2588 35.1094L73.3145 32.2764C77.4285 31.405 80 32.7365 80 34.3408L80 65.9746C80 66.1409 79.9978 66.3155 79.9922 66.4951C79.997 66.6169 79.999 66.7526 79.999 66.9033L79.999 98.0567C79.9849 99.3483 79.7408 100.256 76.7412 100.892L62.6855 103.724Z" fill="#F5F5F5"/>
+<path fill-rule="evenodd" clip-rule="evenodd" d="M30.6855 111.723C26.5716 112.594 24.0001 111.264 24 109.659L24 78.0244C24 77.8588 24.0032 77.6848 24.0088 77.5059C24.0039 77.3838 24.001 77.248 24.001 77.0967L24.001 45.9434C24.015 44.6514 24.2588 43.7439 27.2588 43.1084L41.3145 40.2754C45.4285 39.404 48 40.7355 48 42.3399L48 73.9737C48 74.1399 47.9978 74.3146 47.9922 74.4942C47.997 74.6159 47.999 74.7517 47.999 74.9024L47.999 106.056C47.9849 107.347 47.7408 108.255 44.7412 108.891L30.6855 111.723Z" fill="#F5F5F5"/>
+<path fill-rule="evenodd" clip-rule="evenodd" d="M105.314 24.2754C109.428 23.404 112 24.7355 112 26.3398V37.1845L94.0576 60.5019L111.999 82.7802V90.0576C111.985 91.3492 111.741 92.2571 108.741 92.8925L94.6855 95.7246C90.5717 96.596 88.0002 95.2654 88 93.6611V62.0254C88 61.8598 88.0032 61.6856 88.0088 61.5068C88.0039 61.3848 88.001 61.2488 88.001 61.0976V29.9433C88.0151 28.6516 88.2591 27.7438 91.2588 27.1084L105.314 24.2754Z" fill="#F5F5F5"/>
+</g>
+<defs>
+<clipPath id="clip0_137_2">
+<rect width="136" height="136" fill="white"/>
+</clipPath>
+</defs>
+</svg>
@@ -0,0 +1,110 @@
+# CoreDNS resolver
+
+## Goal
+
+Replace the RouterOS built-in DNS forwarder with a CoreDNS container for
+configurability, and suppress IPv6 (AAAA) resolution by default to keep traffic
+on IPv4.
+
+## Background
+
+The ISP provides no native IPv6 — only a Hurricane Electric (HE) tunnel
+(`2001:470:61a3::/48`). HE addresses fall in ranges some sites flag as
+datacenter/bot traffic, producing endless CAPTCHAs. The goal is to prefer IPv4
+egress while keeping IPv6 available for our own services and any domain
+explicitly trusted over IPv6.
+
+## What this is NOT (and why)
+
+An earlier iteration used **DNS64 + NAT64 (Tayga)** to force traffic through
+IPv4. It was removed:
+
+- **Performance**: Tayga is a userspace translator with no hardware offload.
+  Every translated packet crossed RouterOS twice (v6 in, v4 out) plus a
+  userspace hop, capping throughput at ~250 Mbps against a 1 Gbps line.
+- **SPOF**: two containers (CoreDNS + Tayga) in the datapath of nearly all
+  traffic on a router whose native forwarder had been rock-solid.
+- **Architectural inversion**: NAT64 exists to let IPv6-only clients reach IPv4.
+  We don't want IPv6 egress at all — using NAT64 to avoid IPv6 was solving the
+  problem backwards.
+
+Plain AAAA suppression in CoreDNS achieves the same IPv4-preferred outcome with
+zero datapath overhead — DNS is the only thing touched, packet forwarding stays
+on the RouterOS fastpath at line rate.
+
+The full account of the NAT64/IPv6-mostly attempt and why it was abandoned is in
+[nat64-dns64-postmortem.md](./nat64-dns64-postmortem.md).
+
+## How it works
+
+CoreDNS runs as a single container (`172.20.0.3`), reachable from RouterOS DNS
+which forwards client queries to it. The [Corefile](../mikrotik/coredns/Corefile)
+has three server blocks:
+
+1. **`lumpiasty.xyz`** — our own zone. Forwards normally, keeps AAAA, so internal
+   services reachable over the HE prefix resolve to their real IPv6 addresses.
+2. **`.` (default)** — forwards everything else, but a `template IN AAAA` block
+   returns empty NOERROR for all AAAA queries, so clients fall back to IPv4 and
+   avoid the HE tunnel's flagged egress. A records and all other types pass
+   through untouched.
+
+The whitelist is implemented as a reusable `(aaaa_allowed)` snippet imported by
+zones that should keep AAAA. To trust another domain over IPv6, add a server
+block for it that imports `aaaa_allowed`.
+
+### Why suppression, not NXDOMAIN
+
+The AAAA template returns NOERROR with an empty answer (NODATA), not NXDOMAIN.
+This is correct: the name exists, it just has no (advertised) AAAA. Clients
+treat it as "no IPv6 address" and use the A record. Returning NXDOMAIN would
+wrongly imply the name doesn't exist and break the A lookup.
+
+## Future improvement
+
+The current global-suppress-plus-whitelist is coarse: a domain that is genuinely
+IPv6-only (no A record) and not whitelisted becomes unreachable. The intended
+end state is a plugin that suppresses AAAA only when the domain also has an A
+record, so IPv6-only destinations keep working without manual whitelisting. No
+in-tree CoreDNS plugin does this today.
+
+## Custom image
+
+Built from source with a minimal plugin set (`errors`, `log`, `health`,
+`template`, `cache`, `forward`, `reload`) instead of the default ~40, producing
+a ~6-8 MB image. The `dns64` plugin is no longer compiled in.
+
+Source: [`mikrotik/coredns/`](../mikrotik/coredns/). Built by Woodpecker
+([`.woodpecker/coredns-build.yaml`](../.woodpecker/coredns-build.yaml)) on pushes
+touching `mikrotik/coredns/**`, pushed to `gitea.lumpiasty.xyz/lumpiasty/coredns-mikrotik`.
+
+## RouterOS integration
+
+- `/ip/dns servers=172.20.0.3` — RouterOS forwards client queries to CoreDNS
+- RDNSS in RA (`/ipv6/nd dns=...` on vlan2/vlan5) advertises an IPv6 resolver
+  (the router's per-VLAN address) to dual-stack clients; RouterOS DNS relays to
+  CoreDNS
+- No DHCP option 108, no PREF64 — those belonged to the removed IPv6-mostly setup
+
+## Pitfalls learned (kept for reference)
+
+These were hit during the NAT64 era and the migration; some still apply:
+
+1. **RouterOS static FWD entries corrupt NXDOMAIN.** A `type=FWD match-subdomain=yes`
+   entry returns NOERROR/empty instead of relaying NXDOMAIN. Combined with
+   `ndots:5` and kubernetes pod search domains, `getaddrinfo` stops at the first
+   search-suffixed NODATA candidate and never tries the absolute name — apps fail
+   with `ENOTFOUND` for valid hostnames while `nslookup` (absolute query) works.
+   Our own zone is therefore handled in the Corefile, not via a RouterOS FWD
+   entry. RouterOS DNS does plain forwarding only (plus the Tailscale `ts.net`
+   FWD, which is acceptable as its subdomains genuinely don't exist publicly).
+2. **`advertise-dns=no` on new ND entries.** RouterOS creates per-interface
+   `ipv6 nd` entries with `advertise-dns=no`, suppressing the RDNSS option even
+   when a static `dns=` list is set. Must be enabled explicitly.
+3. **Per-interface ND entries must be created, not modified.** Only the
+   `interface=all` default ships out of the box; `api_find_and_modify` matching a
+   specific interface silently matches nothing. Use `api_modify`.
+
+Verification: `rdisc6` (NixOS package `ndisc6`) dumps RA contents. The CoreDNS
+`log` plugin output is visible via `/log print` on the router (container
+`logging=yes`) and shows the rcode CoreDNS returned — comparing it to what the
+client received isolates which hop corrupts a response.
@@ -84,9 +84,10 @@ subnets would fail routing lookup with "net unreachable" without it.

 | Destination | Source | Distance | Active when |
 |---|---|---|---|
-| `0.0.0.0/0` | static via `pppoe-gpon` | 1 | GPON up |
+| `1.0.0.1/32`, `8.8.4.4/32` | static via `pppoe-gpon` | 1 | always |
+| `0.0.0.0/0` | static via `1.0.0.1`, `8.8.4.4` (recursive) | 1, 2 | GPON ping check succeeds |
 | `0.0.0.0/0` | BGP from D-Link via `192.168.6.2` | 200 | wwan up on D-Link |
-| `2000::/3` | static via `sit1` (HE tunnel) | 1 | sit1 active (HE tunnel works) |
+| `2000::/3` | static via `2001:470:70:dd::1` (HE tunnel) | 1 | HE tunnel ping check succeeds |
 | `2000::/3` | BGP from D-Link via `2001:470:61a3:600::2` | 200 | wwan up on D-Link |

 RouterOS distance comparison is straightforward: distance 1 always wins
@@ -136,11 +137,12 @@ preferred route for D-Link's own traffic.
 - **wwan modem goes down** → BIRD2 device protocol detects wwan0 down →
  static `lte_default` / `lte_default6` become unreachable → BGP withdraws
  announcements → CRS removes BGP-learned default
- **GPON drops** → `pppoe-gpon` interface down → CRS distance-1 default
-  route inactive → distance-200 BGP route activates → CRS withdraws its
-  default-originate announcement to D-Link (since no default is installed
-  any more) → D-Link's kernel default-via-CRS is removed → D-Link uses
-  wwan kernel default → traffic flows from CRS via vlan6 → D-Link → wwan
+- **GPON drops or blackholes** → recursive ping checks (1.0.0.1, 8.8.4.4) over `pppoe-gpon`
+  fail (takes ~20s: 10s ping interval + 10s timeout) → CRS distance-1/2 default routes inactive → distance-200 BGP route
+  activates → CRS withdraws its default-originate announcement to D-Link (loop
+  prevention prevents reflecting D-Link's own route) → D-Link's kernel
+  default-via-CRS is removed → D-Link uses wwan kernel default → traffic flows
+  from CRS via vlan6 → D-Link → wwan

 All transitions are automatic and driven by interface state. No active
 probing (Netwatch / mwan3), no scripts toggling routes.
@@ -241,6 +243,16 @@ QMI initialization within ~1 second.

 Full investigation: see [wwan-bm806c-qmi-workaround.md](./wwan-bm806c-qmi-workaround.md).

+## Multi-WAN Stale Connection Tracking
+
+When the routing table fails over from GPON to LTE (or vice versa), RouterOS does not automatically clear existing connection tracking entries. If an established TCP/UDP connection is routed out the new WAN interface, it retains the NAT translation state (source IP) of the old WAN interface. The packet is sent to the ISP with the wrong source IP and is silently dropped, causing clients (like Tailscale) to hang for minutes until their internal sockets time out.
+
+To solve this purely declaratively without scripts or blanket connection flushes, the `forward` chain is configured to "fast-fail" these shifted connections:
+
+1. Connections are marked with their egress WAN upon establishment (`wan-gpon` or `wan-lte`) via the `mangle` table.
+2. If an established connection with a `wan-gpon` mark attempts to route out `vlan6` (LTE), or a `wan-lte` mark routes out `pppoe-gpon`, it is explicitly rejected (`tcp-reset` for TCP, `icmp-network-unreachable` for UDP) before reaching the NAT table.
+3. This rejection immediately signals the client OS that the route is dead, forcing the application (Tailscale, SIP clients, etc.) to instantly close the socket and establish a new one, which successfully binds to the new WAN interface and NAT state.
+
 ## Implementation files

 | File | Role |
@@ -0,0 +1,136 @@
+# Postmortem: NAT64 / IPv6-mostly attempt
+
+A record of an architecture that was built, run for ~2 days, and removed. Kept
+so the reasoning isn't re-discovered the hard way. For the current DNS setup see
+[coredns.md](./coredns.md); for network overview see [network.md](./network.md).
+
+## The original problem
+
+The ISP provides no native IPv6 — only a Hurricane Electric (HE) 6in4 tunnel
+(`2001:470:61a3::/48`). HE address ranges are widely classified as
+datacenter/hosting space, so some sites (Google, Cloudflare-fronted services,
+various login flows) treat IPv6 traffic from them as bot/VPN traffic: endless
+CAPTCHAs, "unusual traffic" interstitials, or outright blocks. IPv4 egress
+(the ISP's residential PPPoE address) is unaffected.
+
+The goal: keep using the network normally without IPv6 triggering these flags,
+while still wanting some IPv6 (e.g. inbound to self-hosted services).
+
+## What was built
+
+An **IPv6-mostly** network (RFC 8925) with **DNS64 + NAT64**, intended to push
+egress onto IPv4 while presenting IPv6 to clients:
+
+- **CoreDNS container** with the `dns64` plugin (`translate_all`): synthesized
+  `64:ff9b::/96` AAAA records from A records for *all* names, so even dual-stack
+  destinations resolved to a NAT64 address.
+- **Tayga container** (`ghcr.io/apalrd/tayga-nat64`): stateless NAT64 translator.
+  IPv6 traffic to `64:ff9b::/96` was routed to it, translated to IPv4, and
+  masqueraded out the GPON PPPoE interface. So all "IPv6" egress actually left
+  as IPv4 on the residential address — bypassing the HE tunnel and its flagging.
+- **RouterOS RA + DHCP**: DHCP option 108 (IPv6-only preferred) to make capable
+  clients drop IPv4, PREF64 (RFC 8781) to advertise the NAT64 prefix for CLAT,
+  RDNSS (RFC 8106) to hand IPv6-only clients a resolver.
+- Dedicated `nat64` bridge, `fc64::/126` link, `192.168.240.0/20` Tayga pool,
+  static routes, and firewall rules (including NAT64-mapped RFC1918 blocks to
+  prevent the translator being used as a policy bypass).
+
+## Why it was removed
+
+### 1. Performance — the dealbreaker
+
+Throughput collapsed from line rate (~1 Gbps) to **~200-300 Mbps**, saturating
+the router CPU. Causes, all structural:
+
+- Tayga is a **userspace** translator. Every translated packet leaves the kernel
+  fastpath, is copied to userspace, translated, and re-injected.
+- Translated traffic crosses RouterOS **twice** — once as IPv6 (LAN → Tayga),
+  once as IPv4 (Tayga → WAN, with masquerade) — doubling firewall/conntrack work.
+- No hardware offload or fasttrack applies to either leg.
+
+With `translate_all`, *nearly all* internet traffic went through this path, so
+the penalty hit everything, not just IPv4-only destinations.
+
+### 2. Single point of failure
+
+DNS (CoreDNS) and most of the datapath (Tayga) became two containers in the
+critical path on a router whose built-in forwarder had been completely reliable.
+Container restarts, image pulls, or a crash now took down connectivity.
+
+### 3. Architectural inversion
+
+NAT64 exists to let **IPv6-only** clients reach the **IPv4** internet. The actual
+goal here was the opposite — *avoid* IPv6 egress entirely. Building an IPv6-only
+client environment (option 108, CLAT, PREF64) and then translating all of it back
+to IPv4 was solving the problem backwards. The complexity existed only to route
+around a property of the HE tunnel.
+
+### 4. Firewall complexity and a translation bypass hole
+
+NAT64 punched a hole in the firewall model. RouterOS filters IPv4 and IPv6
+independently, but NAT64 traffic enters as IPv6 and *leaves* as IPv4 after
+translation — so the carefully-built IPv4 forward policy (inter-VLAN isolation,
+RFC1918-to-WAN blocks) was simply bypassed for anything arriving via the
+translator. A client could reach a private IPv4 range by encoding it in the
+NAT64 prefix (`64:ff9b::c0a8:xxyy` = `192.168.x.y`), and the IPv4 rules would
+never see it because the packet was IPv6 until Tayga rewrote it.
+
+Plugging this required mirroring the IPv4 policy in the IPv6 chain: explicit
+`reject` rules for every NAT64-mapped RFC1918 block (`64:ff9b::a00:0/104`,
+`64:ff9b::ac10:0/108`, `64:ff9b::c0a8:0/112`), per-VLAN accept rules toward the
+`nat64` interface, plus a separate masquerade and LB hairpin-accept for the
+Tayga pool. That is a parallel, easy-to-get-wrong copy of the existing ruleset,
+whose correctness depended on getting CIDR-to-prefix arithmetic right. Removing
+NAT64 deleted all of it.
+
+### 5. Operational fragility (see coredns.md for detail)
+
+The setup had a long tail of subtle failure modes, each presenting identically
+as "client can't connect":
+
+- RouterOS static `FWD` entries return `NOERROR`/empty instead of relaying
+  `NXDOMAIN`, which broke `getaddrinfo` search-domain handling in Kubernetes
+  pods (`ENOTFOUND` for valid names).
+- `translate_all` discarded real AAAA for IPv6-only internal services, and
+  returned empty answers for names with no A record.
+- Per-interface RouterOS `ipv6 nd` entries default to `advertise-dns=no` and must
+  be *created* (not modified), so RDNSS/PREF64 silently never advertised.
+- Dynamic `from-pool` VLAN addressing made advertised RDNSS addresses point at
+  nonexistent router addresses.
+- Option 108 honoured by clients before the NAT64 path was verified working left
+  them stuck "obtaining IP address".
+
+Each was individually fixable, but the aggregate was a brittle system whose
+benefit didn't justify the surface area.
+
+## What replaced it
+
+Plain CoreDNS forwarder with **AAAA suppression by default** plus a whitelist for
+domains that should keep IPv6 (our own zone over the HE prefix, and any explicitly
+trusted domain). Clients prefer IPv4 because they simply don't receive AAAA for
+most names — no translation, no extra datapath hop, packet forwarding stays on the
+RouterOS fastpath at line rate. DNS is the only thing in the path. See
+[coredns.md](./coredns.md).
+
+Tradeoff accepted: a non-whitelisted IPv6-only destination (no A record) is
+unreachable. In practice essentially everything on the public internet still has
+an A record. The intended future refinement is a CoreDNS plugin that suppresses
+AAAA only when an A record also exists, removing the need for the whitelist; no
+in-tree plugin does this today.
+
+## Lessons
+
+- **Measure throughput before committing to an in-path translator on SOHO-class
+  hardware.** Userspace NAT64 (Tayga/Jool-in-container) on a MikroTik CPU is
+  fine for a few hundred Mbps, not for saturating a gigabit line.
+- **Match the mechanism to the actual goal.** The goal was "prefer IPv4 egress",
+  which is a one-line DNS policy, not a transition technology.
+- **Prefer solutions that stay on the fastpath.** Anything that pulls bulk
+  traffic into userspace or doubles the forwarding work will dominate the CPU.
+- **Fewer moving parts in the critical path.** Two containers carrying all DNS
+  and most traffic is a worse availability story than the stock forwarder, for a
+  cosmetic benefit (avoiding CAPTCHAs on some sites).
+- **Protocol translation breaks the firewall model.** When traffic changes L3
+  protocol mid-path, the two firewall policies must be kept in sync by hand, and
+  any gap is a silent bypass. A solution that doesn't translate keeps a single
+  coherent policy.
@@ -93,8 +93,8 @@ There are also networks, which are not VLANs, but are routed:
  Static assignment on CRS, access to factory IP of ONU
 - Containers on CRS<br>
  Access to every other network<br>
-  IP: 172.17.0.1/16, 2001:470:61a3:500::/64<br>
-  Static IP management
+  IP: 172.20.0.1/24, 2001:470:61a3:500::/64<br>
+  Static IP management, hosts Tailscale and CoreDNS containers

 Whole network is designed to eliminate VLANs, overlays where unnecessary to keep things simple. Only NAT rules are:

@@ -105,6 +105,10 @@ Whole network is designed to eliminate VLANs, overlays where unnecessary to keep
  Tailscale assigns IPv6 from private subnet with no way to configure it, so the assigned IPs are not routable
 - IPv4 port forwards from GPON PPPoE to respective services

+## DNS and IPv6 preference
+
+DNS is served by a CoreDNS container (`172.20.0.3`); RouterOS forwards client queries to it. CoreDNS suppresses AAAA records by default so clients prefer IPv4, avoiding the HE tunnel's datacenter-flagged egress (which triggers CAPTCHAs on some sites). Our own zone (`lumpiasty.xyz`) and any explicitly whitelisted domains keep AAAA for native IPv6. See [CoreDNS resolver](./coredns.md). An earlier NAT64/IPv6-mostly approach to the same problem was built and abandoned; see the [postmortem](./nat64-dns64-postmortem.md).
+
 There is also an UPnP and NAT-PMP enabled to automatically configure port forwards from LAN.

 ## Uplink
@@ -18,7 +18,7 @@ spec:
  chart:
    spec:
      chart: cert-manager-webhook-ovh
-      version: 0.9.10
+      version: 0.9.13
      sourceRef:
        kind: HelmRepository
        name: cert-manager-webhook-ovh
@@ -23,7 +23,7 @@ spec:
  chart:
    spec:
      chart: cilium
-      version: 1.19.4
+      version: 1.19.5
      sourceRef:
        kind: HelmRepository
        name: cilium
@@ -23,7 +23,7 @@ spec:
  chart:
    spec:
      chart: cloudnative-pg
-      version: 0.28.2
+      version: 0.28.3
      sourceRef:
        kind: HelmRepository
        name: cnpg
@@ -110,7 +110,7 @@ spec:
        kubernetes.io/os: linux
      containers:
        - name: coredns
-          image: registry.k8s.io/coredns/coredns:v1.14.3
+          image: registry.k8s.io/coredns/coredns:v1.14.4
          imagePullPolicy: IfNotPresent
          args: ["-conf", "/etc/coredns/Corefile"]
          ports:
@@ -97,7 +97,7 @@ spec:
          env:
            - name: GOMEMLIMIT
              value: 161MiB
-          image: registry.k8s.io/coredns/coredns:v1.14.3
+          image: registry.k8s.io/coredns/coredns:v1.14.4
          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 5
@@ -23,7 +23,7 @@ spec:
  chart:
    spec:
      chart: openbao
-      version: 0.28.3
+      version: 0.28.4
      sourceRef:
        kind: HelmRepository
        name: openbao
@@ -23,7 +23,7 @@ spec:
  chart:
    spec:
      chart: openebs
-      version: 4.4.0
+      version: 4.5.1
      sourceRef:
        kind: HelmRepository
        name: openebs
@@ -0,0 +1,43 @@
+# Mikrotik containers
+
+RouterOS containers running on the CRS418 providing network services that
+RouterOS cannot handle natively.
+
+## CoreDNS
+
+Replaces the built-in RouterOS DNS forwarder. Plain forwarding resolver with
+selective AAAA suppression: AAAA is suppressed by default so clients prefer IPv4
+(avoiding the HE tunnel's datacenter-flagged egress), while our own zone and any
+whitelisted domains keep AAAA for native IPv6.
+
+Source: [`coredns/`](coredns/). Image built by Woodpecker CI
+([`.woodpecker/coredns-build.yaml`](../.woodpecker/coredns-build.yaml)), pushed to
+`gitea.lumpiasty.xyz/lumpiasty/coredns-mikrotik`.
+
+The Corefile is baked into the image — edit [`coredns/Corefile`](coredns/Corefile)
+and push; the pipeline rebuilds and pushes a new image. Custom-built with a
+minimal plugin set (~6-8 MB vs the official ~20 MB image) to fit the CRS flash.
+
+See [docs/coredns.md](../docs/coredns.md) for design rationale, including why
+the earlier NAT64/DNS64 approach was removed.
+
+### Why not the official coredns/coredns image?
+
+The official image ships ~40 plugins and weighs ~20 MB compressed. A custom build with the 7 plugins we actually need fits in ~6-8 MB — important for the CRS internal flash.
+
+## Deployment
+
+The router configuration (container definitions, veth interfaces, bridge ports,
+DNS settings, firewall) is managed declaratively via Ansible, not by manual CLI
+commands. See [`ansible/roles/routeros/`](../ansible/roles/routeros/) and run:
+
+```sh
+cd ansible && ansible-playbook playbooks/routeros.yml
+```
+
+Containers do not auto-start on first image pull; after the initial deploy,
+start manually once (subsequent boots are handled by `start-on-boot=yes`):
+
+```
+/container/start [find name=coredns]
+```
@@ -0,0 +1,53 @@
+# CoreDNS as a plain forwarding resolver with selective AAAA suppression.
+#
+# Background: the ISP provides no native IPv6, only a Hurricane Electric tunnel.
+# HE addresses are flagged as datacenter ranges by some sites (endless CAPTCHAs,
+# bot detection). To avoid this, IPv6 (AAAA) resolution is suppressed by default
+# so clients use IPv4, while a whitelist keeps AAAA for domains where native
+# IPv6 is wanted (our own services reachable over the HE prefix, and any domain
+# explicitly trusted over IPv6).
+#
+# NAT64/DNS64 was tried and removed: it forced most traffic through a userspace
+# Tayga translator, capping throughput at ~250 Mbps on the RB-class CPU (line
+# rate is 1 Gbps) and adding two containers as a SPOF — all to avoid IPv6 egress
+# we don't want in the first place. Plain AAAA suppression achieves the same
+# IPv4-preferred outcome with zero datapath overhead.
+#
+# TODO: replace the global template suppression + whitelist with a plugin that
+# suppresses AAAA only when the domain has no A record (so IPv6-only
+# destinations still work). No such in-tree plugin exists yet.
+
+# Whitelist: domains that keep AAAA resolution (native IPv6 via HE tunnel).
+(aaaa_allowed) {
+    forward . 1.1.1.1 8.8.8.8 {
+        prefer_udp
+    }
+    cache 300
+    errors
+    log . {
+        class error
+    }
+}
+
+# Our own zone: services have native IPv6 on the HE prefix, keep AAAA.
+lumpiasty.xyz:53 {
+    import aaaa_allowed
+}
+
+# Default: forward everything, but suppress AAAA so clients use IPv4 and
+# avoid the HE tunnel's datacenter-flagged egress.
+.:53 {
+    template IN AAAA {
+        rcode NOERROR
+    }
+    forward . 1.1.1.1 8.8.8.8 {
+        prefer_udp
+    }
+    cache 300
+    errors
+    log . {
+        class error
+    }
+    reload
+    health :8080
+}
@@ -0,0 +1,32 @@
+# Stage 1: build CoreDNS with minimal plugin set
+FROM golang:1.26-alpine AS build
+
+RUN apk add --no-cache git make bash
+
+WORKDIR /src
+RUN git clone --depth 1 --branch v1.12.1 \
+    https://github.com/coredns/coredns .
+
+# Overwrite plugin.cfg with our trimmed list before compilation
+COPY plugin.cfg .
+
+RUN go generate && make
+
+# Stage 2: extract CA certificates from a full image
+FROM debian:stable-slim AS certs
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    ca-certificates && rm -rf /var/lib/apt/lists/*
+
+# Stage 3: minimal runtime — scratch + binary + certs only
+FROM scratch
+
+COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
+COPY --from=build /src/coredns /coredns
+COPY Corefile /Corefile
+
+# 53:  DNS (UDP + TCP)
+# 8080: health endpoint
+EXPOSE 53/udp 53/tcp 8080/tcp
+
+# RouterOS requires root to bind port 53 — no USER directive
+ENTRYPOINT ["/coredns", "-conf", "/Corefile"]
@@ -0,0 +1,7 @@
+errors:errors
+log:log
+health:health
+template:template
+cache:cache
+forward:forward
+reload:reload
@@ -0,0 +1,14 @@
+# Allow Woodpecker to run privileged containers
+# Used for example to build multi-arch mikrotik-tailscale image
+# which needs to register binfmt
+
+cluster:
+  apiServer:
+    admissionControl:
+      - name: PodSecurity
+        configuration:
+          apiVersion: pod-security.admission.config.k8s.io/v1beta1
+          kind: PodSecurityConfiguration
+          exemptions:
+            namespaces:
+              - woodpecker
@@ -1,3 +1,7 @@
 path "secret/data/renovate" {
    capabilities = ["read"]
 }
+
+path "secret/data/container-registry" {
+    capabilities = ["read"]
+}
Author	SHA1	Message	Date
Renovate	6cec5c234b	Merge pull request 'Update registry.k8s.io/coredns/coredns Docker tag to v1.14.4' (#341 ) from renovate/registry.k8s.io-coredns-coredns-1.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-21 02:01:27 +00:00
Renovate	cdee1a0798	Update registry.k8s.io/coredns/coredns Docker tag to v1.14.4	2026-06-21 02:01:26 +00:00
Lumpiasty	4034628449	Fast fail connection when WAN failover ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-21 02:38:24 +02:00
Lumpiasty	1e86dc5e2b	Detect GPON blackhole using ping ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-21 02:00:32 +02:00
Renovate	b1751ec427	Merge pull request 'Update Helm release openebs to v4.5.1' (#339 ) from renovate/openebs-4.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-19 02:01:13 +00:00
Renovate	b9458c46bd	Update Helm release openebs to v4.5.1	2026-06-19 02:01:09 +00:00
Renovate	6f73511e2b	Merge pull request 'Update quay.io/openbao/openbao Docker tag to v2.5.5' (#338 ) from renovate/quay.io-openbao-openbao-2.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-18 02:01:32 +00:00
Renovate	de1a31f98b	Merge pull request 'Update Helm release openbao to v0.28.4' (#337 ) from renovate/openbao-0.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline is pending Details	2026-06-18 02:01:25 +00:00
Renovate	319f1c3009	Update quay.io/openbao/openbao Docker tag to v2.5.5	2026-06-18 02:01:25 +00:00
Renovate	c513c575d2	Update Helm release openbao to v0.28.4	2026-06-18 02:01:20 +00:00
Renovate	6f7c4c91b4	Merge pull request 'Update Helm release cilium to v1.19.5' (#336 ) from renovate/cilium-1.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline failed Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-17 02:01:25 +00:00
Renovate	4df35f496f	Merge pull request 'Update alpine Docker tag to v3.24.1' (#335 ) from renovate/alpine-3.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline is pending Details	2026-06-17 02:01:10 +00:00
Renovate	8214677d03	Update Helm release cilium to v1.19.5	2026-06-17 02:01:09 +00:00
Renovate	a01ec90b06	Update alpine Docker tag to v3.24.1	2026-06-17 02:01:07 +00:00
Lumpiasty	d10c3efe68	increase gitea storage size ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-16 23:25:21 +02:00
Lumpiasty	ae7f58240c	Merge pull request 'Update golang Docker tag to v1.26' (#331 ) from renovate/golang-1.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/push/coredns-build Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details Reviewed-on: #331	2026-06-15 23:08:36 +00:00
Lumpiasty	9f5d45d515	Merge pull request 'Update renovate/renovate Docker tag to v43.222.1' (#333 ) from renovate/renovate-renovate-43.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was canceled Details Reviewed-on: #333	2026-06-15 23:08:26 +00:00
Lumpiasty	d3cb2a6e65	Decrease logging of CoreDNS ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/push/coredns-build Pipeline was successful Details	2026-06-16 00:53:56 +02:00
Lumpiasty	679ebb3465	Get rid of NAT64 setup ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/push/coredns-build Pipeline was successful Details	2026-06-16 00:29:45 +02:00
Renovate	de10cba76c	Update renovate/renovate Docker tag to v43.222.1	2026-06-15 02:01:26 +00:00
Renovate	b993115b41	Merge pull request 'Update alpine/k8s Docker tag to v1.36.2' (#332 ) from renovate/alpine-k8s-1.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-15 02:01:26 +00:00
Renovate	f02579a2f2	Update alpine/k8s Docker tag to v1.36.2	2026-06-15 02:01:23 +00:00
Renovate	e458949817	Merge pull request 'Update Helm release cert-manager-webhook-ovh to v0.9.13' (#330 ) from renovate/cert-manager-webhook-ovh-0.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-13 02:01:22 +00:00
Renovate	60ba0cfe90	Update golang Docker tag to v1.26 ci/woodpecker/push/coredns-build Pipeline was successful Details	2026-06-13 02:01:21 +00:00
Renovate	cb5f459182	Update Helm release cert-manager-webhook-ovh to v0.9.13	2026-06-13 02:01:17 +00:00
Lumpiasty	d3a067886e	coredns: fix ENOTFOUND for own zone, enable dns64 for IPv4 clients ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/push/coredns-build Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details Two Corefile changes: - Add lumpiasty.xyz server block without dns64. Replaces the manual RouterOS static FWD entry (\"bypass nat64\") which returned NOERROR with empty answer instead of relaying NXDOMAIN. Combined with ndots:5 and pod search domains this made getaddrinfo stop at the search-suffixed candidate and fail with ENOTFOUND for valid names (kaneo -> authentik OAuth fetch failures). CoreDNS relays rcodes faithfully; internal zone keeps real AAAA for native IPv6. - Add allow_ipv4 to dns64 (previously uncommitted): without it only queries arriving over IPv6 are synthesized, but all clients reach CoreDNS via RouterOS over IPv4, so translate_all never applied. The RouterOS static FWD entry must be removed after deploying the new image - ansible already declares only the ts.net entry, so a playbook run handles it.	2026-06-13 02:45:30 +02:00
Lumpiasty	33e01376b1	Add NAT64, DNS64 to network ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/push/coredns-build Pipeline failed Details	2026-06-13 00:27:43 +02:00
Lumpiasty	374ee146fe	update llama-swap image ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-12 18:17:52 +02:00
Lumpiasty	2380cd16e4	add more gemma 4 26b variants ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-12 18:01:27 +02:00
Renovate	23ddd7c233	Merge pull request 'Update Helm release authentik to v2026.5.3' (#329 ) from renovate/authentik-2026.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-12 02:01:25 +00:00
Renovate	a6bfb3d93c	Update Helm release authentik to v2026.5.3	2026-06-12 02:01:18 +00:00
Lumpiasty	59f32659a1	Merge pull request 'Update renovate/renovate Docker tag to v43.220.0' (#320 ) from renovate/renovate-renovate-43.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details Reviewed-on: #320	2026-06-11 21:47:26 +00:00
Lumpiasty	199b14b810	Merge pull request 'Update Helm release openebs to v4.5.0' (#322 ) from renovate/openebs-4.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was canceled Details Reviewed-on: #322	2026-06-11 21:47:07 +00:00
Lumpiasty	fc971e6e6c	Merge pull request 'Update alpine Docker tag to v3.24.0' (#327 ) from renovate/alpine-3.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was canceled Details Reviewed-on: #327	2026-06-11 21:47:00 +00:00
Lumpiasty	aab4bc279c	Merge pull request 'Update ghcr.io/remsky/kokoro-fastapi-cpu Docker tag to v0.5.0' (#324 ) from renovate/ghcr.io-remsky-kokoro-fastapi-cpu-0.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was canceled Details Reviewed-on: #324	2026-06-11 21:46:39 +00:00
Lumpiasty	7f6439d64a	switch gemma 4 quant, add mtp and nothink variants ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-11 21:06:12 +02:00
Renovate	f0921e903a	Update renovate/renovate Docker tag to v43.220.0	2026-06-11 02:01:22 +00:00
Renovate	5fed73515b	Merge pull request 'Update Helm release cloudnative-pg to v0.28.3' (#328 ) from renovate/cloudnative-pg-0.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-11 02:01:17 +00:00
Renovate	1c092c8044	Update Helm release cloudnative-pg to v0.28.3	2026-06-11 02:01:13 +00:00
Lumpiasty	8860f6782e	add converting proxy to parakeet ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-10 20:46:17 +02:00
Lumpiasty	f863a0a496	use parakeet.cpp instead of whisper ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-10 20:08:14 +02:00
Renovate	6bf31f0ae6	Update alpine Docker tag to v3.24.0	2026-06-10 02:01:07 +00:00
Renovate	979f5796d5	Merge pull request 'Update Helm release cert-manager-webhook-ovh to v0.9.11' (#326 ) from renovate/cert-manager-webhook-ovh-0.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-09 15:42:31 +00:00
Renovate	8bca1cf90f	Merge pull request 'Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-09' (#325 ) from renovate/ghcr.io-mostlygeek-llama-swap-2026.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline is pending Details	2026-06-09 15:42:26 +00:00
Renovate	b3793d11d9	Update Helm release cert-manager-webhook-ovh to v0.9.11	2026-06-09 15:42:26 +00:00
Renovate	fb4fa9b0e7	Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-09	2026-06-09 15:40:20 +00:00
Renovate	13a87e5b00	Update ghcr.io/remsky/kokoro-fastapi-cpu Docker tag to v0.5.0	2026-06-07 02:02:49 +00:00
Renovate	1da43d39e2	Merge pull request 'Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-06' (#323 ) from renovate/ghcr.io-mostlygeek-llama-swap-2026.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-07 02:02:49 +00:00
Renovate	87c56a9ca1	Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-06	2026-06-07 02:01:15 +00:00
Renovate	8ff9126025	Merge pull request 'Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-05' (#321 ) from renovate/ghcr.io-mostlygeek-llama-swap-2026.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-06 02:02:46 +00:00
Renovate	4e0a97d6f8	Update Helm release openebs to v4.5.0	2026-06-06 02:02:45 +00:00
Renovate	43c2036642	Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-05	2026-06-06 02:01:10 +00:00
Renovate	4d51d45f74	Merge pull request 'Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-04' (#319 ) from renovate/ghcr.io-mostlygeek-llama-swap-2026.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-05 02:02:58 +00:00
Renovate	fe607d3fb8	Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-04	2026-06-05 02:01:18 +00:00
Lumpiasty	cd514c71b6	oauth auto redirect and allowed role in openwebui ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-04 18:03:00 +02:00
Lumpiasty	32a483c711	update outdated readme sections	2026-06-04 17:24:17 +02:00
Lumpiasty	0426f86719	Merge pull request 'Update renovate/renovate Docker tag to v43.210.2' (#314 ) from renovate/renovate-renovate-43.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details Reviewed-on: #314	2026-06-04 14:40:48 +00:00
Lumpiasty	da365501e9	Merge pull request 'Update Helm release open-webui to v14.8.0' (#315 ) from renovate/open-webui-14.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was canceled Details Reviewed-on: #315	2026-06-04 14:40:42 +00:00
Renovate	cf7c0075e7	Update renovate/renovate Docker tag to v43.210.2	2026-06-04 02:03:00 +00:00
Renovate	0dd86e3321	Merge pull request 'Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-03' (#318 ) from renovate/ghcr.io-mostlygeek-llama-swap-2026.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-04 02:02:56 +00:00
Renovate	3060950d56	Merge pull request 'Update caddy Docker tag to v2.11.4' (#317 ) from renovate/caddy-2.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline is pending Details	2026-06-04 02:01:07 +00:00
Renovate	9dd0c7eb0a	Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-03	2026-06-04 02:01:07 +00:00
Renovate	68b480299d	Update caddy Docker tag to v2.11.4	2026-06-04 02:01:05 +00:00
Renovate	973b0beb21	Update Helm release open-webui to v14.8.0	2026-06-03 02:03:19 +00:00
Renovate	57fd804712	Merge pull request 'Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-02' (#316 ) from renovate/ghcr.io-mostlygeek-llama-swap-2026.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-03 02:03:19 +00:00
Renovate	149e85762f	Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-06-02	2026-06-03 02:01:32 +00:00
Lumpiasty	9dfa780354	add missing apps to readme ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-06-03 02:27:41 +02:00
Lumpiasty	b1c616a20f	add application guidelines for LLMs ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-06-03 02:08:52 +02:00
Lumpiasty	fa32fdfd28	add missing patch to Makefile	2026-06-03 01:35:01 +02:00
Lumpiasty	1b66a8c230	Change Tailscale distribution ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details gitea.lumpiasty.xyz/Lumpiasty/tailscale-mikrotik allows us to move tailscale to internal flash	2026-06-02 17:29:22 +02:00
Lumpiasty	af4a7fee48	go back to official llama-swap image ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-05-31 16:23:50 +02:00
Lumpiasty	6546676dd6	add llama-swap optimizations recommended by claude ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-05-31 05:18:47 +02:00
Lumpiasty	353155f7ad	Enable DMA transfer queue on llama-swap ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-05-31 04:25:08 +02:00
Lumpiasty	172fbb1ded	Test updated base-image llama-swap ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-05-31 03:15:44 +02:00
Lumpiasty	62f6baf948	hairpin nat cluster	2026-05-31 03:14:46 +02:00
Renovate	8c8147176d	Merge pull request 'Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-05-29' (#313 ) from renovate/ghcr.io-mostlygeek-llama-swap-2026.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-05-30 02:02:44 +00:00
Renovate	fabd6bb2e0	Merge pull request 'Update dependency kaneo to v2.7.7' (#312 ) from renovate/kaneo into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline is pending Details	2026-05-30 02:01:06 +00:00
Renovate	fa85180736	Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-05-29	2026-05-30 02:01:05 +00:00
Renovate	f8243da487	Update dependency kaneo to v2.7.7	2026-05-30 02:01:02 +00:00
Renovate	c2e87933a1	Merge pull request 'Update Helm release woodpecker to v3.6.4' (#311 ) from renovate/woodpecker-3.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline is running Details ci/woodpecker/cron/renovate Pipeline was successful Details	2026-05-29 02:03:11 +00:00
Renovate	b7bdd06d75	Merge pull request 'Update Helm release authentik to v2026.5.2' (#310 ) from renovate/authentik-2026.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was canceled Details	2026-05-29 02:03:05 +00:00
Renovate	97281091f7	Update Helm release woodpecker to v3.6.4	2026-05-29 02:03:03 +00:00
Renovate	d86fbf6aa1	Merge pull request 'Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-05-28' (#309 ) from renovate/ghcr.io-mostlygeek-llama-swap-2026.x into fresh-start ci/woodpecker/push/flux-reconcile-source Pipeline was canceled Details	2026-05-29 02:02:59 +00:00
Renovate	26391c1039	Update Helm release authentik to v2026.5.2	2026-05-29 02:02:58 +00:00
Renovate	eb579d2632	Update ghcr.io/mostlygeek/llama-swap Docker tag to unified-vulkan-2026-05-28	2026-05-29 02:01:16 +00:00
Lumpiasty	5aa898e166	add privileged access to woodpecker pods ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details ci/woodpecker/cron/renovate Pipeline was canceled Details	2026-05-29 01:04:27 +02:00
Lumpiasty	c874776e6e	allow woodpecker to publish docker images ci/woodpecker/push/flux-reconcile-source Pipeline was successful Details	2026-05-29 00:46:17 +02:00