NetBird login loop / 404 on Keycloak auth endpoint when both services share the same parent domain

[lame-engineer]

Before posting

• I searched existing discussions and issues, including closed ones, and checked the relevant docs.
• I believe this is a product bug rather than a configuration or setup question.
• I can reproduce this issue, or for intermittent issues I've included trigger, frequency, and timing details below.
• I removed or anonymized sensitive data from logs, screenshots, and configuration.

Affected area

Login / Authentication / IdP

Deployment type

Self-hosted - quickstart script

Operating system or environment

Linux

NetBird version and upgrade status

0.65

Did this work before?

Not sure

Regression details

No response

Summary

When NetBird's embedded Dex redirects the browser to Keycloak for authentication, the Keycloak auth endpoint returns 404. The browser loops back to the NetBird login page without completing authentication.
The issue is domain-specific — moving Keycloak to a different parent domain resolves it immediately.

Current behavior

Keycloak auth endpoint returns 404. Keycloak logs show:

WARN Parameter 'client_id' not present or present multiple times in the HTTP request parameters
ERROR KC-SERVICES0092: Missing parameter: response_type

This indicates query parameters are being stripped before reaching Keycloak's application layer.

Expected behavior

Browser is redirected to Keycloak login page, user authenticates, and is returned to NetBird dashboard with a valid session.

Steps to reproduce

• Deploy NetBird self-hosted with Keycloak as external IdP
• Host both services under the same parent domain (netbird.example.com and keycloak.example.com)
• Place a load balancer in front in TCP passthrough (SNI routing) mode
• Configure Keycloak IdP via NetBird Dashboard → Settings → Identity Providers
• Attempt login on the NetBird dashboard

Environment and topology

Deployment Topology

Edge load balancer (TCP/SNI passthrough) sitting in front of two backend nodes on a private network:

• Node 1 — Keycloak IdP (Apache reverse proxy → Keycloak container)
• Node 2 — NetBird server (Nginx reverse proxy → Docker Compose stack)

Both nodes are on the same private subnet, virtualised on a hypervisor cluster.

---

NetBird Stack (Node 2)

• Nginx running on host (not containerised)
• Docker Compose managing two containers:
  • netbird-server — combined container (Management + Signal + Relay + STUN), exposed on loopback only
  • netbird-dashboard — UI, exposed on loopback only
• Persistent storage via named Docker volume
• Config injected via mounted config.yaml

---

Identity Provider (Node 1)

• Keycloak running as a Docker container
• Apache mod_proxy terminating TLS and proxying to Keycloak on loopback
• Shared TLS certificate across services

---

Network Flow

Client → Load Balancer (SNI routing) → Node 1 / Node 2
                                           ↓
                                     Apache / Nginx (TLS termination)
                                           ↓
                                     Container (loopback)

---

Key Constraints

• Both services share the same parent domain — root cause of the same-site auth issue
• Load balancer operates at Layer 4 (TCP) — no HTTP header inspection or modification
• No IPv6 assigned to load balancer node at network level — NAT64 traffic from IPv6-only clients was not reaching backends correctly

Self-hosted details, if available

No response

Logs, status output, or debug evidence

**Nginx Error Log — gRPC Connection Reset**

[error] recv() failed (104: Unknown error) while reading response header 
from upstream, request: "POST /management.ManagementService/GetServerKey 
HTTP/2.0", upstream: "grpc://127.0.0.1:<port>"

[error] recv() failed (104: Unknown error) while reading response header 
from upstream, request: "POST /signalexchange.SignalExchange/Send HTTP/2.0", 
upstream: "grpc://127.0.0.1:<port>"

---

NetBird Server Log — gRPC Context Cancelled (normal, not actionable)

WARN [context: GRPC] management/internals/shared/grpc/server.go:369: 
recv job response error: rpc error: code = Canceled desc = context canceled

INFO [context: HTTP] management/server/http/handlers/instance/instance_handler.go:49: 
instance setup status: false

---

NetBird Server Log — Dex IdP Error (Keycloak redirect URI mismatch)

ERRO [err: Unregistered redirect_uri ("<masked-domain>/nb-auth")] 
idp/dex/logrus_handler.go:83: failed to parse authorization request

---

Keycloak Log — Query Parameter Stripping by Apache

WARN [org.keycloak.events] type="LOGIN_ERROR", realmName="<realm>", 
clientId="null", error="invalid_request", 
reason="Parameter 'client_id' not present or present multiple times 
in the HTTP request parameters"

ERROR [org.keycloak.services] KC-SERVICES0092: Missing parameter: response_type

WARN [org.keycloak.events] type="LOGIN_ERROR", realmName="<realm>", 
clientId="<client-id>", error="invalid_request", 
reason="Missing parameter: response_type"

---

Keycloak Log — Unrelated (other services, masked)

WARN [org.keycloak.events] type="LOGIN_ERROR", realmName="<realm>", 
clientId="<client-id>", error="invalid_user_credentials", 
ipAddress="<internal-ip>"

WARN [org.keycloak.events] type="REFRESH_TOKEN_ERROR", realmName="<realm>", 
clientId="<client-id>", error="invalid_token", 
reason="Token is not active"

---

Browser — Failed Request Chain

# Initial request (correct)
GET /oauth2/auth?client_id=<client-id>&redirect_uri=https://<netbird-domain>/nb-auth
  → Status: 302 (redirect to Keycloak)

# Keycloak auth request (correct after fixes)
GET https://<keycloak-domain>/realms/<realm>/protocol/openid-connect/auth
  ?client_id=<client-id>
  &redirect_uri=https://<netbird-domain>/oauth2/callback
  &response_type=code
  &scope=openid+profile+email
  → Status: 404 (before Apache fix) / 200 (after domain change)

# Remote address — NAT64 synthesised (before IPv6 fix)
Remote Address: [64:ff9b::<masked>]:443

# Remote address — IPv4 (after IPv6 fix)  
Remote Address: <masked-public-ip>:443

### Related issues or discussions

_No response_

### Impact

- workaround
  - Host Keycloak on a domain with a different parent (e.g. auth.otherdomain.com) to change the sec-fetch-site context from same-site to cross-site.

### Additional context

_No response_

[eben-vranken]

This is an Apache config bug, not a NetBird bug. Your own logs name the cause: the query string is being stripped at the Apache reverse proxy in front of Keycloak.

WARN  Parameter 'client_id' not present or present multiple times
ERROR KC-SERVICES0092: Missing parameter: response_type

The browser request leaving NetBird is correct (it has client_id, response_type, redirect_uri). By the time it reaches Keycloak's application layer those params are gone, so Keycloak rejects it and you get the 404 + login loop. NetBird is sending a valid request; Apache is mangling it.

Fix: make the Apache proxy preserve the query string. Common culprits:

• A RewriteRule whose target contains a ?, or that lacks the [QSA] flag. Add [QSA] (or drop the ? from the substitution).
• Mixing RewriteRule with ProxyPass on the same path. Prefer a plain ProxyPass / ProxyPassReverse with no rewriting so the full request line passes through untouched.

The present multiple times warning specifically points at a rewrite appending the query string on top of one that's already there.

On the "different parent domain fixes it" workaround: that's almost certainly coincidental. Changing the domain likely routed the request through a different vhost / proxy block that happened to preserve the query string. The shared parent domain and the sec-fetch-site angle are a separate concern from the 404, and chasing them won't fix the param stripping.

Also worth checking (separate issue):

ERRO Unregistered redirect_uri ("<masked>/nb-auth")

Your redirect_uri differs between requests (/nb-auth vs /oauth2/callback). Make sure the Keycloak client's allowed redirect URIs match exactly what NetBird/Dex sends, or you'll hit a different failure once the proxy is fixed.

[lame-engineer]

Thanks for the detailed breakdown

That said, a few clarifications from my side:

On the Apache query string stripping:
I did apply the nocanon + AllowEncodedSlashes NoDecode fix to the Apache VirtualHost, and while it cleaned up the client_id / response_type errors in Keycloak logs, the login loop kept going. The 404 was still there in the browser even after confirming the parameters were reaching Keycloak correctly. So Apache was definitely adding noise, just not the whole story.

On the different parent domain being coincidental:
I thought about this too, but it felt too clean to be a coincidence. Same Apache config, same Keycloak client, same NetBird setup only the parent domain changed and it just worked. The sec-fetch-site header shifting from same-site to cross-site is the only variable I can point to. I did check if a different vhost was being hit but found nothing both domains had identical VirtualHost blocks.

On the redirect URI mismatch:
Yeah that was a separate mess from an earlier misconfigured state Dex had a stale redirect URI sitting in its SQLite database. Once I cleared that, everything was correctly sending redirect_uri=https://<netbird-domain>/oauth2/callback matching the Keycloak client exactly. The two errors showed up together in the logs which made it confusing, but they were completely unrelated.

Where I landed:
Hosting NetBird and Keycloak under the same parent domain, behind a Layer 4 load balancer, consistently broke authentication. Moving to separate parent domains consistently fixed it. Whether that is a sec-fetch-site validation path in Keycloak, Apache same-site cookie behaviour, or something deeper in the stack is honestly what I am still not sure about which is why I raised it here rather than just closing it off.

Happy to share sanitised logs and configs if it helps narrow it down.

[eben-vranken]

This is HTTP/2 connection coalescing, which also explains why a different parent domain fixes it.

Two conditions are both true in your setup:

• both subdomains resolve to the same load balancer IP, and
• you serve a shared cert valid for both names.

Per RFC 7540 §9.1.1, when both hold, Chrome/Firefox may send keycloak.example.com requests over the already-open connection to netbird.example.com with no new TLS handshake, so no new SNI. Your L4 load balancer routes purely on SNI, so it can't re-route that coalesced request. It lands on the NetBird backend's Nginx, which has no /realms/<realm>/protocol/openid-connect/auth route and returns your 404.

This fits the evidence better than sec-fetch-site:

• the 404 is on the Keycloak URL but served by the wrong backend, not by Keycloak;
• your Apache fix cleaned up params on requests that did reach Keycloak, but the coalesced redirect never got there, so the loop survived;
• changing the parent domain breaks the "one cert covers both names" condition, so the browser stops coalescing and opens a fresh connection with correct SNI. sec-fetch-site shifting is a correlate of the same registrable-domain condition, not the cause.

Confirm in one step: check Node 2 (NetBird) Nginx access log for a 404 on /realms/<realm>/protocol/openid-connect/auth. If that request is in the NetBird backend's log, coalescing is proven, since it was meant for Node 1.

Fix (any one):

• separate certs so no single cert is valid for both hostnames (minimal change, makes them ineligible to coalesce);
• disable HTTP/2 / h2 ALPN at the edge so it falls back to HTTP/1.1, which doesn't coalesce across hosts;
• use an L7 load balancer that terminates TLS and routes on Host / :authority.

So it's a real interaction bug in this topology, but it lives at the cert + L4-SNI + HTTP/2 layer, not inside Keycloak or NetBird. Your separate-domain workaround is just an instance of the first fix.

[lame-engineer]

Thank you for the detailed breakdown the HTTP/2 coalescing pointer was the key that unlocked everything. Would not have got there without it. Appreciate the time.

[eben-vranken]

no worries! you can mark a correct answer or just close this thread and itll stop bots from posting