Lots of DNS Server events 5504 on AD DNS server from Cloudflare etc

MCT

Feb 17, 2026

Hello,

Event ID 5504 on a Windows DNS server typically indicates that the server received a packet that it could not interpret as a valid DNS message. In environments using forwarders and root hints, this is most commonly related to EDNS0 handling or upstream device interference, not Active Directory itself.

You already confirmed two important facts:

Disabling root hints suppresses the events.

Disabling EDNS0 with dnscmd /config /enableednsprobes 0 suppresses the events.

That strongly points to an EDNS0 compatibility issue.

What is happening

When your DNS server queries external resolvers such as Cloudflare, it uses EDNS0 to allow larger UDP packet sizes. Some firewalls, ISP equipment, or NAT devices incorrectly handle fragmented UDP or EDNS0 extensions.

They may:

Truncate packets improperly
Rewrite DNS payloads
Drop fragments
Return malformed responses

When Windows DNS receives such a response, it logs Event 5504 because the packet structure does not match expected DNS format.

Why root hints matter

If forwarders fail or timeout, Windows falls back to root hints. That increases the number of direct Internet DNS exchanges, increasing exposure to malformed EDNS0 responses.

Why disabling EDNS0 reduces events

Without EDNS0, DNS queries fall back to standard 512 byte UDP packets, which are less likely to fragment and therefore less likely to be mishandled by intermediate devices.

This confirms the issue is on the WAN path, not inside AD DNS.

How to validate

Capture traffic directly on the DNS server using:

netsh trace start capture yes persistent no tracefile=c:\dns.etl

Then analyse DNS responses from external resolvers and check for: