Forum Discussion

Aleksey D's avatar
Aleksey D
Copper Contributor
Jan 19, 2026

Lots of DNS Server events 5504 on AD DNS server from Cloudflare etc

Hi! I'm getting about 18 events with id 5504 while trying to resolve some DNS names, like fullfiles.xyz.

The DNS server is configured to use provider DNS and root hints.

I can suppress these messages by disabling root hints or by disabling EDNS0 with dnscmd /config /enablednsprobes 0.

I tried to use packet capture on the DC and on the router, and analyzed the results with AI, which answered:

"You receive malformed patterns on the WAN interface."

Can anybody explain the cause of this problem? Any ideas to fix it?

Thanks!

Active Directory
DNS
DNS issues
DNS resolution

1 Reply

  • FIRAT_BOYAN's avatar
    FIRAT_BOYAN
    MCT
    Feb 17, 2026

    Hello,

    Event ID 5504 on a Windows DNS server typically indicates that the server received a packet that it could not interpret as a valid DNS message. In environments using forwarders and root hints, this is most commonly related to EDNS0 handling or upstream device interference, not Active Directory itself.

    You already confirmed two important facts:

    Disabling root hints suppresses the events.

    Disabling EDNS0 with dnscmd /config /enableednsprobes 0 suppresses the events.

    That strongly points to an EDNS0 compatibility issue.

    What is happening

    When your DNS server queries external resolvers such as Cloudflare, it uses EDNS0 to allow larger UDP packet sizes. Some firewalls, ISP equipment, or NAT devices incorrectly handle fragmented UDP or EDNS0 extensions.

    They may:

    • Truncate packets improperly
    • Rewrite DNS payloads
    • Drop fragments
    • Return malformed responses

     

    When Windows DNS receives such a response, it logs Event 5504 because the packet structure does not match expected DNS format.

    Why root hints matter

    If forwarders fail or timeout, Windows falls back to root hints. That increases the number of direct Internet DNS exchanges, increasing exposure to malformed EDNS0 responses.

    Why disabling EDNS0 reduces events

    Without EDNS0, DNS queries fall back to standard 512 byte UDP packets, which are less likely to fragment and therefore less likely to be mishandled by intermediate devices.

    This confirms the issue is on the WAN path, not inside AD DNS.

    How to validate

    Capture traffic directly on the DNS server using:

    netsh trace start capture yes persistent no tracefile=c:\dns.etl

    Then analyse DNS responses from external resolvers and check for:

    • UDP fragmentation
    • Incorrect DNS header flags
    • Truncated OPT records

     

    If possible, test by temporarily:

    • Using only trusted forwarders such as 1.1.1.1 or 8.8.8.8
    • Disabling root hints entirely
    • Forcing DNS over TCP for testing

     

    Most likely root cause

    A firewall, ISP router, or security appliance modifying or mishandling EDNS0 traffic. This is very common with older firmware or DPI enabled devices.

    This is not an AD DNS corruption issue. It is an upstream DNS packet handling problem on the Internet path.