DHCP Failover Issue – Standby Server Responding When It Should Not
Hi everyone, I'm encountering an issue with my DHCP failover setup in Hot Standby mode, and I need insights into why the standby server is providing DHCP leases when it shouldn’t. Setup Overview: I manage a network with over 100 sites worldwide, each having a local DHCP server. Each site has a dedicated DHCP server running on the server VLAN. Clients reside on different VLANs, and IP helpers (DHCP relay) are configured on a Checkpoint firewall at each site. The IP helper forwards DHCP requests to: The local DHCP server (primary) in the site's server VLAN. The standby DHCP server (failover), located at an on-premises data center (DC). DHCP servers are configured in Hot Standby mode using Microsoft DHCP Failover. Issue: Despite the Hot Standby configuration, I noticed that my Cisco Meraki dashboard frequently reports a new DHCP server detected, referring to the standby DHCP server, even though the primary DHCP server at the local site is available. Cisco Meraki triggers this alert when it detects DHCPACK packets from the standby DHCP server traversing the local networks. However, in Hot Standby mode, the failover server should only issue leases if the primary server is unreachable. Example: Site-1's primary DHCP server (DHCP-1) has a failover partnership with Failover-1 at the DC. Site-1's connectivity to the DC is stable, yet Cisco Meraki occasionally detects DHCPACK packets from Failover-1, triggering alerts. Troubleshooting Done So Far: Verified that failover mode is correctly set to Hot Standby (not Load Balance). Confirmed that the primary DHCP server is healthy and responding. Checked DHCP logs on both servers but found no clear failover events. Performed packet captures of DHCP traffic, but the results were inconclusive. Investigated whether Checkpoint firewall’s IP helper can prioritize the primary DHCP server, but it appears not to support this functionality. Created a PowerShell script to check for failover-related event logs (Event IDs: 20254 and 20255). This provided better visibility but did not correlate with the Meraki alerts. Questions: Are there any known scenarios where a standby DHCP server in Hot Standby mode might mistakenly issue leases, even when the primary is active? Is there any detailed information on the failover “heartbeat” mechanism between primary and standby servers? I found that it uses TCP port 647, but I couldn’t locate official documentation on the interval and failure conditions. Could failover state synchronization delays cause this behavior? Are there specific logs or PowerShell commands I should check to confirm why the standby server is responding? Is there a way to prevent the standby server from responding unless the primary is truly unreachable (e.g., registry settings, advanced configuration)? Any guidance or troubleshooting steps would be greatly appreciated! Thanks in advance.