Rogue DHCP

Is it a rogue DHCP? Not exactly. This is what happened on a Monday morning when there are 20+ users complaining that they were not able to access any network resources. A quick check shows that they are getting IP addresses not from our allocated ranges.

Tried releasing the IP address using ipconfig /release and renew the IP address using ipconfig /renew but no luck.

Traced the network points from the users’ end to the switch ports. Found that all of them were connected to the same stack of switches. There was no problem with other stack of switches. Also found out that the IP address range that they got were from a DHCP server on a separate network. This DHCP server has multi-homed connections. One connection to our network and the other to another network with their own IP range which serve a different set of machine. For some reasons, that DHCP was offering IP addresses to that 20+ users.

From the look of it, the only way the 20+ machines can reach that DHCP server is through the connection that server has with our network. But still, those 20+ machines will not get IP address from that DHCP server because there is no IP Helper configured on our switches for that DHCP server. Did a show run command on the switches to confirm that. Nevertheless, I disconnected the connection between that DHCP server and our network. Tried renewing the IP address on the user’s machine but still no luck. Did a route print on the user’s machine and it showed a dynamic routing table of the other network. Did a show ip route command on the switches and confirmed that it is not coming from our switches.

That left me suspecting that a device that had a connection to our stack of switch was also having a connection to the other network. This device is performing routing and even acting as a DHCP relay agent. Did a check on those machines with this multi-homed connections but none was actually configured to do routing. When we were almost at wit’s end, I decided to check on the stack of switches. However, to check on each individual switch port (there are 196 switch ports) will be time-consuming.

Somehow, I thought of listing all the MAC addresses registered on the stack of switches. So I used the command show mac address-table to list out all the MAC addresses and their associated switch ports. Then I got the MAC address of the network card (the one that is connected to their own network) of that DHCP server using ipconfig /all. Found the MAC address in the list of MAC addresses on the stack of switches. Removed the cable from the switch port having that MAC address and it solved the problem. We traced the source of that cable and found out that someone connected the end point to a switch which is connected to the other network. This explained why that 20+ machines can route to the other network and not getting IP addresses from our DHCP servers.

This took us one whole day to troubleshoot, thanks to that whoever person that plug in the cable!!!