HP iLO stops responding to ping from other subnets in a couple of minutes

SYMPTOMS:

iLO network interface (be it shared or dedicated) is set up with static IP address and a gateway. One can access and ping it from the same LAN subnet with no issues; however when pinged from a different subnet through the router pings stop after random amount of time, but generally within a few minutes. Resetting the iLO resolves the issue for another couple of minutes. This applies to iLO 3, iLO 4, iLO 5 and probably iLO 6 as well.

This problem has been described on the Internet number of times [1] [2] [3] with no clear  resolution provided.

CAUSE: 

 iLO management interface honors ICMP Type 9 - Mobile IP Router Advertisements (RA) for IPv4 even when static gateway IP address has been set in the configuration. When such RA is received by iLO, the gateway being used is replaced with one specified in RA; this, however, is not being reflected in the Web interface. 

iLO then proceeds with sending ARP request for this new gateway address and forwards all non-local traffic through it.

At this moment communication with any IP addresses not on the same LAN subnet ceases until the iLO is reset.

In the example Wireshark capture below, the Cisco SG350 switch with address 192.168.3.4 sends rogue Mobile IP RA that is being picked up by iLO3 with address 192.168.3.6. The switch itself has a defunct default gateway 192.168.3.252 configured that it tries to unsuccessfully resolve.

As this capture has been performed on the router, no further communication is observed from the iLO, as it communicates with the switch directly.

 

RESOLUTION: 

 Find the device sending ICMP Type 9 RAs and either disable this functionality, or make sure that the information provided in RAs is correct and the advertising router does have ability to route the traffic appropriately - e.g. there's a correct default gateway set on the advertising router itself.

If your switch supports multicast filtering, it might be useful to filter all-hosts multicast group 224.0.0.1 or 01:00:5e:00:00:01 on interfaces facing iLOs.

Comments

Post a Comment

Popular posts from this blog

HP DL380 G7 won't power up after a power cord removal/power loss

Greetings,      If you came here it means that you see what I'd seen - a once healthy DL380G7 after power cord removal sitting with orange power led and no reaction to power button, or the same server with a red health light blinking or solid and a bizarre combination of orange leds on health panel indicating various nonexistent failures (e.g. CPU failure, Fan solution not sufficient, DIMM error, etc).      Likewise, if you're here, you've likely read all the musings about this same situation on HPE/serverfault/stackexchange and neither of those did help your recovery. You might have even contacted HPE to get a solid quote on motherboard replacement. Been there, seen that. In fact, we've almost trashed one of our own servers, but then we've decided to go into a deep and thorough troubleshooting.      Now, before you decide to trash your server(s) and opt to HPE services, there's a small but important step you might have missed, cause its not included in all
Read more

ESXi 7.0/6.7 and MegaRaid on Alder Lake Asrock Z690M-ITX/ax

Image
I'm going to upgrade my home server that runs Core i5-2500K on old P8H67-I mainboard with LSI 9265-8i SAS controller and I've done some experiments prior to this upgrade on a new hardware, that is Asrock Z690M-ITX/ax motherboard with i5-12600/i5-12600K processor. Generally I always have a common platform between my main PC and the server so that I can try some stuff ahead of server migration with minimal hassle and it paid off this time. IMPORTANT NOTICE: This configuration is not on HCL and unsupported, you may incur data loss, please read this article carefully before making decisions or changing your setup. Always have a backup available off the system! Part I: MegaRAID cards firmware woes The experimental setup was Asrock Z690M-ITX/ax BIOS ver 7.02, i5-12600K (server will have plain 12600 though) and LSI 9260-4i card (LSI 2108 family, Dell PERC H700 compatible) with RAID-1. The server I'm going to upgrade has 9265-8i, that is LSI 2208 or Dell PERC H710, different family
Read more

Accessing MegaRAID BIOS (WebBIOS, Ctrl-H) on consumer motherboards

Greetings,             It's been a while I'm using LSI Logic 9260-4i RAID controller in a mini-ITX motherboard, in my case it is ASUS P8H67-I. What's bug you most when you plan a setup like this is "is my RAID card going to initialize properly?", as most consumer boards assume you'll put some graphics card in them, and won't even consider a scenario above.             Good news is that when I've started this setup two years ago, it worked like a charm. I was able to get into WebBIOS just by pressing Ctrl-H, and everything worked like it should. This, however, changed as soon as I've upgraded firmware on the MegaRAID - pressing Ctrl-H yielded nothing, the system continued to boot without any sign of WebBIOS appearing.
Read more