I have a switch stack with three WS-C3750X-48 switches running 12.2(55)SE. Recently, stack member 3 went offline for a few minutes, after being online for over a year. A show version on member 3 reports that the system returned to ROM by power-on.

Stack member 3 shares a UPS with a router that did not lose power and all three stack members have stack power with four 350 WATT power supplies between the three of them. There are no crashinfo files on any of the switches, log files show the stack ports on the other switches going offline then switch 3 being removed from the stack.

Surveillance recordings show there was no one near the switch during the time and AAA accounting shows no clues. Is it possible the switch crashed without leaving a trace in the logs, a crashinfo file and report that it returned to ROM by power-on?

I've read before that loose stack cables can cause stack member reboots. Would such an event generate log messages other than the stack ports changing states to down?

Appreciate any help in determining exactly what it was that caused the switch to reload.

did you check the flash on the local switch for crash files?
dir flash3:
I believe would be the command.

check the power plug, I remember a a year or two my company had issues with power cords falling out of switches,
and yes they were mounted with the back of the switch facing down.

_________________
"If you're good at anticipating the human mind. It leaves nothing to chance."
-Jigsaw

may not help you now, but you should consider using stack power, the 3750x's have the ability to use each others power supplys.

Thanks for your suggestions.


I had already checked the individual flashes on each stack member and checked for loose power cables. The cables are in there pretty good and secured to the power supply to prevent accidental unplugging or jiggling issues.


We already have stack power in place. The use of stack power and the fact that a router on the same UPS did not see a service disruption is the reason why I suspect it might have been something other than a power issue, though I'm not ruling it out.

Coincidentally, I was able to confirm that stack power is working correctly. I removed AC power for the only power supply connected to stack member 3 and it remained on using stack power. I'm now leaning towards a hardware or software problem.

should have mentioned this earlier,
are there any events reported in the syslog server
around the time of the failure

_________________
"If you're good at anticipating the human mind. It leaves nothing to chance."
-Jigsaw

Syslogs show the two neighbor switches in the stack detected that their stack ports connected to the problem switch changed state to down and then that switch member 3 being removed from the stack. Nothing else in the logs seems to hint at what happened.

The fact that both neighbor switches reported their stack ports going down at the exact same time seems to rule out my previous theory that it might have been a loose stack cable.

To top it off, no SmartNet

The same switch in the stack mysteriously reloaded again almost seven days later. We got TAC to take a look but couldn't identify any specific signs of what the cause was, they just said it was a bad switch and issued an RMA. New switch in place for almost two weeks now without issue.