http://www.theregister.co.uk/2012/06/01 ... rnet_loop/

I'm presuming that it was a spanning tree issue, of course.
Still, whoops.

Yeah this affected us big time yesterday. Over 300 peers down. Traffic was all over the place. Our router CPU was going through the roof


Remember there is no such thing as a spanning-tree loop. Spanning-tree prevents loops. What you're looking at is a layer2 loop!

_________________
www.mellowd.co.uk/ccie/

Whoops a daisy!



Perhaps someone didn't configure it?

I love how they used the very technical term "titsup" to describe what happened.

_________________
blog.brokennetwork.ca

Damn that would suck. Loops are a pain in the ass.

Titsup indeed.

LINX has 2 discrete LANS. It used to be Extreme and Brocade, but they swapped out all the Brocade with Juniper in the last few months.

Around 16:30 yesterday I noticed a quick OSPF flap between my router connected to LINX and another router. When I logged on I saw the CPU was at around 99% and half my Linx sessions were down. I tried pinging a few neighbours on the Extreme LAN and it was very intermittent so I shut that link down and hoped that would be it. But 5 minutes later I was dropping half my peers on the Juniper LAN as well. So we shut that one as well. We still had transit but of course our transit providers are also connected at various peering points and the bigger players (BBC, Google, Facebook) are all on the Linx as well so it was a good 10 minutes of nothing working.

Either this new system they switched on was not properly tested, or a Linx member somehow bridged their 2 interfaces together. But that would've needed 2 member to do that so I don't think it's feasible. Either way we have not been given a good reason for what actually happened. At least 60 of my peers are down so they are not enabling their interfaces yet.

_________________
www.mellowd.co.uk/ccie/

Glad you mentioned it I would have read right past that without a second thought!

Does suck though!! This is the new PTX 5000 they just deployed so Im wondering if it is a bug of some sort???

_________________
http://blog.movingonesandzeros.net/

Just out of interest, this is what happened on our side

_________________
www.mellowd.co.uk/ccie/

I like the photo caption - "Juniper's shiny PTX goes live..."

-Otanx

Isn't that what the folks in the UK call a wobbly?

We call it Tits

_________________
www.mellowd.co.uk/ccie/

I find that term sexist and derogatory, I am offended. I will be suing your country to stop it's usage and
monetary damages for pain and suffering I have experienced having this knowledge.

_________________
"If you're good at anticipating the human mind. It leaves nothing to chance."
-Jigsaw

Sue the country

_________________
www.mellowd.co.uk/ccie/

I thought it was titsup? No?

Depends if she's on her back

_________________
www.mellowd.co.uk/ccie/

Another graph. This is the LINX traffic flow itself

_________________
www.mellowd.co.uk/ccie/

Nothing wrong with tits.

_________________
"Right actions in the future are the best apologies for bad actions in the past."

I thought it was tits up

_________________
"Right actions in the future are the best apologies for bad actions in the past."