Hi Guys,

OK a rundown on the network:

- Two watchguard XTM510 firewalls in an active/standby cluster.
- Seven Cisco SG300-28P Small Business switches (two core, five access)
- Seven VLANS.
- The VLAN purposes are as follows:

VLAN10 (management vlan): All IT equipment - access points, firewalls, switches, wireless controllers (2504) etc. etc.

VLAN20: AV Sources - Apple TVs, DVD players etc. etc.

VLAN30: Kaleidescape AV Source (Kaleidescape recommend this goes on its own VLAN which is why it isnt on 20)

VLAN40: Automated room/building control hardware - touchscreens controllers, processors to control the touchscreens, local room boxes that distribute HDMI and audio from the master Digital media switcher.

VLAN50: Staff VLAN

VLAN60: Building management systems

VLAN 80: VoIP/Cameras

TOPOLOGY

The two core switches both have one link to each Watchguard. Every access switch has one link to each core switch.

The Problem

We are experiencing some packet loss on the network. Now, I don't know if this is expected in redundant networks but I haven't seen it before. The packet loss is around 0.25-0.4% over an hour period depending on where you are on the network hard wired in to a switch port. It doesn't seem to be affecting anything but we are seeing things drop off the network momentarily from our monitoring station.

My first thought was that it was something to do with spanning tree topology changes, because it has been misconfigured, but no spanning tree settings have been changed apart from making the core switches the root bridges. I've mapped out the spanning tree topology and everything looks okay; there are no blocked ports where there shouldn't be and no designated ports where there shouldn't be.

My next move was onto the Watchguards. I pulled up the logs and found an error message pertaining to "received packet with source address as own address on interface eth2.20, eth2.30, eth2.40....". This implies, to me, that there was a loop but having mapped out the spanning tree topology there are no loops. I contact WG support who say that this message is because their software has a bug in it..."okay, so what are you going to do about it" I said. Their response was anything but conclusive and concise... Anyway, I disconnected one of the WGs but still got the packet loss issue on the network, but no error log message like the above. So I made the assumption that the Watchguards perhaps weren't playing a role in this problem.

I have stripped down to single links to each of the switches to one core switch and the problem goes away, so I'm thinking its got to be RSTP or some kind broadcast storm.

So my question to you guys is...do you know what is going on or what my next step should be?

NO I dont think it has to do with RSTP...Its just an extension of STP for better uptime.
By default one port will be blocked..leaving a loop free topology..



BUt the after effect of STP can cause an issue...
Like because of STP if a port goes down,that port may be shortest path for some vlans.
So now those Vlans may have to take a longer path...
Now on that longer it will face link utilization cos of other protocols.

Your problem seems like that...


1) Whether u use Any Gateway load balancing protocol (harp/vrrp) on core for these vlans.
2) If yes then who is the active gateway (core-1/core-2)..
3) take one VLAN and check which is the block port for that (In cable unplugged scenario).Must be one port among your access switch ports.
4) Atlast check whether that port is the direct connected port to ACTIVE GATEWAY.
Which will cause traffic to go to other core (Not Active gateway) -----then use Uplink between CORE1-CORE2 and reach the actual ACTIVE GATEWAY.

And be cos of Heavy traffic on that Uplink is causing drops at uneven times...

I have put a lot of assumptions on ur topology (As per best practices) .. As those informations are not in ur description...

Hope this helps...

_________________
- The Sky is Limit

Hi

I was wondering how you got your watchguard to work with spanning tree? I keep getting broadcast storms!!

Thanks