RIP, EIGRP, OSPF, IS-IS, BGP, MPLS, VTP, STP.
bushido
Member
Posts:
129
Joined:
Fri Jan 19, 2007 8:22 am

etherchannel traffic utilization

Wed Dec 03, 2008 8:28 am

hi, i have a question i can't seem to find an answer to on cisco.com.

we have hard-coded etherchannel's on 6500 switches running to 3750 stacks. by hard coded, i mean "channel-group 1 mode on". i can't figure out how to get an accurate reading on how each link of the port-channel is being utilized.

when doing a a "show etherchannel detail", i get the following output:

Group: 8
----------
Group state = L2
Ports: 2 Maxports = 8
Port-channels: 1 Max Port-channels = 1
Protocol: -
Ports in the group:
-------------------
Port: Gi3/11
------------

Port state = Up Mstr In-Bndl
Channel group = 8 Mode = On/FEC Gcchange = -
Port-channel = Po8 GC = - Pseudo port-channel = Po8
Port index = 0 Load = 0x55 Protocol = -

Age of the port in the current state: 49d:17h:02m:47s

Port: Gi4/11
------------

Port state = Up Mstr In-Bndl
Channel group = 8 Mode = On/FEC Gcchange = -
Port-channel = Po8 GC = - Pseudo port-channel = Po8
Port index = 1 Load = 0xAA Protocol = -

Age of the port in the current state: 49d:17h:02m:47s

Port-channels in the group:
----------------------

Port-channel: Po8
------------

Age of the Port-channel = 49d:17h:02m:47s
Logical slot/port = 14/8 Number of ports = 2
GC = 0x00000000 HotStandBy port = null
Port state = Port-channel Ag-Inuse
Protocol = -

Ports in the Port-channel:

Index Load Port EC state No of bits
------+------+------+------------------+-----------
0 55 Gi3/11 On/FEC 4
1 AA Gi4/11 On/FEC 4

Time since last port bundled: 49d:17h:02m:47s Gi4/11


it states the load, but that is in hex. any idea what that correlates to ? on cisco.com they list a traffic utilization command but that is for CatOS - we run IOS. any help is appreciated! thanks!

User avatar
raven
CCIE #20728
Posts:
1450
Joined:
Thu Aug 09, 2007 11:22 am

Wed Dec 03, 2008 8:52 am

why dont you just use the show interface statistics for each physical link?
http://ccie20728.wordpress.com/

bushido
Member
Posts:
129
Joined:
Fri Jan 19, 2007 8:22 am

Wed Dec 03, 2008 9:15 am

i have done that, but i still would like to know what that "load" actually means.

User avatar
raven
CCIE #20728
Posts:
1450
Joined:
Thu Aug 09, 2007 11:22 am

Wed Dec 03, 2008 11:09 am

good question, I did not find an explanation yet but I don't think that it has something to do with the actual load of the link, since our etherchannels got the same hex values as your output :)
http://ccie20728.wordpress.com/

geko29
Ultimate Member
Posts:
952
Joined:
Mon Feb 06, 2006 5:11 pm

Wed Dec 03, 2008 11:47 am

Load is the percentage utilization, where 0 (0x00) is no traffic, and 255 (0xFF) is 100% utilization.

In your output, you have 33% (0x55) and 67% (0xAA) utilization, respectively.

User avatar
ibarrere
Cisco Inferno
Posts:
10283
Joined:
Mon Jul 10, 2006 12:58 am

Wed Dec 03, 2008 12:32 pm

Oh nice! Does it get those statistics from the controller utilization (which you can usually display by "show control util")?

bushido
Member
Posts:
129
Joined:
Fri Jan 19, 2007 8:22 am

Wed Dec 03, 2008 1:05 pm

thanks for the quick reply. but what does that MEAN exactly ? :)

we see those same numbers for all of our 2 channel port-channels. for all of our 3 channel port channels, we see 49, 92 and 24 for the three ports (and in that order). some of those are static and some are pagp - all the same.

so does that mean it pre-allocates the traffic based on those percentages ?

User avatar
ristau5741
Post Whore
Posts:
10547
Joined:
Tue Aug 21, 2007 2:15 pm
Certs:
Instanity

Wed Dec 03, 2008 3:12 pm

show etherchannel load-balance
will tell you the balancing scheme,


I don't think geko is correct, mine are the same numbers

using XOR destination IP as the load balance
show etherchannel port-channel

2 port port-channel
AA
55

3 port port channel
49
92
24

4 port port-channel
11
22
44
88

geko29
Ultimate Member
Posts:
952
Joined:
Mon Feb 06, 2006 5:11 pm

Wed Dec 03, 2008 4:01 pm

ristau5741 wrote:I don't think geko is correct, mine are the same numbers


I think you're right. That's the EIGRP definition of Load, but it apparently doesn't carry over here. It does appear to somehow demonstrate the relative portion of traffic that an interface will carry, however. You'll notice in every single one of your examples, the values all add up to FF (255). Beyond that I don't really know.

bob
Post Whore
Posts:
1277
Joined:
Wed May 14, 2008 1:02 pm

Wed Dec 03, 2008 8:35 pm

in order to get a real picture over an extended period of time, try polling the router w/ cricket (or vitalnet if you have it). you can compile stats for each link over whatever time period you like and then you have a pretty graph to show to management when you want to justify upgrading

User avatar
raven
CCIE #20728
Posts:
1450
Joined:
Thu Aug 09, 2007 11:22 am

Thu Dec 04, 2008 4:57 am

I think ristau got the idea into the right direction. The Load Values are static but they might change with either adding more physical ports or changing the load-balancing method.

Might be interesting to dig further into that thing :-)

Another thing, ristau already showed that the values change with more ports :)
http://ccie20728.wordpress.com/

bushido
Member
Posts:
129
Joined:
Fri Jan 19, 2007 8:22 am

Thu Dec 04, 2008 8:32 am

we are using source XOR destination load-balancing method, and my values are the same as ristau's...

User avatar
ristau5741
Post Whore
Posts:
10547
Joined:
Tue Aug 21, 2007 2:15 pm
Certs:
Instanity

Thu Dec 04, 2008 9:38 am

geko29 wrote:Load is the percentage utilization, where 0 (0x00) is no traffic, and 255 (0xFF) is 100% utilization.

In your output, you have 33% (0x55) and 67% (0xAA) utilization, respectively.


A little research

Geko is correct, right out of the BSMSN study guide

Code: Select all
To verify how effectively a configured load-balancing method is performing, you can use the show etherchannel port-channel command. Each link in the channel is displayed, alongwith a hex “Load” value. Although this information is not intuitive, you can use the hex values
to get an idea of each link’s traffic loads relative to the others.

User avatar
raven
CCIE #20728
Posts:
1450
Joined:
Thu Aug 09, 2007 11:22 am

Thu Dec 04, 2008 10:16 am

Nothing against the study guide but it looks like its not correct. show etherchannel port-channel it shows following output on one of our 6500:

Code: Select all
Ports in the Port-channel:

Index   Load   Port     EC state        No of bits
------+------+------+------------------+-----------
  0     55     Gi2/2    On/FEC    4
  1     AA     Gi5/2    On/FEC    4


Based on that gig 2/2 should have 33% and gig 5/2 should have 67% load but in real its:

Code: Select all
sh int gig 2/2 | i 5 minu
  5 minute input rate 58868000 bits/sec, 6572 packets/sec
  5 minute output rate 54510000 bits/sec, 6266 packets/sec


While 58'868'000 is around 59 Mbit/s on a gig link, thats no where close to 33%.

Only thing I could see is that in average over X period of time, 33% of all traffic is passing over gig2/2 while 67% of all traffic is passing over gig5/2.
Which means that either they never try to get a 50/50 traffic split with their formula (cause Bushido got the same values) or our traffic pattern does not really match our load balancing configuration :)
http://ccie20728.wordpress.com/

bushido
Member
Posts:
129
Joined:
Fri Jan 19, 2007 8:22 am

Thu Dec 04, 2008 10:49 am

yea, that study guide can't be right....i think i'll send this off to our cisco rep and see if he can find anything. i'll let you guys know.

bushido
Member
Posts:
129
Joined:
Fri Jan 19, 2007 8:22 am

Thu Dec 04, 2008 3:01 pm

well, the cisco rep came through. he provided me with a document, quoted below. case solved.

Catatalyst 6000/6500 Load Balancing algorithm :


First, find out what version of the EARL the customer is having.
Get the output of 'show mod'. At the end of the output you would see something like:

Mod Sub-Type Sub-Model Sub-Serial Sub-Hw
--- ----------------------- ------------------ ---------- ------
1 L3 Switching Engine WS-F6K-PFC SAD03490532 1.0
In the above output, look at the 'Sub-Model' number and determine the EARL version.

Earl differences:
WS-F6020 = EARL4
WS-F6K-PFC = EARL5
WS-F6k-PFC2 = EARL6
WS-F6K-PFC3 = EARL7
WS-F6K-PFC3B = EARL7 (with SuperTycho or above)
WS-F6K-PFC3BXL = EARL7 (with SuperTycho or above)


Bundle Hash Algorithm used:

EARL4/5 - Hash Algorithm - X^17+x^4+1 - supports only L2 (SA (or) DA (or) SA/DA) & L3 (SA (or) DA ( or) SA/DA) based load balancing
EARL6 - Hash Algorithm - X^17+X^3+1 - supports L2 (SA (or) DA (or) SA/DA) & L3 (SA (or) DA (or) SA/DA) & L4 (SA (or) DA (or) SA/DA)
based load balancing
EARL7 - Hash Algorithm - X^17+X^4+1 - supports L2 (SA (or) DA (or) SA/DA) & L3 (SA (or) DA (or) SA/DA) & L4 (SA (or) DA (or) SA/DA) (or) MPLS tag based load balancing (More options available Check ENG-275202)

It's a complicated 17th-degree polunomila algorithm that was implemented in Hardware. It's only mentioned in CAFE HW Specs.(ENG-15114). The way it worksis: The switch takes the IP or MAC address(depends on the HW Rev.) and applies the algorithm to it and generates a 3-bit hash value. This is done separately or both Source and Destination addresses.

The hash values are combined in Exclusive-OR way to generate a unique 3-bit value. Based on this value, one of the 8-ports are chosen. Before this is done, anotherregister(dsel_reg) will be set based on the number of ports in a channel. For example, if only 2-ports are in a channel, the dsel register is set in such a way that the alternate hash values are applied to each port of a channel. If you look at the following command, by varying the number of ports of a channel, you will get a clear picture as how dsel_reg will be set.


Note:

The way the traffic is distributed among the bundle is, the earl computes rbh (result bundle hash) value in the range 0..7 for every flow based on the config (mac addres or ip or port numbers, destination only or source only or src xor destn). The ports are setup with a mask which indicates which values it would accept for transmission.

so if you had 8 ports, each port would accept one value.
1, 2, 4, 8, 10 etc.
if you had 4 ports, each port would accept two values. mask would
be
11, 22, 33, 44
if you had 2 ports, each port would accept four values
AA, 55
When you have odd number of ports like 3, 5 or 7, some ports will have to takeup extra values than other ports.

3 ports : the distribution would be port 1 - 3 values,
port 2 - 3 values,
port 3 - 2 values
5 ports : the distribution would be 2, 2, 2, 1, 1

So, in essense perfect load balancing even with randomized addresses can be achieved only if you have 2,4 and 8 port channels.

For other port channels the load balancing would be :
For 6 port channel the load balancing would be 2:2:1:1:1:1
For 3 port channel the load balancing would be 3:3:2
For 5 port channel the load balancing would be 2:2:2:1:1
For 6 port channel the load balancing would be 2:2:1:1:1:1
For 7 port channel the load balancing would be 2:1:1:1:1:1:1

For Native IOS:

esc-6513#show interfaces port-channel 1 etherchannel
Age of the Port-channel = 00d:00h:08m:58s
Logical slot/port = 14/3 Number of ports = 2
GC = 0x00010001 HotStandBy port = null
Passive port list = Gi9/1 Gi9/3
Port state = Port-channel L3-Ag Ag-Inuse
Ports in the Port-channel:
Index Load Port EC state
------+------+------+------------
0 55 Gi9/1 on <---for eg, we have a 2-port channel here and '55' is 4-bits used.
1 AA Gi9/3 on and 'AA' is 4-bits used. so, the load balancing will be 4:4 (ie equal)

Time since last port bundled: 00d:00h:01m:38s Gi9/3
Time since last port Un-bundled: 00d:00h:02m:22s Gi9/3

For CatOS:

In this eg, I have a 3-port channel, as you can see port 1 takes 3 values, port 2 takes 3 values and port 3 takes
3 values. So, the load balancing would be 3:3:2
esc-c6509-c (enable) show agport
--- 13/1 ---
old_mem_cnt = 0; path_cost = 2; path_vlan_cost = 2
trunk_id = 768, time_stamp = 109395971, agifindex = 799
chnl_list = 1/1-2,2/1
agport_list = 1/1-2,2/1
bndlctrl: prtcnt = 3, num_map = e0, dist_req = 2, dist_port = 0
mod port bndl_port bndl_sel bndl_sel* act_flag no_bits
1 1 0 e0 e0 2 3
1 2 1 1c 1c 2 3
2 1 2 03 03 2 2
0 0 0 00 00 0 0
0 0 0 00 00 0 0
0 0 0 00 00 0 0
0 0 0 00 00 0 0
0 0 0 00 00 0 0


In the Engineering mode, there is a command to give an ip-address (or) mac-address
and checkout what the output port would be.
For eg:
esc-6500-b> (debug-eng) show bndlhash 00-d0-bc-eb-70-b8 00-d0-bc-eb-70-b9
Usage: show bndlhash <channle_id> <address1> [address2]
mac address use: xx-xx-xx-xx-xx-xx; ip address use:xxx.xxx.xxx.xxx
if channel_id is 0 then return hash value only, otherwise mod/port
esc-6500-b> (debug-eng) sh port channel
Port Status Channel Admin Ch
Mode Group Id
----- ---------- --------- ----- -----
4/1 connected desirable 215 865
4/2 connected desirable 215 865

From the above command, you have the 'Channel-id' (in this case 865), then you can find
the output port for a given pair of mac-addresses (or) ip-addresses:
esc-6500-b> (debug-eng) show bndlhash 865 00-d0-bc-eb-70-b8 00-d0-bc-eb-70-b9
Selected port: 4/2
esc-6500-b> (debug-eng) show bndlhash 865 10.10.10.1 20.20.20.3
Selected port: 4/2
esc-6500-b> (debug-eng) show bndlhash 865 10.10.10.1 20.20.20.4
Selected port: 4/1

(Note : Starting from Orlando (6.1.x and above), all the above commands are moved from Engineering
mode to priviledged mode. The commands would be 'set channel hash & show channel hash'.

geko29
Ultimate Member
Posts:
952
Joined:
Mon Feb 06, 2006 5:11 pm

Thu Dec 04, 2008 3:31 pm

That's very interesting, and nicely explains why the values always add up to 255 (if they didn't, some traffic would not match a port and would be dropped).

Thanks for that bushido.

User avatar
raven
CCIE #20728
Posts:
1450
Joined:
Thu Aug 09, 2007 11:22 am

Fri Dec 05, 2008 4:27 am

yes thanks for that post :-)
http://ccie20728.wordpress.com/

User avatar
ristau5741
Post Whore
Posts:
10547
Joined:
Tue Aug 21, 2007 2:15 pm
Certs:
Instanity

Fri Dec 05, 2008 9:25 am

still doesn't answer the original question.

User avatar
b1tst0rm
Member
Posts:
169
Joined:
Wed Dec 05, 2007 11:10 pm
Certs:
CCDP/CCNP/CCSP/CCIP

Fri Dec 05, 2008 9:19 pm

try this:

Console> (enable) show channel traffic
ChanId Port Rx-Ucst Tx-Ucst Rx-Mcst Tx-Mcst Rx-Bcst Tx-Bcst
------ ----- ------- ------- ------- ------- ------- -------
808 2/16 0.00% 0.00% 50.00% 75.75% 0.00% 0.00%
808 2/17 0.00% 0.00% 50.00% 25.25% 0.00% 0.00%
816 2/31 0.00% 0.00% 25.25% 50.50% 0.00% 0.00%
816 2/32 0.00% 0.00% 75.75% 50.50% 0.00% 0.00%
Console> (enable)


http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/catos/6.x/configuration/guide/channel.html#wp1022181

'
Next

Return to Cisco Routing and Switching

Who is online

Users browsing this forum: No registered users and 34 guests