All other Cisco networking related discussions.
Reggle
Post Whore
Posts:
1956
Joined:
Sun May 15, 2011 4:16 pm
Certs:
CCNA Security, CCNP, CCDP

WCCP not working between Squid (OpenBSD) and 3560.

Thu Jul 26, 2012 4:51 am

Hi,

I'm testing WCCP in a lab environment (Another checkbox on my way to CCIE).

The setup
- a WS-C3560-8PC switch running IOS 15.0(1), IP Services with crypto.
- Two client computers connected by wire to the switch, running Windows 7.
- A virtual machine in bridged mode running on one of the machines, running OpenBSD 5.0 with Squid 2.7 installed and running.
- Everything in the same subnet: 192.168.163.0/24, the OpenBSD is at .5, the switch at .3 and functions as the default-gateway for the computers with no ICMP redirects (the real gateway is at .1 but the switch forwards everything).

Squid seems to work, albeit inefficient, but that's not the issue. Filling in the IP of the OpenBSD in the browser as proxy with the proper port works.

Since the 3560 does only support WCCP over layer 2 adjacencies and masks, not hash buckets, I've configured these options on both the Squid and the 3560.

3560 relevant configuration:
Code: Select all
ip wccp web-cache
ip wccp 0 group-list ACL-WCCP
int vlan1
 ip wccp 0 redirect in

Standard IP access list ACL-WCCP
    10 permit any
Extended IP access list ACL-PROXY
    5 deny ip host 192.168.163.5 any
    10 permit tcp 192.168.163.0 0.0.0.255 any eq www
    20 deny ip any any

It should be noted that if I do a 'ip wccp 0 redirect-list ACL-PROXY', the command works, but the 'ip wccp 0 group-list ACL-WCCP' disappears from the running config, and visa versa. Mutually exclusive and it's not clear to me why.

The Squid config under /etc/squid/squid.conf:
Code: Select all
wccp2_router 192.168.163.3
wccp2_forwarding_method 2 (1 = GRE, 2 = layer 2 forwarding to Squid)
wccp2_return_method 2 (1 = GRE, 2 = layer 2 return traffic)
wccp2_assignment_method 2   (1 = hash, 2 = mask)
wccp2_service standard 0
wccp2_weight 10000 (0, the default, does not change anything)

The logdata
Wireshark capture:
http://cloudshark.org/captures/adea08a50624

Logging output on the 3560 with debugging:
Code: Select all
WS-C3560-8PC#
Jul 23 22:19:18.852: WCCP-EVNT:S0: updating wc orig assign info
Jul 23 22:19:18.852: WCCP-EVNT:S0: allocate wc orig mask info (28 bytes)
Jul 23 22:19:18.852: WCCP-PKT:S0: Sending I_See_You packet to 192.168.163.5 w/ rcv_id 0000020B
WS-C3560-8PC#
Jul 23 22:19:28.818: WCCP-EVNT:S0: updating wc orig assign info
Jul 23 22:19:28.818: WCCP-EVNT:S0: reuse wc orig mask info (28 bytes)
Jul 23 22:19:28.818: WCCP-EVNT:S0: no srvc grp mask data to validate
Jul 23 22:19:28.818: WCCP-EVNT:S0: nexthop update oce for wc 192.168.163.5 0x5529170
Jul 23 22:19:28.818: WCCP-EVNT:S0: track nexthop for wc 192.168.163.5 (OK)
Jul 23 22:19:28.818: WCCP-EVNT:S0: created adjacency interest, 192.168.163.5
Jul 23 22:19:28.818: WCCP-EVNT:S0: L2 adjacency added for 192.168.163.5
Jul 24 00:19:28.818: %WCCP-5-SERVICEFOUND: Service web-cache acquired on WCCP Client 192.168.163.5
WS-C3560-8PC#
Jul 23 22:19:28.818: WCCP-PKT:S0: Received valid Here_I_Am packet from 192.168.163.5 w/rcv_id 0000020B
Jul 23 22:19:28.818: WCCP-EVNT:S0: Building new router view
Jul 23 22:19:28.818: WCCP-EVNT:S0: deallocate rtr_view (24 bytes)
Jul 23 22:19:28.818: WCCP-EVNT:S0: allocate mask rtr_view (60 bytes)
Jul 23 22:19:28.818: WCCP-EVNT:S0: copy orig info (28 bytes)
Jul 23 22:19:28.818: WCCP-EVNT:S0: Assignment wait timer started
Jul 23 22:19:28.826: WCCP-EVNT:S0: Built new router view: 1 routers, 1 usable WCCP clients, change # 00000008
Jul 23 22:19:28.826: WCCP-PKT:S0: Sending I_See_You packet to 192.168.163.5 w/ rcv_id 0000020C
WS-C3560-8PC#
Jul 23 22:19:38.784: WCCP-EVNT:S0: updating wc orig assign info
Jul 23 22:19:38.784: WCCP-EVNT:S0: reuse wc orig mask info (28 bytes)
Jul 23 22:19:38.784: WCCP-EVNT:S0: no srvc grp mask data to validate
Jul 23 22:19:38.792: WCCP-EVNT:S0: L2 adjacency added for 192.168.163.5
Jul 23 22:19:38.792: WCCP-PKT:S0: Sending I_See_You packet to 192.168.163.5 w/ rcv_id 0000020D
WS-C3560-8PC#
Jul 23 22:19:43.767: WCCP-EVNT:S0: setting up wc mask assignments
Jul 23 22:19:43.767: WCCP-EVNT:S0: allocate current assign info (1052 bytes)
Jul 23 22:19:43.767: WCCP-EVNT:S0: set wc current assign info (1052 bytes)
Jul 23 22:19:43.767: WCCP-EVNT:S0: verifying mask-value adjacency map (64)
Jul 23 22:19:43.767: WCCP-EVNT:S0: Building new router view
Jul 23 22:19:43.767: WCCP-EVNT:S0: reuse rtr_view (44 of 60 bytes)
Jul 23 22:19:43.767: WCCP-EVNT:S0: copy blank current info
Jul 23 22:19:43.767: WCCP-EVNT:S0: Assignment wait timer stopped
Jul 23 22:19:43.767: WCCP-EVNT:S0: Built new router view: 1 routers, 1 usable WCCP clients, change # 00000008
WS-C3560-8PC#
Jul 23 22:19:43.767: WCCP-EVNT:S0: Redirect_Assignment packet from 192.168.163.5, cache info updated
Jul 23 22:19:43.767: WCCP-PKT:S0: Received valid Redirect_Assignment packet from 192.168.163.5 w/rcv_id 0000020D
WS-C3560-8PC#show ip wccp 0 view
    WCCP Routers Informed of:
        -none-

    WCCP Clients Visible:
        -none-

    WCCP Clients NOT Visible:
        -none-

WS-C3560-8PC#
Jul 23 22:19:48.758: WCCP-EVNT:S0: updating wc orig assign info
Jul 23 22:19:48.758: WCCP-EVNT:S0: reuse wc orig mask info (28 bytes)
Jul 23 22:19:48.758: WCCP-EVNT:S0: wc assignment validated
Jul 23 22:19:48.758: WCCP-EVNT:S0: L2 adjacency added for 192.168.163.5
Jul 23 22:19:48.758: WCCP-PKT:S0: Sending I_See_You packet to 192.168.163.5 w/ rcv_id 0000020E
WS-C3560-8PC#

After that it just loops the last lines.

The problem
The problem is that a WCCP neighborship seems to form between Squid and the switch, and the switch even reacts to the WCCP frames sent out by Squid, but the switch does not start any actual forwarding of http traffic towards Squid. Wireshark only shows WCCP control frames and an occasional ARP, but nothing else is sent towards 192.168.163.5 .

Any thoughts or input are welcome!
http://reggle.wordpress.com

Retired Account
Post Whore
Posts:
3512
Joined:
Mon Nov 16, 2009 8:10 pm

Re: WCCP not working between Squid (OpenBSD) and 3560.

Fri Jul 27, 2012 12:34 pm

Hmmm, actually I'm not sure if the neighborship is formed and stable. It shows building the router view twice.. it should build once then be stable until a change.

- You're using service ID 0 for Squid, so remove the web-cache command from the 3560. Web-cache is a "well-known" service ID - you're using a dynamic ID. Also, does the Squid know to tie ports to that service ID, or are you missing configuration? It is the responsibility of the WCCP client (your Squid) to define the characteristics of the service group (src/dest port) to the WCCP server (your switch) when using a dynamic ID.
- You've configured L2/MASK/L2 which is good for 95% of switches.
- I can't remember if that switch supports utilizing an ACL to redirect traffic, but it looks like you have one configured and not tied to anything. If you want to use the ACL to identify interesting traffic to redirect, your config line needs to look like:

ip wccp 0 group-list ACL-WCCP redirect-list ACL-PROXY

Otherwise you redirect everything that matches the port definition of the service group, except traffic from the WCCP client (your Squid).
Last edited by Retired Account on Fri Jul 27, 2012 12:43 pm, edited 1 time in total.

Retired Account
Post Whore
Posts:
3512
Joined:
Mon Nov 16, 2009 8:10 pm

Re: WCCP not working between Squid (OpenBSD) and 3560.

Fri Jul 27, 2012 12:41 pm

If you want more information I did two write-ups on WCCP here: http://www.networks-wetworks.com/search/label/WCCP

The overview doc I've since updated, and it was using a Websense Content Gateway, but the fundamental concepts may help.

Reggle
Post Whore
Posts:
1956
Joined:
Sun May 15, 2011 4:16 pm
Certs:
CCNA Security, CCNP, CCDP

Re: WCCP not working between Squid (OpenBSD) and 3560.

Fri Jul 27, 2012 7:00 pm

'ip wccp web-cache' makes the switch respond to WCCP advertisements of Squid, so I have to leave that command.
Your 'ip wccp 0 group-list ACL-WCCP redirect-list ACL-PROXY' was key to the config on the switch, I had tried both separately and they excluded each other when not done in a single command.
Defining the proper service group was also needed as you suggested.
After this a neighborship clearly formed:
Code: Select all
WS-C3560-8PC#show ip wccp 0 view
    WCCP Routers Informed of:
        192.168.163.3

    WCCP Clients Visible:
        192.168.163.5

    WCCP Clients NOT Visible:
        -none-

However, still no forwarding of layer 2 frames. I changed the ACLs a few times to no avail. Off to bed now, more in the morning.

I've read your website already but I think I'll reread it then too.
http://reggle.wordpress.com

User avatar
Vito_Corleone
Moderator
Posts:
9850
Joined:
Mon Apr 07, 2008 10:38 am
Certs:
CCNP RS, CCNP DC, CCDP, CCIP

Re: WCCP not working between Squid (OpenBSD) and 3560.

Fri Jul 27, 2012 7:58 pm

Can't do denies in the WCCP ACLs on those switches.
http://blog.alwaysthenetwork.com

User avatar
burnyd
Post Whore
Posts:
3160
Joined:
Fri Nov 13, 2009 5:15 pm
Certs:
CCIE R&S/SP,CCNP-SP,JNCIA,VCP510,VCA-DCV

Re: WCCP not working between Squid (OpenBSD) and 3560.

Fri Jul 27, 2012 8:15 pm

get rid of your grouplist and get rid of the services you are denying caching for. Make sure that FreeBSD box is running it on the L2 mode as I believe all switchs other than 6500s cannot run in tunnel mode.

I couldnt get my 3560 to work properly like this so I moved it to my 2811 router to redirect through a gre tunnel to a Ubuntu box.
http://danielhertzberg.wordpress.com - I blog about networks!

Reggle
Post Whore
Posts:
1956
Joined:
Sun May 15, 2011 4:16 pm
Certs:
CCNA Security, CCNP, CCDP

Re: WCCP not working between Squid (OpenBSD) and 3560.

Fri Jul 27, 2012 8:19 pm

Vito: removed denies, same result. But you did help pinpoint it a bit more: I see hits on the ACL-WCCP which increase because of the control frames, but I see no hits on the ACL-PROXY.

And yes, insomnia.

Burnd: the OpenBSD is running it on the L2 mode, I'm sure. I'm not sure I understand what you mean with 'get rid of the grouplist'.
http://reggle.wordpress.com

javentre
Post Whore
Posts:
1971
Joined:
Fri Jul 09, 2010 7:38 pm

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sat Jul 28, 2012 9:09 am

What SDM template are you using?

-Sent using Tapatalk.
http://networking.ventrefamily.com

Reggle
Post Whore
Posts:
1956
Joined:
Sun May 15, 2011 4:16 pm
Certs:
CCNA Security, CCNP, CCDP

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sun Jul 29, 2012 6:42 am

Code: Select all
WS-C3560-8PC#show sdm prefer
 The current template is "desktop IPv4 and IPv6 routing" template.
 The selected template optimizes the resources in
 the switch to support this level of features for
 8 routed interfaces and 1024 VLANs.

  number of unicast mac addresses:                  1.5K
  number of IPv4 IGMP groups + multicast routes:    1K
  number of IPv4 unicast routes:                    2.75K
    number of directly-connected IPv4 hosts:        1.5K
    number of indirect IPv4 routes:                 1.25K
  number of IPv6 multicast groups:                  1.125k
  number of directly-connected IPv6 addresses:      1.5K
  number of indirect IPv6 unicast routes:           1.25K
  number of IPv4 policy based routing aces:         0.25K
  number of IPv4/MAC qos aces:                      0.5K
  number of IPv4/MAC security aces:                 0.5K
  number of IPv6 policy based routing aces:         0.25K
  number of IPv6 qos aces:                          0.625k
  number of IPv6 security aces:                     0.5K
http://reggle.wordpress.com

Reggle
Post Whore
Posts:
1956
Joined:
Sun May 15, 2011 4:16 pm
Certs:
CCNA Security, CCNP, CCDP

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sun Jul 29, 2012 9:09 am

Update: I tried putting the proxy in a second VLAN, directly attached to the switch. Theory behind it was that maybe the switch wouldn't like to send redirected frames out of the same interface they entered.
For some weird reason this stopped Squid from sending WCCP frames altogether, despite updated IPs and firewall rules. Still researching.
http://reggle.wordpress.com

javentre
Post Whore
Posts:
1971
Joined:
Fri Jul 09, 2010 7:38 pm

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sun Jul 29, 2012 11:23 am

Did the switch come up with that SDM template, from a reload?
http://networking.ventrefamily.com

Reggle
Post Whore
Posts:
1956
Joined:
Sun May 15, 2011 4:16 pm
Certs:
CCNA Security, CCNP, CCDP

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sun Jul 29, 2012 2:40 pm

It's not it's default template if that's what you ask. I've set it to dual-ipv4-and-ipv6 months ago and it has had several reboots now, even while experimenting with WCCP. I haven't changed the template while trying WCCP however. I haven't received any warnings about it either, and it's one of the few templates that allocates resources to each of those features.
http://reggle.wordpress.com

javentre
Post Whore
Posts:
1971
Joined:
Fri Jul 09, 2010 7:38 pm

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sun Jul 29, 2012 2:51 pm

Reggle wrote:I've set it to dual-ipv4-and-ipv6 months ago and it has had several reboots now
OK. I asked because many of your symptoms sound like what I've experienced, more than a few times, when I kept it at the default. The syslog entry is there when I go back and look, but I often miss it.
http://networking.ventrefamily.com

Reggle
Post Whore
Posts:
1956
Joined:
Sun May 15, 2011 4:16 pm
Certs:
CCNA Security, CCNP, CCDP

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sun Jul 29, 2012 4:12 pm

Well, I hate to say this, but I'm giving up.
I'm unable to get Squid going on another VLAN despite reconfiguring three times, just changing the IP's stops it from sending WCCP frames, no idea why.
In the same VLAN the neighborship is okay but no frames are sent. I've unchecked IPv6 everywhere (noticed some websites go over IPv6 and thus don't match the IPv4 ACL), expanded the ACL to match traffic, rebooted everything, rechecked everything. No syslogs indicating any issue.

Code: Select all
Standard IP access list ACL-WCCP
    10 permit 192.168.163.5 (119 matches)
Extended IP access list ACL-PROXY
    10 permit tcp 192.168.163.0 0.0.0.255 any eq www
    20 permit tcp 192.168.163.0 0.0.0.255 any
    30 permit tcp any any eq www
    40 permit ip 192.168.163.0 0.0.0.255 any
    50 permit ip any any


No matches on the ACL. It's my home lab. If this was production environment, I'd be opening a TAC case right now. Seems the switch is just not forwarding data despite all attempts to make it so.
http://reggle.wordpress.com

Retired Account
Post Whore
Posts:
3512
Joined:
Mon Nov 16, 2009 8:10 pm

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sun Jul 29, 2012 9:12 pm

The group list just defines who can participate in the service group. It's completely optional - you could have not used that at all which just means that any device advertising HIA packets for a service ID configured on the switch can participate.

For now, keep it simple and change your redirect list to allow everything. Once you've verified it starts working, then fine tune it. Also, you don't need to specify ports in the redirect list ACL (This is already defined in the service group) - again just keep it simple. The only time I've really needed it is when working with ASAs. I always use more specific ACLs there.

In the same VLAN the neighborship is okay but no frames are sent

Since you're using L2, the proxy needs to reside on a seperate subnet from the hosts it is servicing.

Lastly, I don't understand why using web-cache is required. You specifying web-cache and and service ID 0 is conflicting. Also, in your capture I only see the web-cache service, not ID 0. Just like the Websense CG, you should be able to define a service group, what ports are associated, and it should only use that service group. You shouldn't be utilizing two service groups for one service (HTTP).
Last edited by Retired Account on Sun Jul 29, 2012 9:16 pm, edited 1 time in total.

User avatar
Vito_Corleone
Moderator
Posts:
9850
Joined:
Mon Apr 07, 2008 10:38 am
Certs:
CCNP RS, CCNP DC, CCDP, CCIP

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sun Jul 29, 2012 9:14 pm

I use specific ports when I deploy WCCP. No point in sending packets that won't be accepted.
http://blog.alwaysthenetwork.com

Retired Account
Post Whore
Posts:
3512
Joined:
Mon Nov 16, 2009 8:10 pm

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sun Jul 29, 2012 9:28 pm

That's a myth outside of the ASA. That's why I use specific L4 statements in redirect lists with ASAs.

What ports to be redirected are defined in the service group already (Do a pcap on udp port 2048 to confirm) - test it yourself on a switch. It will only redirect what you've defined in the service group (port 80, port 443, etc.) despite allowing everything in the redirect list. I've had hit/miss behavior with this on an ASA, so I always specify what ports are to be redirected. However, it doesn't hurt to specify outside of the ASA as well.

Another nice thing about built-in mechanisms of WCCP is that you don't have to specify a deny statement for your proxy (Another myth was that a deny statement had to be present to avoid loops) - THe WCCP Server has a view of what WCCP clients it has, so if it recieves traffic from one of it's clients, it will not redirect it - even if you specifically permit it in the redirect list ACL.
Last edited by Retired Account on Sun Jul 29, 2012 9:47 pm, edited 1 time in total.

Retired Account
Post Whore
Posts:
3512
Joined:
Mon Nov 16, 2009 8:10 pm

Re: WCCP not working between Squid (OpenBSD) and 3560.

Sun Jul 29, 2012 9:29 pm

This is a silly question, but the traffic -is- sourcing from that /24 subnet right? It's not hitting a NAT beforehand?

Reggle
Post Whore
Posts:
1956
Joined:
Sun May 15, 2011 4:16 pm
Certs:
CCNA Security, CCNP, CCDP

Re: WCCP not working between Squid (OpenBSD) and 3560.

Mon Jul 30, 2012 1:56 am

Steven King wrote:This is a silly question, but the traffic -is- sourcing from that /24 subnet right? It's not hitting a NAT beforehand?

No NAT involved anywhere, just the local subnet. Currently not in my lab, I will take into account your tips later.
http://reggle.wordpress.com

User avatar
mellowd
CCIE #38070
Posts:
13814
Joined:
Wed Jun 18, 2008 7:49 am
Certs:
CCIE (RS,SP), JNCIE-SP, BC-/SPNE/NP

Re: WCCP not working between Squid (OpenBSD) and 3560.

Mon Jul 30, 2012 3:00 am

Reggle wrote:Well, I hate to say this, but I'm giving up.


You can't give up in the lab :)

'
Next

Return to Cisco General

Who is online

Users browsing this forum: Majestic-12 [Bot] and 58 guests