Oohhhhh boy I'm in deep.

So, I'm working a project to replace some datacenter equipment in a server farm. Currently, 65xx switches are used mostly, with some 35xx switches for smaller areas. We're looking at certifying Nexus, as well as Brocade's NetIron family. Not only am I new to Nexus, and NetIron, I'm also completely new data centers. I had to look up what End-of-Row and Top-of-Rack meant.

I was on a call with the architect and I'm pretty sure he has a gameplan in mind, but he's testing me and asking how I would view the future of the server farm. Honestly I have no idea. I don't even know what a server farm looks like, physically. The architect is also in the mindset that he doesn't want to "drink the vendor's orange juice", and wants us to tell them what we want it to look like.

So now I'm in scramble mode trying to learn as much as I can about where switches are placed, the logic behind, cabling mgmt, etc.

If you were going to replace a bunch of chassis switches with Nexus, how would you go about it? Would you put a Nexus in the center of the row, with fabric extenders for ToR?

Are there any good resources you would recommend I read to get up to speed on this? I'm learning alot on my own, but I'm sure it would be more efficient if I had the guidance of experts.

_________________
Regards,

Steven King
San Diego Cisco User Group - http://www.sdcug.com
"The only time something is impossible is when you think it is." - Kevin Corbin, CCIE #11577

read up on SDN's

also check out Cisco data center and virtualization site.
http://www.cisco.com/en/US/netsol/ns340 ... index.html

and no, don't plan on putting 1000v's into your virtuals, and plan on connecting then to you 2248's in the TOR.
It won't work.

_________________
"If you're good at anticipating the human mind. It leaves nothing to chance."
-Jigsaw

About SDN... am I reading this correctly that the control plane is being moved to servers? If so, I doubt this will happen in the DoD space any time soon. That's a wild assumption on my part though.

_________________
Regards,

Steven King
San Diego Cisco User Group - http://www.sdcug.com
"The only time something is impossible is when you think it is." - Kevin Corbin, CCIE #11577

I'm not sure exactly what you are asking, and I'm not familiar with the Nexus line except for the 7010 and 1000v.

What I have seen done a lot is placing a 3750 or similar switch at the top of the server racks. The 3750 would have two fiber uplinks, one to each core switch. These would both be trunk ports. This prevents having to run a ton of copper to each rack.

Depending on server redundancy needs, you can feed servers from the switch at the top of it's rack and the switch of a neighboring rack, or if you have the money you can put in two switches at the top of the rack. Each server is connected to both switches for redundancy.

If up time is more critical than cost, the 3750X is modular and has ability to have dual power supplies. Some of the Nexus ones might be the same I just don't know the models.

Hopefully that helps some, if you've got other questions, fire away.

Netirons are more core ISP devices, not really DC switches. I believe Brocade do make whole lines just for DC.

Saying that, you can do some really awesome MPLs stuff with the Netirons

_________________
www.mellowd.co.uk/ccie/

Hehe, I'm not exactly sure what I'm asking either

But that's the jist - kind of what's being done, what's the logic behind it, where stuff is usually racked and how it's connected to the rest of the row, etc.

Here's a picture from a training I recently attended:

_________________
Regards,

Steven King
San Diego Cisco User Group - http://www.sdcug.com
"The only time something is impossible is when you think it is." - Kevin Corbin, CCIE #11577

If you have the money go with the Nexus line. And call your Cisco rep, they should be able to sit down and help you get it designed right.

They are always willing to meet with people that are willing to spend that kind of money on gear. Take advantage of it.

Hopefully you have one close by. I forget that I'm lucky in that regard being in the Houston area. They have big office here so it's never difficult to get a meeting with them.

Perfect world (for a Cisco design) is 7-5-2.

7Ks for core/distro
5K/2K for DC access

Where you physically put everything will depend. I've done some ToR, but with copper passing between racks as customers don't want to spend the money for 2xFEX ToR. I've also done a MoR with FEXes. It depends on a lot of factors.

_________________
http://blog.alwaysthenetwork.com

I wouldn't let a Cisco SE design my DC. Work with a partner who has experience with these things.

_________________
http://blog.alwaysthenetwork.com

Why?

Also, I don't mean let them design everything from scratch but from the sound of it, he needs help from someone that knows what they are doing. I don't care who it is I guess, but he needs to sit down with someone. A partner would be a good choice, but I've worked with some of them that didn't seem to have a clue at times as well.

Yeah coming from a previous company, I learned the difference between an Engineer and a Sales Engineer.

We have a CCIE team here onsite, so I have plenty of access, but like I said earlier, the architect doesn't want to let the vendor spoon-feed us - we should tell them what we want to accomplish. The problem is, I don't know what we want to accomplish. I assume consolidation, performance, ROI, security, agility, etc. etc., but they are all general terms to me.... I don't know specifics.

_________________
Regards,

Steven King
San Diego Cisco User Group - http://www.sdcug.com
"The only time something is impossible is when you think it is." - Kevin Corbin, CCIE #11577

Going to be hard for us to help you when you can't tell us what you want. Where did this initiative come from and what do they want to accomplish? What is the driving force of the project? What are your top 5 needs? Are you using blades or rack mount servers?

How large is the data center? 3 racks? 30 racks? Are you consolidating servers as part of this?

Because SEs don't do real work, just pre-sales. I'd trust them (to a degree) to build me a BoM, but I'd never let them do a detailed design.

Yes, there are partner engineers who suck, but that's why you pick a large partner with solid experience.

_________________
http://blog.alwaysthenetwork.com

I realized after Steven responded you meant sales engineer. Yeah, I wouldn't trust them either.

Unfortunately I don't think I can get into detail without running the risk of security compliance violations.

We're using 65xx switches for distro and access mostly.. and 35xx switches for some (few) parts of access. We're looking to update/replace them with Nexus and/or Brocade. LOTS of servers. Multiple farms.

EDIT - I guess I should have mentioned earlier that I am not on location so I have no visibility outside of documentation as to what is in place currently. I'm tasked with reviewing current documentation, seeing what Nexus and/or Brocade can do to improve, then certifying and documenting the updated solution.

EDIT #2 - I'm not asking for a specific solution so to speak... just what you've guys done in the past, the general methodologies/fundamentals, and/or resources I should read up on to close the knowledge gap.

_________________
Regards,

Steven King
San Diego Cisco User Group - http://www.sdcug.com
"The only time something is impossible is when you think it is." - Kevin Corbin, CCIE #11577

If I were you, I'd press real hard for a visit of the site. It helps me a lot to have a clear picture in my head of what I'm working with.

Other than that, check out the link that was posted earlier for Cisco DC.

so true about SE's, they live that pipe dream that they know it all and sell it all.

_________________
"I will prepare and some day my chance will come." - Abraham Lincoln
http://danielhertzberg.wordpress.com - I blog about networks!

No, I mean a Cisco Systems Engineer. What kind of engineer are you talking about with Cisco? TAC? CE?

_________________
http://blog.alwaysthenetwork.com

I saw SE and was thinking systems engineer, then with Stevens response realized you might have meant sales engineer. Normally when I meet with them there is a sales guy but also a couple others they bring along.

Check out Chris Marget's blog (http://www.fragmentationneeded.net/) on Nexus and DC design and it will give you a good start on some of the things you need to think about and their cost impact. For Nexus check out the 6509 vs Nexus post I had a while back. It has a good number of topics.
viewtopic.php?f=33&t=29738

Ivan Pepelnjak also has a number of webinars on DC design and fabircs that are pretty good.
http://www.ipspace.net/Webinars

I also hear Vito is getting quite a bit of experience with Nexus these days.

_________________
http://blog.movingonesandzeros.net/