networking-forum.com
Community BlogCommunity Wiki * Register  * Search  * Login
View unanswered postsView active topics

All times are UTC - 6 hours [ DST ]



Post new topic Reply to topic  [ 40 posts ]  Go to page Previous  1, 2
Author Message
PostPosted: Wed May 09, 2012 3:01 pm 
Offline
New Member
New Member

Joined: Wed May 02, 2012 9:37 pm
Posts: 20
Location: England
Certs: CCNA
As it was mentioned before
ristau5741 wrote:
all the code has "issues" vulnerabilities, things that are just plain broken. there is no non-vulnerable and stable, IOS code as it is a work in progress


It might be a good idea to check in a testing environment the code you trying to deploy. I am assuming that it's not an option in this case. However, would you be able to do it one by one at least, in order to minimise the impact?

Check the bug toolkit, so you know what to expect and where it might break

http://tools.cisco.com/Support/BugToolKit/


Top
 Profile  
 
PostPosted: Wed May 09, 2012 3:13 pm 
Offline
Ultimate Member
Ultimate Member

Joined: Wed Aug 03, 2011 12:24 pm
Posts: 504
Location: Charleston, SC
Certs: MCSE, MCP+I, SEC+ (working on CCENT/CCNA)
Normally that's how I'd do it- but this has been sprung on me without much notice, so I can't...


Top
 Profile  
 
PostPosted: Wed May 09, 2012 3:21 pm 
Offline
Senior Member
Senior Member
User avatar

Joined: Fri Sep 02, 2011 6:59 am
Posts: 362
Location: Birmingham, UK
Certs: CCNA, CCNP
Upgrade is good and all but take the time and do it right and in steps, I know boss ask and boss gets, but try to tell them that shit can go the wrong way because you had no time to plan this right. Not sure how many devices you have, still 2 days for all out OS upgrade is a push IMO even if you have your network map printed at the back of your skull.

Best of luck!


Top
 Profile  
 
PostPosted: Wed May 09, 2012 5:02 pm 
Offline
Ultimate Member
Ultimate Member
User avatar

Joined: Mon Oct 06, 2008 8:01 am
Posts: 669
Certs: CCNA,CCNP
DieselJeeper wrote:
Dan-

Rgoer that. The web-gui IST VERBOTEN in this environment, among other things.. and crypto is *required*.

Did I mention, I am pretty much learning as I go? My boss is famous for saying "this is a teaching hospital-- just don't kill the patient"

.. killing the patient, accidentally, PETRIFIES me. I've spent a few sleepless nights, I'll be honest.

I am having trouble getting the "dir flash" and "show flash" commands to work on a 4500 running 12.2. Off I go to research yet more....

and again, gents- *thank you*


Hmm, I work in healthcare too.
Just be careful, try and take your time making sure you have a rollback plan if things go wrong. The 4500 is a different beast to the 3560 / 3750's as they are chassis based and have supervisors. The upgrade is similar but you need to take into account the storage location i.e. flash card, supbootflash and whether or not there are dual supervisors.

Like mlan says, 'dir bootflash:' should work, also 'dir slavebootflash:' will show you the standby sup flash if you have one.. or 'dir ?' for anything else i.e. flash disks...


Top
 Profile  
 
PostPosted: Thu May 10, 2012 7:54 am 
Offline
Ultimate Member
Ultimate Member

Joined: Wed Aug 03, 2011 12:24 pm
Posts: 504
Location: Charleston, SC
Certs: MCSE, MCP+I, SEC+ (working on CCENT/CCNA)
DanC: I don't work in healthcare. I used to work on a global mesh VPN project for literally a full.global.mesh. Yeah, bet you can figure out who paid for that... :)

I now work in a transportation facility. In some ways much more laid back, in other ways much less. I like it here :)

The switches I am pretty confident I can handle- after all, we don't have to "jump trains" to the newer IOS. IE: we stay on the same basic version, just going for the latter refinements.

This business about sup engines and so on, I am reticent about.


Top
 Profile  
 
PostPosted: Thu May 10, 2012 8:22 am 
Offline
Post Whore
Post Whore
User avatar

Joined: Tue Aug 21, 2007 2:15 pm
Posts: 8298
Location: Frederick MD
Certs: Instanity
Agent Zed wrote:
still 2 days for all out OS upgrade is a push IMO even if you have your network map printed at the back of your skull.

Best of luck!


we used to do 800+ devices in a 12 hour maintenance window.
not just switches and routers, voice, wireless, video gear + linux, too..

_________________
"If you're good at anticipating the human mind. It leaves nothing to chance."
-Jigsaw


Top
 Profile  
 
PostPosted: Thu May 10, 2012 9:08 am 
Offline
Senior Member
Senior Member
User avatar

Joined: Fri Sep 02, 2011 6:59 am
Posts: 362
Location: Birmingham, UK
Certs: CCNA, CCNP
ristau5741 wrote:
Agent Zed wrote:
still 2 days for all out OS upgrade is a push IMO even if you have your network map printed at the back of your skull.

Best of luck!


we used to do 800+ devices in a 12 hour maintenance window.
not just switches and routers, voice, wireless, video gear + linux, too..
Sure thing for the 12h change window, but from "Here's the project" to "done" in 2 days is a bit crazy I think. Perhaps I'm just not super quick thinking.


Top
 Profile  
 
PostPosted: Thu May 10, 2012 10:18 am 
Offline
Ultimate Member
Ultimate Member

Joined: Wed Aug 03, 2011 12:24 pm
Posts: 504
Location: Charleston, SC
Certs: MCSE, MCP+I, SEC+ (working on CCENT/CCNA)
Agent Zed wrote:
ristau5741 wrote:
Agent Zed wrote:
still 2 days for all out OS upgrade is a push IMO even if you have your network map printed at the back of your skull.

Best of luck!


we used to do 800+ devices in a 12 hour maintenance window.
not just switches and routers, voice, wireless, video gear + linux, too..
Sure thing for the 12h change window, but from "Here's the project" to "done" in 2 days is a bit crazy I think. Perhaps I'm just not super quick thinking.


Agent Zed:

I think the point is- he's talking "we" useed to do this. How many people is "we"? Plus, knowing ristau, he worked with highly-capable folks, since he is one. :bowdown:

In my case, it's me. Noone else. And to be honest with you, I am not highly capable at this stuff... yet. :cheers:

I mean, FFS, I had to be told how to find out how much memory's on the darn things!


Top
 Profile  
 
PostPosted: Thu May 10, 2012 11:34 am 
Offline
Post Whore
Post Whore
User avatar

Joined: Tue Aug 21, 2007 2:15 pm
Posts: 8298
Location: Frederick MD
Certs: Instanity
DieselJeeper wrote:
Agent Zed wrote:
ristau5741 wrote:
we used to do 800+ devices in a 12 hour maintenance window.
not just switches and routers, voice, wireless, video gear + linux, too..
Sure thing for the 12h change window, but from "Here's the project" to "done" in 2 days is a bit crazy I think. Perhaps I'm just not super quick thinking.


Agent Zed:

I think the point is- he's talking "we" useed to do this. How many people is "we"? Plus, knowing ristau, he worked with highly-capable folks, since he is one. :bowdown:

In my case, it's me. Noone else. And to be honest with you, I am not highly capable at this stuff... yet. :cheers:

I mean, FFS, I had to be told how to find out how much memory's on the darn things!



there was also _alot_ of scripting involved,

each device shouldn't take more then 15-20 minutes to load the image and reboot.
staging helps early on, specifically routers, switches no so, if they are routing, the archive
command steals too many cpu cycles and the neighbor adjacencies drop (as least in my past experience)

you can also use the 'reload in' or 'reload at' commands to schedule reloads.

make sure you schedule properly, for example I was starting the archive upgrade
on the WAN switch , then started a archive upgrade on remote switch, when the
wan switch reloaded, the remote switch lost it's file transfer and I had to restart it.

you will also need to verify your versions after upgrade, check logs, and verify everything
is working properly, to make sure all is good before going to bed for the morning.

_________________
"If you're good at anticipating the human mind. It leaves nothing to chance."
-Jigsaw


Top
 Profile  
 
PostPosted: Thu May 10, 2012 11:38 am 
Offline
Ultimate Member
Ultimate Member

Joined: Wed Aug 03, 2011 12:24 pm
Posts: 504
Location: Charleston, SC
Certs: MCSE, MCP+I, SEC+ (working on CCENT/CCNA)
Everything I am responsible for is within 15 minutes of my location, so I don't think I am going to bother with the archive command or scripting- I've not dealt with that yet.


Top
 Profile  
 
PostPosted: Thu May 10, 2012 11:42 am 
Offline
Post Whore
Post Whore
User avatar

Joined: Tue Aug 21, 2007 2:15 pm
Posts: 8298
Location: Frederick MD
Certs: Instanity
the archive command is easier than manual upgrade for switches, it will do everything you need, include reload if desired.
but you may run into problems if the .bin files are not in the expected locations for the archive to run, this will cause out of memory errors. so if you've done things manually in the past, stick with that.

_________________
"If you're good at anticipating the human mind. It leaves nothing to chance."
-Jigsaw


Top
 Profile  
 
PostPosted: Thu May 10, 2012 12:13 pm 
Offline
Post Whore
Post Whore
User avatar

Joined: Fri Apr 25, 2008 2:16 pm
Posts: 1363
Location: Jacksonville, FL
Certs: CCNP, CCNA:Sec, MCSE
Keep in mind that if you have stacked switches and manually upgrade, you have to manually copy the file to each switch's Flash, and do the 'boot system' command a little different.

_________________
"A problem well stated is a problem half solved". (Charles Kettering)


Top
 Profile  
 
PostPosted: Thu May 10, 2012 12:33 pm 
Offline
Ultimate Member
Ultimate Member

Joined: Wed Aug 03, 2011 12:24 pm
Posts: 504
Location: Charleston, SC
Certs: MCSE, MCP+I, SEC+ (working on CCENT/CCNA)
by "stacked switches" I assume you mean, switches with cascade cables between them, aka (I think) clusters?

No, none of that here.


Top
 Profile  
 
PostPosted: Thu May 10, 2012 6:21 pm 
Offline
Ultimate Member
Ultimate Member

Joined: Wed Aug 03, 2011 12:24 pm
Posts: 504
Location: Charleston, SC
Certs: MCSE, MCP+I, SEC+ (working on CCENT/CCNA)
Grr, this is NOT the time for problems...

I've done 1 switch and tried 3 more. First went fine, like this:

hop into the switch via console. Ethernet patched between switch and laptop, too.

dir flash: (check and make sure there's enough room for the .bin. The .bin I am using is 16meg (15,965,969 bytes). I have about 17,500,000 free on the flash)
copy tftp flash:
(copy process starts- I am using a proven TFTP server on the laptop, made sure it was started, etc)
a couple of copy errors show in the TFTP server log... but it claims to complete successfully)
I compared the filesize manually- it's the same as it should be- 15,965,969 bytes.
then do a "verify flash:(filename)". This fails. "File system hash verification failed for (filename). No such file or directory"...

but I can SEE the damn thing, RTF there... and am understandably reticent to continue... Thoughts?


Top
 Profile  
 
PostPosted: Thu May 10, 2012 7:46 pm 
Offline
Ultimate Member
Ultimate Member

Joined: Wed Aug 03, 2011 12:24 pm
Posts: 504
Location: Charleston, SC
Certs: MCSE, MCP+I, SEC+ (working on CCENT/CCNA)
I have done some research and honestly believe this to be a bug.

Feeling courageous (and still being at work some 13 hours after my day started, and everyone else went home 4 hours ago), I have the "least damaging if things go TU") switch proceeding with booting off of the "corrupt" load.

... and while that's spinning, I'm gonna go research restoring a cisco switch from a bad IOS upgrade... :yikers:


Top
 Profile  
 
PostPosted: Thu May 10, 2012 11:17 pm 
Offline
Ultimate Member
Ultimate Member

Joined: Wed Sep 01, 2010 3:37 pm
Posts: 907
Location: Las Vegas, NV
Certs: Sec+, MCSE, MCITP:EA, CCNP
DieselJeeper wrote:
I have done some research and honestly believe this to be a bug.

Feeling courageous (and still being at work some 13 hours after my day started, and everyone else went home 4 hours ago), I have the "least damaging if things go TU") switch proceeding with booting off of the "corrupt" load.

... and while that's spinning, I'm gonna go research restoring a cisco switch from a bad IOS upgrade... :yikers:


Do you still have the original IOS on there? If so the recovery isn't too bad. Just boot to ROMMON and tell it to boot with the other image. If the only IOS on the flash is bad it can be a pain. You have to xmodem a new one up, and it can take a few hours.

-Otanx


Top
 Profile  
 
PostPosted: Fri May 11, 2012 8:40 am 
Offline
Post Whore
Post Whore
User avatar

Joined: Tue Aug 21, 2007 2:15 pm
Posts: 8298
Location: Frederick MD
Certs: Instanity
HUMM... waiting to hear how this saga turned out....

I hope this wasn't the season finale and
I don't have to wait until next season....

_________________
"If you're good at anticipating the human mind. It leaves nothing to chance."
-Jigsaw


Top
 Profile  
 
PostPosted: Fri May 11, 2012 10:45 am 
Offline
Ultimate Member
Ultimate Member

Joined: Wed Aug 03, 2011 12:24 pm
Posts: 504
Location: Charleston, SC
Certs: MCSE, MCP+I, SEC+ (working on CCENT/CCNA)
I was right.

Followed process right past the "warning! Warning Will Robinson" crap that the verify command brought up, made the machine boot off the new IOS. All golden. I did 3 more switches afterwards. All is good.

... 9 switches to go, then I have some routers. Did I mention I've never done an upgrade on a router?


Top
 Profile  
 
PostPosted: Fri May 11, 2012 1:39 pm 
Offline
Post Whore
Post Whore
User avatar

Joined: Tue Aug 21, 2007 2:15 pm
Posts: 8298
Location: Frederick MD
Certs: Instanity
easy peazy, just gotta hope everything comes back up afterwards.


copy the ios,
verify the image
clear the old boot var
set the new boot var
reload

_________________
"If you're good at anticipating the human mind. It leaves nothing to chance."
-Jigsaw


Top
 Profile  
 
PostPosted: Fri May 11, 2012 1:49 pm 
Offline
Post Whore
Post Whore
User avatar

Joined: Fri Apr 25, 2008 2:16 pm
Posts: 1363
Location: Jacksonville, FL
Certs: CCNP, CCNA:Sec, MCSE
and don't forget to 'wr mem' before you reload :-)

_________________
"A problem well stated is a problem half solved". (Charles Kettering)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 40 posts ]  Go to page Previous  1, 2

All times are UTC - 6 hours [ DST ]


Who is online

Users browsing this forum: carfret, Google Feedfetcher and 20 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group