News:

Main Menu

Server hotswapping

Started by frailer, June 27, 2017, 10:20:34 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

frailer

Been a bit of a saga, but I just ordered a drive from Leizig to replace one of the three running our system drive on the HP Proliant, which runs XMF. I hadn't noticed one was red-lighted. :embarrassed:

After some argy-bargy, I went for this one, as with price+freight seems pretty good. Fujifilm guy is off to the other side of the country for 2 weeks next week. Our IT guy, who's in the centre of the city, and visits on a similar time frame to Halley's Comet, has not yet proffered a reply by email. However, and opinion from the Screen guy, (we'll be upgrading to Screen thermal), says hot-swapping may be better, as it should start striping over once it's in and detects the new drive (SCSI Controller stuff?). He said that this may obviate the need to get into BIOS to reset stuff. Am too old for that learning curve.

The Leipzig guys seem to be implying that its wiped/formatted ready to go.

My new compadre is back in Tuesday, and I doubt it'll be here before then, he'll be of actual and moral support.

I'm basing doing this on Mr Murphy being a constant presence, potentially. Will be weeks before the new server comes, I think.

.... oh, it's 3 disk RAID 5, but am verifying that.

Thoughts are most welcome.

The one I bought---¬

http://www.ebay.com.au/itm/272041842344
Forgotten good guys: Dennis Ritchie, Burrell Smith, Bill Atkinson, Richard Stallman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now just an honorary member.

frailer

Just spoke with our IT guy. Feeling much better about it now. His take:

-if it's hot swappable, it'll start to rebuild the drive straight away. 

-If it's not, it will be there as a new drive and build with new information over time.

I don't quite get this, bit... does that mean it achieves the same result over a longer time?
Forgotten good guys: Dennis Ritchie, Burrell Smith, Bill Atkinson, Richard Stallman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now just an honorary member.

Joe

Well that sounds like BS to me. If it is hot swappable and my guess is that it is it should rebuild once you replace it. If it is not hot swappable when you put it in it will basically become a drive that is unused.

Somewhere in your server their should be some kind of HP ProLiant Array Configuration Utility that might be helpful to look at. You can see which drive is bad and it should tell you if the drives are hot swappable and then it should show that it is rebuilding once you replace the drive. Also you really should make sure that drive you are buying is the same model and size as the other drives.
Mac OS Sonoma 14.2.1 (c) | (retired)

The seven ages of man: spills, drills, thrills, bills, ills, pills and wills.

Farabomb

I'm with Joe. The drive has to match or you're asking for trouble. Can you get away with a larger one? yes but you're throwing away the extra space and the RAID only will see it as the same size as the existing array.
Speed doesn't kill, rapidly becoming stationary is the problem

I'd rather have stories told than be telling stories of what I could have done.

Quote from: Ear on April 06, 2016, 11:54:16 AM
Quote from: Farabomb on April 06, 2016, 11:39:41 AMIt's more like grip, grip, grip, noise, then spin and 2 feet in and feel shame.
I once knew a plus-sized girl and this pretty much describes teh secks. :rotf:
They who can give up essential liberty to obtain a little temporary safety, deserve neither liberty nor safety.
         —Benjamin Franklin

My other job

DigiCorn

I once had an issue with a Hot Swappable RAID-5. I made the boss do it. Immediately on pulling the drive, it killed the server. The RAID-5 never lived again. Luckily, everything was backed up, and we actually had pre-ordered a new server that was sitting and waiting. But it wasn't MY fault.
"There's been a lot of research recently on how hard it is to dislodge an impression once it's been implanted in someone's mind. (This is why political attack ads don't have to be true to be effective. The other side can point out their inaccuracies, but the voter's mind privileges the memory of the original accusation, which was juicier than any counterargument ever could be.)"
― Johnny Carson

"Selling my soul would be a lot easier if I could just find it."
– Nikki Sixx

"Always do sober what you said you'd do drunk. That will teach you to keep your mouth shut."
― Ernest Hemingway

mattbeals

If it's a hot swap drive system AND it's RAID5 (hardware or software doesn't matter) AND you swap the correct drive, the RAID array will automatically rebuild itself.

IF you pull the wrong drive, well I hope you have a good and recent backup.
IF you replace the dead drive with a larger drive you will loose the drive capacity beyond what the other drives have.
IF you have a free drive slot AND a compatible RAID controller you should add a 4th drive as a "hot spare" so that you reduce the likelihood of running in "degraded" mode like you are now.

I've never had a RAID 5 die on me except for the time when someone pulled the wrong drive. That was bad. Very bad... Always have a RAID configuration that includes a hot spare. Not a cold spare, but a hot spare. In the even a drive fails you'll be glad the hot spare was there. And always be sure that the replacement drive is the same RPM as the old drive. I've seen guys replace 15k RPM disks with 10k RPM disks. The array slows down, not what you want.
Matt Beals

Everything I say is my own personal opinion and has nothing to do with my employer or their views.

Joe

Quote from: mattbeals on June 28, 2017, 11:22:05 AMIf it's a hot swap drive system AND it's RAID5 (hardware or software doesn't matter) AND you swap the correct drive, the RAID array will automatically rebuild itself.

IF you pull the wrong drive, well I hope you have a good and recent backup.

Very good point. That is why you need the raid utility. You can tell it to make the light blink on the bad drive so you are sure you are pulling the right one. As Matt said....if you pull the wrong drive your day is going to get a lot worse.
Mac OS Sonoma 14.2.1 (c) | (retired)

The seven ages of man: spills, drills, thrills, bills, ills, pills and wills.

frailer

#7
I'll look for that drive utility, but it's lit up, anyway.

I've physically flagged it, and I'll get compadre to check. I'm not doing this alone.
haven't dismissed the idea of getting in-house Co IT guy to come up and do it, (we're out on the fringes of suburbia; our parent company in the centre). If stuff goes wrong, he can at least do more than I ever could.

Pic is conveniently outa focus so you can't see the dust.

The bottom 3 are the 72.8 sys RAID. Top 3 are the 146 x 3 genstore/job data RAID. Only about6~8 weeks to go, but Mr Murphy would not give a flying fuck about that. In fact he may even quite enjoy watching stuff fail.

Oh, that's right... it's what he likes to do, isn't it.

The drive is a 'perfect match', AFAICPT, to the ones in the array.

Oh, and thanks for the input...
Forgotten good guys: Dennis Ritchie, Burrell Smith, Bill Atkinson, Richard Stallman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now just an honorary member.

Joe

You most likely have two separate RAID array's there. The top 3 is one array and the bottom three are the second array. Drives in an array are numbered, in your case 0, 1, 2 for each array. I would still find that utility and see which drive it says is bad and then tell it to blink just so you know for sure the one that is lit up is the same one that blinks and is the bad one.

If you can't find the utility you can usually reboot the server and then watch the screen. There should be a RAID setup and telling you what keys to push to enter it when it is going through the boot process and you can do the same thing from it as you can from the RAID utility within Windows.
Mac OS Sonoma 14.2.1 (c) | (retired)

The seven ages of man: spills, drills, thrills, bills, ills, pills and wills.

mattbeals

Quote from: frailer on June 28, 2017, 06:03:38 PMI'll look for that drive utility, but it's lit up, anyway.

I've physically flagged it, and I'll get compadre to check. I'm not doing this alone.
haven't dismissed the idea of getting in-house Co IT guy to come up and do it, (we're out on the fringes of suburbia; our parent company in the centre). If stuff goes wrong, he can at least do more than I ever could.

Pic is conveniently outa focus so you can't see the dust.

The bottom 3 are the 72.8 sys RAID. Top 3 are the 146 x 3 genstore/job data RAID. Only about6~8 weeks to go, but Mr Murphy would not give a flying fuck about that. In fact he may even quite enjoy watching stuff fail.

Oh, that's right... it's what he likes to do, isn't it.

The drive is a 'perfect match', AFAICPT, to the ones in the array.

Oh, and thanks for the input...

Dude... Seriously??? I can see the dust bunnies coming out between the drives and the blanks even though it's out of focus. You need to get that thing cleaned ASAP before you have more problems. The heat will kill the components, fans or power supply first. You really need a PM cycle on that. Since you don't have room in the drive cage for any spares I'd suggest you order an extra 72GB and 146GB (or two) drives just in case.
Matt Beals

Everything I say is my own personal opinion and has nothing to do with my employer or their views.

mattbeals

Quote from: Joe on June 28, 2017, 08:38:02 PMYou most likely have two separate RAID array's there. The top 3 is one array and the bottom three are the second array. Drives in an array are numbered, in your case 0, 1, 2 for each array. I would still find that utility and see which drive it says is bad and then tell it to blink just so you know for sure the one that is lit up is the same one that blinks and is the bad one.

If you can't find the utility you can usually reboot the server and then watch the screen. There should be a RAID setup and telling you what keys to push to enter it when it is going through the boot process and you can do the same thing from it as you can from the RAID utility within Windows.

If you have two different sets of drives then you have two different arrays. You probably have this RAID controller: Smart Array P400i ( http://h18000.www1.hp.com/products/quickspecs/archives_division/12400_div_v3/12400_div.html ) There's a web management tool for the RAID controller, this might be compatible with yours: http://h20564.www2.hpe.com/hpsc/swd/public/detail?swItemId=MTX_4effd70562304a50b3be5c4b96

On the HP Proliants that I had the RAID controllers would detect the new drive and start rebuilding the array automatically. No manual intervention needed.
Matt Beals

Everything I say is my own personal opinion and has nothing to do with my employer or their views.

Joe

Same thing on my Dell servers. Swap the drive out and it automatically starts rebuilding.
Mac OS Sonoma 14.2.1 (c) | (retired)

The seven ages of man: spills, drills, thrills, bills, ills, pills and wills.

frailer

Deutschepost's idea of tracking is.... you go to the site, paste in the Tracking Number, and each time it says... "You item has been posted".

:rotf:
Forgotten good guys: Dennis Ritchie, Burrell Smith, Bill Atkinson, Richard Stallman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now just an honorary member.

frailer

The moment of truth has arrived from Leipzig. Zose Chermans are pretty efficient.

Mañana though. I need to approach with caution, and some 'help'. A bit too pre-occupied today.
Forgotten good guys: Dennis Ritchie, Burrell Smith, Bill Atkinson, Richard Stallman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now just an honorary member.

Joe

SCSI drives...reminds me of my youth. :cane:
Mac OS Sonoma 14.2.1 (c) | (retired)

The seven ages of man: spills, drills, thrills, bills, ills, pills and wills.