Recent

Author Topic: Server hotswapping  (Read 6384 times)

0 Members and 1 Guest are viewing this topic.

Offline frailer

  • Sydney, OZ.
  • Global Moderator
  • Supreme Ninja
  • *****
  • Posts: 11800
  • Gender: Male
  • Older, and digitally challenged
Server hotswapping
« on: June 28, 2017, 12:20:34 AM »
Been a bit of a saga, but I just ordered a drive from Leizig to replace one of the three running our system drive on the HP Proliant, which runs XMF. I hadn't noticed one was red-lighted. :embarrassed:

After some argy-bargy, I went for this one, as with price+freight seems pretty good. Fujifilm guy is off to the other side of the country for 2 weeks next week. Our IT guy, who's in the centre of the city, and visits on a similar time frame to Halley's Comet, has not yet proffered a reply by email. However, and opinion from the Screen guy, (we'll be upgrading to Screen thermal), says hot-swapping may be better, as it should start striping over once it's in and detects the new drive (SCSI Controller stuff?). He said that this may obviate the need to get into BIOS to reset stuff. Am too old for that learning curve.

The Leipzig guys seem to be implying that its wiped/formatted ready to go.

My new compadre is back in Tuesday, and I doubt it'll be here before then, he'll be of actual and moral support.

I'm basing doing this on Mr Murphy being a constant presence, potentially. Will be weeks before the new server comes, I think.

.... oh, it's 3 disk RAID 5, but am verifying that.

Thoughts are most welcome.

The one I bought---¬

http://www.ebay.com.au/itm/272041842344
« Last Edit: June 28, 2017, 12:27:37 AM by frailer »
Fujifilm XMF/AdobeCC/PitStop Pro/Black Magic/ScreenPlateRite_Fujifilm ZP processless. 28" Komoris  |  Forgotten good guys: Dennis Ritchie, Burrell Smith, Bill Atkinson, Richard Stallman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Too old for this.. but.. there's that question of income. And lack of it if I stop.
~~~~~~

Offline frailer

  • Sydney, OZ.
  • Global Moderator
  • Supreme Ninja
  • *****
  • Posts: 11800
  • Gender: Male
  • Older, and digitally challenged
Re: Server hotswapping
« Reply #1 on: June 28, 2017, 01:53:44 AM »
Just spoke with our IT guy. Feeling much better about it now. His take:

-if it's hot swappable, it'll start to rebuild the drive straight away. 

-If it's not, it will be there as a new drive and build with new information over time.

I don't quite get this, bit... does that mean it achieves the same result over a longer time?
Fujifilm XMF/AdobeCC/PitStop Pro/Black Magic/ScreenPlateRite_Fujifilm ZP processless. 28" Komoris  |  Forgotten good guys: Dennis Ritchie, Burrell Smith, Bill Atkinson, Richard Stallman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Too old for this.. but.. there's that question of income. And lack of it if I stop.
~~~~~~

Offline Joe

  • Master of Nothing
  • Administrator
  • Ninja Warlord
  • *****
  • Posts: 37624
  • Gender: Male
    • B4Print.com
Re: Server hotswapping
« Reply #2 on: June 28, 2017, 08:43:37 AM »
Well that sounds like BS to me. If it is hot swappable and my guess is that it is it should rebuild once you replace it. If it is not hot swappable when you put it in it will basically become a drive that is unused.

Somewhere in your server their should be some kind of HP ProLiant Array Configuration Utility that might be helpful to look at. You can see which drive is bad and it should tell you if the drives are hot swappable and then it should show that it is rebuilding once you replace the drive. Also you really should make sure that drive you are buying is the same model and size as the other drives.
Mac OS 10.14.6 Mojave | Prinergy 8.2 | Adobe Creative Cloud 2019 | Two Luscher XPose 160 CTP units

Prepress: One who does precision guess work based on unreliable data provided by those of questionable knowledge.

Offline Farabomb

  • #NoLivesMatter
  • Supreme Ninja
  • ***
  • Posts: 17255
  • Gender: Male
  • Wait, what?
    • My other job
Re: Server hotswapping
« Reply #3 on: June 28, 2017, 08:54:07 AM »
I'm with Joe. The drive has to match or you're asking for trouble. Can you get away with a larger one? yes but you're throwing away the extra space and the RAID only will see it as the same size as the existing array.
Speed doesn't kill, rapidly becoming stationary is the problem

I'd rather have stories told than be telling stories of what I could have done.

Quote from: Ear date=1459965256 link=msg=278846
Quote from: Farabomb date=1459964381 link=msg=278844
It's more like grip, grip, grip, noise, then spin and 2 feet in and feel shame.
I once knew a plus-sized girl and this pretty much describes teh secks. :rotf:
They who can give up essential liberty to obtain a little temporary safety, deserve neither liberty nor safety.
         —Benjamin Franklin

My other job

Offline DigiCorn

  • Terrorizing humans since 1971
  • Supreme Ninja
  • ***
  • Posts: 15450
  • Gender: Male
  • There is no Dana, only Zul!
    • Clerks.
Re: Server hotswapping
« Reply #4 on: June 28, 2017, 11:31:04 AM »
I once had an issue with a Hot Swappable RAID-5. I made the boss do it. Immediately on pulling the drive, it killed the server. The RAID-5 never lived again. Luckily, everything was backed up, and we actually had pre-ordered a new server that was sitting and waiting. But it wasn't MY fault.
"Selling my soul would be a lot easier if I could just find it."
– Nikki Sixx

“Always do sober what you said you'd do drunk. That will teach you to keep your mouth shut.”
― Ernest Hemingway

Offline mattbeals

  • Just a dude
  • Hero Member
  • *****
  • Posts: 885
  • Gender: Male
    • Matt Beals
Re: Server hotswapping
« Reply #5 on: June 28, 2017, 01:22:05 PM »
If it's a hot swap drive system AND it's RAID5 (hardware or software doesn't matter) AND you swap the correct drive, the RAID array will automatically rebuild itself.

IF you pull the wrong drive, well I hope you have a good and recent backup.
IF you replace the dead drive with a larger drive you will loose the drive capacity beyond what the other drives have.
IF you have a free drive slot AND a compatible RAID controller you should add a 4th drive as a "hot spare" so that you reduce the likelihood of running in "degraded" mode like you are now.

I've never had a RAID 5 die on me except for the time when someone pulled the wrong drive. That was bad. Very bad... Always have a RAID configuration that includes a hot spare. Not a cold spare, but a hot spare. In the even a drive fails you'll be glad the hot spare was there. And always be sure that the replacement drive is the same RPM as the old drive. I've seen guys replace 15k RPM disks with 10k RPM disks. The array slows down, not what you want.
Matt Beals

Everything I say is my own personal opinion and has nothing to do with my employer or their views.

Offline Joe

  • Master of Nothing
  • Administrator
  • Ninja Warlord
  • *****
  • Posts: 37624
  • Gender: Male
    • B4Print.com
Re: Server hotswapping
« Reply #6 on: June 28, 2017, 01:44:16 PM »
Quote from: mattbeals date=1498670525 link=msg=292280

If it's a hot swap drive system AND it's RAID5 (hardware or software doesn't matter) AND you swap the correct drive, the RAID array will automatically rebuild itself.

IF you pull the wrong drive, well I hope you have a good and recent backup.


Very good point. That is why you need the raid utility. You can tell it to make the light blink on the bad drive so you are sure you are pulling the right one. As Matt said....if you pull the wrong drive your day is going to get a lot worse.
Mac OS 10.14.6 Mojave | Prinergy 8.2 | Adobe Creative Cloud 2019 | Two Luscher XPose 160 CTP units

Prepress: One who does precision guess work based on unreliable data provided by those of questionable knowledge.

Offline frailer

  • Sydney, OZ.
  • Global Moderator
  • Supreme Ninja
  • *****
  • Posts: 11800
  • Gender: Male
  • Older, and digitally challenged
Re: Server hotswapping
« Reply #7 on: June 28, 2017, 08:03:38 PM »
I'll look for that drive utility, but it's lit up, anyway.

I've physically flagged it, and I'll get compadre to check. I'm not doing this alone.
haven't dismissed the idea of getting in-house Co IT guy to come up and do it, (we're out on the fringes of suburbia; our parent company in the centre). If stuff goes wrong, he can at least do more than I ever could.

Pic is conveniently outa focus so you can't see the dust.

The bottom 3 are the 72.8 sys RAID. Top 3 are the 146 x 3 genstore/job data RAID. Only about6~8 weeks to go, but Mr Murphy would not give a flying fuck about that. In fact he may even quite enjoy watching stuff fail.

Oh, that's right... it's what he likes to do, isn't it.

The drive is a 'perfect match', AFAICPT, to the ones in the array.

Oh, and thanks for the input...
« Last Edit: June 28, 2017, 08:28:46 PM by frailer »
Fujifilm XMF/AdobeCC/PitStop Pro/Black Magic/ScreenPlateRite_Fujifilm ZP processless. 28" Komoris  |  Forgotten good guys: Dennis Ritchie, Burrell Smith, Bill Atkinson, Richard Stallman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Too old for this.. but.. there's that question of income. And lack of it if I stop.
~~~~~~

Offline Joe

  • Master of Nothing
  • Administrator
  • Ninja Warlord
  • *****
  • Posts: 37624
  • Gender: Male
    • B4Print.com
Re: Server hotswapping
« Reply #8 on: June 28, 2017, 10:38:02 PM »
You most likely have two separate RAID array's there. The top 3 is one array and the bottom three are the second array. Drives in an array are numbered, in your case 0, 1, 2 for each array. I would still find that utility and see which drive it says is bad and then tell it to blink just so you know for sure the one that is lit up is the same one that blinks and is the bad one.

If you can't find the utility you can usually reboot the server and then watch the screen. There should be a RAID setup and telling you what keys to push to enter it when it is going through the boot process and you can do the same thing from it as you can from the RAID utility within Windows.
Mac OS 10.14.6 Mojave | Prinergy 8.2 | Adobe Creative Cloud 2019 | Two Luscher XPose 160 CTP units

Prepress: One who does precision guess work based on unreliable data provided by those of questionable knowledge.

Offline mattbeals

  • Just a dude
  • Hero Member
  • *****
  • Posts: 885
  • Gender: Male
    • Matt Beals
Re: Server hotswapping
« Reply #9 on: June 28, 2017, 10:48:30 PM »
Quote from: frailer date=1498694618 link=msg=292285

I'll look for that drive utility, but it's lit up, anyway.

I've physically flagged it, and I'll get compadre to check. I'm not doing this alone.
haven't dismissed the idea of getting in-house Co IT guy to come up and do it, (we're out on the fringes of suburbia; our parent company in the centre). If stuff goes wrong, he can at least do more than I ever could.

Pic is conveniently outa focus so you can't see the dust.

The bottom 3 are the 72.8 sys RAID. Top 3 are the 146 x 3 genstore/job data RAID. Only about6~8 weeks to go, but Mr Murphy would not give a flying fuck about that. In fact he may even quite enjoy watching stuff fail.

Oh, that's right... it's what he likes to do, isn't it.

The drive is a 'perfect match', AFAICPT, to the ones in the array.

Oh, and thanks for the input...


Dude... Seriously??? I can see the dust bunnies coming out between the drives and the blanks even though it's out of focus. You need to get that thing cleaned ASAP before you have more problems. The heat will kill the components, fans or power supply first. You really need a PM cycle on that. Since you don't have room in the drive cage for any spares I'd suggest you order an extra 72GB and 146GB (or two) drives just in case.
Matt Beals

Everything I say is my own personal opinion and has nothing to do with my employer or their views.

Offline mattbeals

  • Just a dude
  • Hero Member
  • *****
  • Posts: 885
  • Gender: Male
    • Matt Beals
Re: Server hotswapping
« Reply #10 on: June 28, 2017, 10:59:19 PM »
Quote from: Joe link=msg=292287 date=1498703882

You most likely have two separate RAID array's there. The top 3 is one array and the bottom three are the second array. Drives in an array are numbered, in your case 0, 1, 2 for each array. I would still find that utility and see which drive it says is bad and then tell it to blink just so you know for sure the one that is lit up is the same one that blinks and is the bad one.

If you can't find the utility you can usually reboot the server and then watch the screen. There should be a RAID setup and telling you what keys to push to enter it when it is going through the boot process and you can do the same thing from it as you can from the RAID utility within Windows.


If you have two different sets of drives then you have two different arrays. You probably have this RAID controller: Smart Array P400i ( http://h18000.www1.hp.com/products/quickspecs/archives_division/12400_div_v3/12400_div.html ) There's a web management tool for the RAID controller, this might be compatible with yours: http://h20564.www2.hpe.com/hpsc/swd/public/detail?swItemId=MTX_4effd70562304a50b3be5c4b96

On the HP Proliants that I had the RAID controllers would detect the new drive and start rebuilding the array automatically. No manual intervention needed.
Matt Beals

Everything I say is my own personal opinion and has nothing to do with my employer or their views.

Offline Joe

  • Master of Nothing
  • Administrator
  • Ninja Warlord
  • *****
  • Posts: 37624
  • Gender: Male
    • B4Print.com
Re: Server hotswapping
« Reply #11 on: June 29, 2017, 12:42:57 AM »
Same thing on my Dell servers. Swap the drive out and it automatically starts rebuilding.
Mac OS 10.14.6 Mojave | Prinergy 8.2 | Adobe Creative Cloud 2019 | Two Luscher XPose 160 CTP units

Prepress: One who does precision guess work based on unreliable data provided by those of questionable knowledge.

Offline frailer

  • Sydney, OZ.
  • Global Moderator
  • Supreme Ninja
  • *****
  • Posts: 11800
  • Gender: Male
  • Older, and digitally challenged
Re: Server hotswapping
« Reply #12 on: July 03, 2017, 12:25:54 AM »
Deutschepost's idea of tracking is.... you go to the site, paste in the Tracking Number, and each time it says... "You item has been posted".

:rotf:
Fujifilm XMF/AdobeCC/PitStop Pro/Black Magic/ScreenPlateRite_Fujifilm ZP processless. 28" Komoris  |  Forgotten good guys: Dennis Ritchie, Burrell Smith, Bill Atkinson, Richard Stallman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Too old for this.. but.. there's that question of income. And lack of it if I stop.
~~~~~~

Offline frailer

  • Sydney, OZ.
  • Global Moderator
  • Supreme Ninja
  • *****
  • Posts: 11800
  • Gender: Male
  • Older, and digitally challenged
Re: Server hotswapping
« Reply #13 on: July 05, 2017, 10:49:23 PM »
The moment of truth has arrived from Leipzig. Zose Chermans are pretty efficient.

Mañana though. I need to approach with caution, and some 'help'. A bit too pre-occupied today.
Fujifilm XMF/AdobeCC/PitStop Pro/Black Magic/ScreenPlateRite_Fujifilm ZP processless. 28" Komoris  |  Forgotten good guys: Dennis Ritchie, Burrell Smith, Bill Atkinson, Richard Stallman
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Too old for this.. but.. there's that question of income. And lack of it if I stop.
~~~~~~

Offline Joe

  • Master of Nothing
  • Administrator
  • Ninja Warlord
  • *****
  • Posts: 37624
  • Gender: Male
    • B4Print.com
Re: Server hotswapping
« Reply #14 on: July 06, 2017, 08:56:19 AM »
SCSI drives...reminds me of my youth. :cane:
Mac OS 10.14.6 Mojave | Prinergy 8.2 | Adobe Creative Cloud 2019 | Two Luscher XPose 160 CTP units

Prepress: One who does precision guess work based on unreliable data provided by those of questionable knowledge.