flat assembler
Message board for the users of flat assembler.

Index > Heap > Backup HDD in an external rack issue

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
Ok so I ordered a few days ago an external 500GB 3.5" HDD for backup purposes, with a Sunbeam Airbox rack, and the drive went pretty cool in usage. The only issue I've seen was that, when defragmenting (after the copying, because of NTFS compression they were fragmented for some reason) the HDD simply stopped its operation halfway through -- the program was still waiting for it to respond or something (it got 'stuck') but the HDD simply stopped data transfer or whatever. (note that it was fully working, all I had to do was to stop the defragmentation and start it again).

My question is, should I be worried? Is this a bad symptom or something? (I've seen it pretty popular for 'external hard-drives' but IMO those are not reliable, not to the extent that a normal HDD+rack is anyway).

(also btw ManOfSteel, see what I mean with convenience, I can run all programs and everything else directly from this drive as if it were my main drive with absolutely zero additional system configuration effort Razz).

_________________
Previously known as The_Grey_Beast
Post 17 Jan 2010, 21:43
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17279
Location: In your JS exploiting you and your system
revolution
Check your defrag proggy. Why blame the HDD when the defragger could have some bug with larger drives?
Post 19 Jan 2010, 12:13
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
Well you're right that it could very well be a software problem, not just the defrag but also drivers. (seeing as though I used eSATA for transfer)

It's just that I never had it happening before on "internal" drives (even bigger, 750GB for instance), but again it could be because of drivers or maybe even the rack circuitry itself having a problem with too many requests. (I don't have a problem with that, as long as the HDD inside it is fine Wink).

_________________
Previously known as The_Grey_Beast
Post 19 Jan 2010, 17:45
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17279
Location: In your JS exploiting you and your system
revolution
BTW: Why are you defragging a backup drive? Question Backups don't need defragging, they need to to reliable, not 1% faster for access.
Post 19 Jan 2010, 17:47
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
Cause I like stuff to be ordered Razz

...and I only do it once anyway -- and only for compressed files (=text and 'padded' data things that compresses relatively well under the weak NTFS compression)

(I would even defrag a SSD if it meant doing it once for read-only stuff. ordered habits of mine)
Post 19 Jan 2010, 17:48
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17279
Location: In your JS exploiting you and your system
revolution
Borsuc wrote:
(I would even defrag a SSD if it meant doing it once for read-only stuff. ordered habits of mine)
That is crazy. SSDs don't have any read head so defragging is completely pointless.
Post 19 Jan 2010, 17:54
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
Crazy but ordered Razz (and doing it once doesn't cause pain anyway).
Post 19 Jan 2010, 18:07
View user's profile Send private message Reply with quote
Rookie



Joined: 21 Aug 2003
Posts: 44
Location: Planet Romania
Rookie
Just crazy, no order. The wear-levelling algorithms take care of that.

Your 'internal' hdd+external rack solution is the definition of every 'external' hdd out there. And never mind the defragmentation part, why are you backing up to hdd in the first place?

_________________
This is who I choose to be.
Post 23 Jan 2010, 09:00
View user's profile Send private message Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
revolution wrote:
Borsuc wrote:
(I would even defrag a SSD if it meant doing it once for read-only stuff. ordered habits of mine)
That is crazy. SSDs don't have any read head so defragging is completely pointless.
Yeah - while there's still some tiny penalty for fragmented data on a SSD, I doubt you're going to be able to quantify it under normal situations... and considering that SSDs have a limited amount of erase cycles, defragmenting them is borderline stupid. Heck, it's past the border.

_________________
Image - carpe noctem
Post 23 Jan 2010, 09:13
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17279
Location: In your JS exploiting you and your system
revolution
Also, SSDs have a limited data life (like 10 years) so using SSDs for backups is kind of silly (and expensive), a normal spinny HDD would be both cheaper and a longer term solution.
Post 23 Jan 2010, 09:29
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
Rookie wrote:
Just crazy, no order. The wear-levelling algorithms take care of that.
What do you mean? If I view it with a hex editor, or even with the Windows defrag utility, it will show a continuous stream of bytes, not fragmented files: ordered. Confused

Rookie wrote:
Your 'internal' hdd+external rack solution is the definition of every 'external' hdd out there.
I'm not sure I understand what you mean here. I only said that "pre-packaged" external HDDs are crap, just look at the negative reviews about them, I hardly think that an internal HDD would have such fail rate since they are made to operate daily probably. Not to mention it was a cheaper solution (than comparable quality external HDDs).

With external HDDs you don't even know what kind of crap HDD they put inside the enclosure Rolling Eyes

Rookie wrote:
And never mind the defragmentation part, why are you backing up to hdd in the first place?
What other options are out there? I mean easy solutions.

Besides, if you are referring to optical disks, they are extremely unreliable. I used to make backups on them but 70% of them failed within one year (couldn't be read anymore). Home-made burned CDs or DVDs or whatever are very unreliable, they are 'burned' with a laser, not etched like in the factory -- the holes aren't physical, just change the properties of the material. Unfortunately that also makes them very susceptible to 'wear and tear', unless they are stored at absolute zero temperatures Razz

Not to mention I would need a lot of DVDs to cover the space I needed to backup (around 160GB). What else would you recommend? HDDs were the easiest and cheapest solution.

f0dder wrote:
Yeah - while there's still some tiny penalty for fragmented data on a SSD, I doubt you're going to be able to quantify it under normal situations... and considering that SSDs have a limited amount of erase cycles, defragmenting them is borderline stupid. Heck, it's past the border.
How many write cycles does it have? 100,000? 1 million? more?

Is a write a few times really that significant? Since I already said I only defragment it once/read-only media Wink

revolution wrote:
Also, SSDs have a limited data life (like 10 years) so using SSDs for backups is kind of silly (and expensive), a normal spinny HDD would be both cheaper and a longer term solution.
While that is true in theory, in practice HDDs are lucky to last 4 years until they break -- either the motor or whatever else. (and data recovery is expensive)

And yes I do agree that SSD for backup is silly at this point, because it's too expensive.

My philosophy of backup is more like "mobile" backup, that is, I'll probably buy a new backup HDD after a few years to store my data, not until this one breaks. Because I know how often HDDs can fail, I've seen it first-hand Sad

_________________
Previously known as The_Grey_Beast
Post 23 Jan 2010, 20:15
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17279
Location: In your JS exploiting you and your system
revolution
Borsuc wrote:
If I view it with a hex editor, or even with the Windows defrag utility, it will show a continuous stream of bytes, not fragmented files: ordered.
That is just an illusion created by the drive. Internally you would have no idea how the data is ordered.
Borsuc wrote:
... in practice HDDs are lucky to last 4 years until they break ...
You must have some really bad luck with HDDs. I've never had even one fail. All my old drives from 1990's are still in working order.
Post 24 Jan 2010, 01:57
View user's profile Send private message Visit poster's website Reply with quote
Coddy41



Joined: 18 Jan 2009
Posts: 384
Location: Ohio, USA
Coddy41
Borsuc wrote:
HDDs are lucky to last 4 years until they break -- either the motor or whatever else.

Wow Shocked What do you do run them hard 25/7? The only HDD I have seen break is
that one I never got back together Laughing

_________________
Want hosting for free for your asm project? You can PM me. (*.fasm4u.net)
Post 24 Jan 2010, 02:10
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
revolution wrote:
Also, SSDs have a limited data life (like 10 years) so using SSDs for backups is kind of silly (and expensive), a normal spinny HDD would be both cheaper and a longer term solution.
Do the flash memory cells degrade without use? Do they degrade when read?

Borsuc wrote:
With external HDDs you don't even know what kind of crap HDD they put inside the enclosure Rolling Eyes
If I buy an enclosure from WD, I kinda expect it to have a WD disk inside? Smile - but if you buy an enclosure from a company that don't manufacture drives themselves, I guess all bets are off unless they specifically list which drives they use.

Borsuc wrote:
f0dder wrote:
Yeah - while there's still some tiny penalty for fragmented data on a SSD, I doubt you're going to be able to quantify it under normal situations... and considering that SSDs have a limited amount of erase cycles, defragmenting them is borderline stupid. Heck, it's past the border.
How many write cycles does it have? 100,000? 1 million? more?
No idea, really - except that SLC has more than MLC, and I bet there's quality differences between individual brands as well. Also, it's going to be a bit hard to predict just how fast you use those erase cycles because of things like block size and wear-levelling.

Thing it that with a SSD, there really isn't much/any point in defragmenting, so you might as well not waste any erase cycles by doing a defragment Smile

Borsuc wrote:
And yes I do agree that SSD for backup is silly at this point, because it's too expensive.
Indeed - much better to use them for reliable system+source+docs drive, and do (incremental and versioned!) backups to cheap external harddisk-based storage. Even if an SSD is unlikely to fail, and (afaik) failures should just mean individual cells turn read-only (as opposed to harddrives unreadable sectors or failing motors, head crashes, ...) a SSD doesn't guard you against "logical" crashes...

revolution wrote:
Borsuc wrote:
... in practice HDDs are lucky to last 4 years until they break ...
You must have some really bad luck with HDDs. I've never had even one fail. All my old drives from 1990's are still in working order.
Sounds more like you have been very lucky with your drives Smile

I've been through a relatively large amount of drives - a lot of them for my own computer, but I've also been the one that has helped friends and family with computers, and I spent a number of years managing IT at a local museum. My observation is that some disks die way before expected, some survive eerily long. Apart from unlucky production series (IBM deathstars still being the most famous, I think), there hasn't been much difference between the various disk vendors; nobody's perfect, you can't guarantee against flukes, and even higher-end disks like Raptors aren't immune.

One thing I've noticed (non-scientifically) is that keeping the disks cool tend to result in less problems. We're not talking datacenter level cool, just decent airflow (like a 120mm intake fan in front of your disks). The first server I built for the museum was placed in a really small storage room (old toilet, about two square metres max) and had a way too cramped casing as well... after replacing two dead drives, I moved components to a larger casing and requested a ventilation system for the storage room (which was as simple as a window-pane ventilator - did the trick).

But again: even with proper cooling, you're not safe from flukes. One of the 500GB WesternDigital drives in my fileserver died without warning; pretty glad I have mirroring in that system. This also taught me the importance of having your RAID monitoring daemon properly configured; it hadn't sent me a warning mail, so it took two weeks before I discovered it by accident while checking server logs for something unrelated Smile

_________________
Image - carpe noctem
Post 24 Jan 2010, 15:58
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
revolution wrote:
That is just an illusion created by the drive. Internally you would have no idea how the data is ordered.
You're right but I don't think I care about that, seeing as I can't see it (the internal storage) without opening up and breaking the drive Razz

f0dder wrote:
If I buy an enclosure from WD, I kinda expect it to have a WD disk inside? Smile
yeah but not what kind of "class" the drive is in. I guess they use cheaper drives in those situations because they think probably that they are much less used than normal internal drives which get powered everytime the computer does also. I don't know much about WD seeing as I had really bad experiences with those drives (but I know it was just me being unlucky, still I stick with Seagate Razz), but I know Seagate have different series and Barracude is made to be more reliable -- I remember seeing a review about a dead external HDD and the guy saying the HDD inside was crap or something. Still, it's something to look out for, especially considering that the external rack + internal HDD was cheaper solution (with the rack also being cool having a fan and all that).

f0dder wrote:
One thing I've noticed (non-scientifically) is that keeping the disks cool tend to result in less problems.
I sort of arrived at the same conclusion, which is why with the new rig I put the HDD into a Scythe cooler, the drive never went above 30 degrees even in the summer -- I was impressed Smile

(on the other hand, very low temperatures are also damaging to the HDD, the recommended is between 22-30 Celsius on operation obviously, not the environment temperature (which is obviously lower than that!))

_________________
Previously known as The_Grey_Beast
Post 24 Jan 2010, 20:40
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17279
Location: In your JS exploiting you and your system
revolution
f0dder wrote:
Do the flash memory cells degrade without use?
Yes. The little floating gate will gradually lose the ability to store charge each time it is erased/prorgammed.
f0dder wrote:
Do they degrade when read?
No. But just the passing of time will allow electrons to leak out of the floating gate and some bits will change state. If you have a stasis machine then you can use that to extend the storage life of SSDs.
f0dder wrote:
One thing I've noticed (non-scientifically) is that keeping the disks cool tend to result in less problems.
Which is weird because the Google report mentioned that they found that heat had no bearing upon HDD life. Neither beneficial or detrimental. And they have lots of drives in various usage situations, their testing base is very large.
Post 24 Jan 2010, 23:44
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
revolution wrote:
f0dder wrote:
Do the flash memory cells degrade without use?
Yes. The little floating gate will gradually lose the ability to store charge each time it is erased/prorgammed.
I know that it degrades with erase, but I was asking about without use - ie., just storing the drive.

revolution wrote:
f0dder wrote:
Do they degrade when read?
No. But just the passing of time will allow electrons to leak out of the floating gate and some bits will change state.
Any idea/estimate for how rapidly this happens?

revolution wrote:
f0dder wrote:
One thing I've noticed (non-scientifically) is that keeping the disks cool tend to result in less problems.
Which is weird because the Google report mentioned that they found that heat had no bearing upon HDD life. Neither beneficial or detrimental. And they have lots of drives in various usage situations, their testing base is very large.
Yes, I saw that report (only read a summary), and the "temperature doesn't matter" bit kinda surprised me - it doesn't correlate with my experiences. But I guess it might be "within normal limits", whereas some of the situations I've had have been with pretty high temperatures (bad airflow, perhaps lots of stacked drives, perhaps with pretty high temperature because of powerful GPU (which you won't see in a server), etc).

Guess I should look more deeply into the report, and see if it mentions measured temperatures... makes a difference whether the claim is "20C vs 40C drive operating temperature doesn't matter", or if they're saying that something even higher has no effect (I've had >50C under stress, which is when I first installed a 120mm intake fan... dropped it down to ~30).

Google does run their data centres at quite higher temperatures than other people (service techs running around in shorts and t-shirt), but the article where I read that did mention that they do still actively cool the centres (just that having ambinet heat at 27C (can't remember if that's the correct temperature, might be higher or lower) takes less effort (thus energy thus money) than getting it down to 20C or whatever normal data centres use. The article also mentioned that while using "cheap" hardware and going by the "individual servers will break, and daily" they've chosen components specifically for being reliable under those relatively high temperatures.

_________________
Image - carpe noctem
Post 25 Jan 2010, 00:55
View user's profile Send private message Visit poster's website Reply with quote
Tyler



Joined: 19 Nov 2009
Posts: 1216
Location: NC, USA
Tyler
f0dder wrote:

Any idea/estimate for how rapidly this happens?

I have no knowledge on the topic, but it seems that all your questions have already been answered.
revolution wrote:

...SSDs have a limited data life (like 10 years)...

f0dder wrote:

...I was asking about without use...

revolution wrote:

...just the passing of time will allow electrons to leak out of the floating gate and some bits will change state.
Post 25 Jan 2010, 01:12
View user's profile Send private message Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
Tyler wrote:
I have no knowledge on the topic, but it seems that all your questions have already been answered.
Well, not exactly - where does the "10 years" estimate come from? Does it cover MLC, SLC, or both? Is it a hard "decay time for unused drives" timespan, or based on regular use? If based on regular use, what kind of write patterns?

Smile
Post 25 Jan 2010, 01:17
View user's profile Send private message Visit poster's website Reply with quote
Tyler



Joined: 19 Nov 2009
Posts: 1216
Location: NC, USA
Tyler
I see your point now, I didn't really consider how many variables go into something like that. I think what revolution is talking about is that a flash drive stored for 10 years without use would lose some of the data stored on it, just by the things that store the bits losing charge.
Post 25 Jan 2010, 01:40
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar.

Powered by rwasa.