flat assembler
Message board for the users of flat assembler.

Index > Heap > Petabox and the death of NTFS

Goto page Previous  1, 2, 3 ... 8, 9, 10, 11, 12  Next
Author
Thread Post new topic Reply to topic
bitRAKE



Joined: 21 Jul 2003
Posts: 2887
Location: [RSP+8*5]
bitRAKE
RAID-0 most likely.
Post 13 Aug 2009, 13:36
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
revolution wrote:
Borsuc wrote:
If power does go down, I copy the stuff from the ramdrive, takes around 5-6 seconds to copy 1 GB from it ...
Wow, you can get ~200MB/s transfer to your HDD! Shocked I guess it must be the latest SATA interface at 3Gb/s? I've not had the chance to try these latest drives lately.

[edit]
The fastest drive I can find is the Seagate Cheetah 15K.5. Even that can only get to 125MB/s for such a large transfer as 1GB (and that is with a 3Gb/s interface). What drive are you using? I am curious to see what you bought that is so fast.
No it's not RAID, but I guess it may be because of NCQ and a 32MB buffer, because after copying is done it still flashes 3-4 seconds.

I never copied 1GB, just 500MB and it was about half the time I said (around 3 seconds or so), that's why I mentioned that way. Not sure if copying 1GB is gonna slow it down.

EDIT: the reason I actually even got this HDD with huge capacity (For me) is because of the buffer. Any with less capacity in the store didn't have NCQ + 32MB buffer Confused

_________________
Previously known as The_Grey_Beast
Post 13 Aug 2009, 17:01
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17247
Location: In your JS exploiting you and your system
revolution
Borsuc: Make and model?
Post 13 Aug 2009, 20:55
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17247
Location: In your JS exploiting you and your system
revolution
Borsuc wrote:
No it's not RAID, but I guess it may be because of NCQ and a 32MB buffer, because after copying is done it still flashes 3-4 seconds.
I think I see what is happening here. You see the copy dialogue box disappear after ~3 seconds, but in the background the OS is still copying the last part of the data from internal buffers to the drive. That internal 32MB buffer is not going to last for 3-4 seconds, it is just too small. I really really seriously doubt that you have a drive that is able to write data as fast as the timing implies. There is just no commercially available drive available that can do it.

Also those extra "3-4 seconds" are important to note because your UPS has to be able to hold up the power for that much longer to ensure that you don't lose anything.

Are you using a program detect that power failure and start the transfer to HDD? Or is it manual?
Post 14 Aug 2009, 00:04
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
revolution: not to forget that he mentions he has only copied 500megs from the 1gig ramdrive Smile
Post 14 Aug 2009, 12:11
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
I don't know exact model. It's a Seagate Barracuda 7200 750GB with 32MB buffer and NCQ (I never actually look at exact HDD model when I buy one, just specs), and yes I guess you are right about it that after the dialog finishes it's still important to let it end. So probably it takes around 10-12 seconds, plenty of time available. The UPS lasts several minutes anyway. Smile

_________________
Previously known as The_Grey_Beast
Post 14 Aug 2009, 19:43
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17247
Location: In your JS exploiting you and your system
revolution
Borsuc wrote:
So probably it takes around 10-12 seconds, plenty of time available.
The sounds quite plausible, and I would expect for a 7200rpm drive for the time to be more likely at the high end of that range and perhaps a little bit more depending upon the drive location you are writing to.
Borsuc wrote:
The UPS lasts several minutes anyway.
But do you need to manually start the transfer? If so, then you need to be at your PC during power failures. Do you leave you PC on when you are not around?
Post 15 Aug 2009, 14:14
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
I don't leave stuff on the ramdisk when I'm away for long periods (let's say not for a leak Razz). Either leave it on or put it in standby mode, but don't keep important stuff on the ramdisk (however if it's only for cache purposes then I won't clear it, why would I?)

_________________
Previously known as The_Grey_Beast
Post 16 Aug 2009, 15:32
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17247
Location: In your JS exploiting you and your system
revolution
One-terabyte SSDs ... about the size of the average postage stamp.

http://www.physorg.com/news185438129.html

So just one thousand postage stamps and we have the petabox drive.
Post 15 Feb 2010, 14:23
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
revolution wrote:
One-terabyte SSDs ... about the size of the average postage stamp.

http://www.physorg.com/news185438129.html

So just one thousand postage stamps and we have the petabox drive.
I wonder about speed, reliability and... *shudders* price Smile

Interesting that they say the stamp-size is including controller - but I guess that'll be the NAND-memory access controller, not a full SATA interface controller. In other words: drives will be larger. But hey, 2.5" form-factor for 1TB SSD wouldn't be bad Smile, and with postage-stamp size, they could cramp several terabytes in a 2.5" enclosure.

_________________
Image - carpe noctem
Post 15 Feb 2010, 14:29
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
256GB is hugely expensive and they talk about 1TB SSDs? Confused Sad

_________________
Previously known as The_Grey_Beast
Post 15 Feb 2010, 17:26
View user's profile Send private message Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
Borsuc wrote:
256GB is hugely expensive and they talk about 1TB SSDs? Confused Sad
It's some years in the future, and the new fabrication method probably reduces costs as well?

_________________
Image - carpe noctem
Post 15 Feb 2010, 17:27
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
I surely hope so. But you know, I kinda feel bad when 1TB is announced and I am years away from affording 256GB (i.e around same price as a 500GB HDD or so). Razz

_________________
Previously known as The_Grey_Beast
Post 15 Feb 2010, 20:28
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17247
Location: In your JS exploiting you and your system
revolution
In general I still feel that currently SSDs are a bad idea. Expensive, small, and limited write endurance. If the OS does not understand the underlying storage medium then you could potentially burn through your write endurance prematurely. Hopefully this can change in the future. Time will tell.
Post 15 Feb 2010, 20:47
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
revolution: what do you feel an OS can do that can't be achieved with wear leveling? (serious question). I assume there's issues to think of at both caching and filesystem levels, knowledge about erase block size, block alignment of data structures (some of that will be relevant when moving to 4kb sectors on regular harddrives), etc.

But we'd still want our filesystems to be journaled, won't we? That makes some of the "extra" erase cycles unavoidable.

I wish there was a way to see per-block erase count (and other stats - I wonder if cell "health" is measurable?) for a SSD, would be quite interesting.

Anyway: expensive and small, yes... definitely not useful for bulk storage as things are now. But apart from databases, normal harddrives are pretty decent for bulk storage, whereas current SSDs are pretty nice workstation-wise for OS, applications, data files etc. I definitely enjoy the one I have in my workstation, and I'd like to replace the HDD in my latop with an SSD (makes a lot of sense to avoid movable parts in something you carry around, and the speed difference from laptop-HDD to SSD is even more noticable than Raptor->SSD).
Post 15 Feb 2010, 21:00
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17247
Location: In your JS exploiting you and your system
revolution
If the OS knows what the real internal block size is then writes can be better optimised. Currently SSDs fake things and report a 512 byte size so the OS will happily overwrite adjacent sectors and not realise that the SSD has to erase and relocate an entire block (usually 128kB) just to accommodate a 512 byte write.
Post 15 Feb 2010, 21:11
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
revolution wrote:
If the OS knows what the real internal block size is then writes can be better optimised. Currently SSDs fake things and report a 512 byte size so the OS will happily overwrite adjacent sectors and not realise that the SSD has to erase and relocate an entire block (usually 128kB) just to accommodate a 512 byte write.
Yup - OSes tend to do write caching though, which reduces this effect somewhat. And you can tune cluster/block sizes to avoid multiple files in one erase-block (but ugh, 128kb cluster size... that's a fair amount of wasted space).

So I think the biggest issue is structure & alignment of fs metadata, and how often it's written?

PS: I recall reading something about the ATA specs allowing querying of "real" sector size? Didn't really look into it, but it would seem relevant for both SSDs and the new 4kb-sector HDDs. Dunno what a SSD would/should report though, even if this command exists... smallest addressable sector is smaller than erase-block size, right? And you "only" need to erase a block if writing to a non-zero sector - and could that be further optimized to only erase if it's determined that bits have cleared (and would it be worth it? Smile).

_________________
Image - carpe noctem
Post 15 Feb 2010, 21:18
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17247
Location: In your JS exploiting you and your system
revolution
NAND does not usually allow multiple writes with just clearing bits. You are forced to erase and start over. I'm not sure why but all the FLASH chips I use tell me that.

NOR is different, you can overwrite and just program single bits without restriction. However that is not really useful for a drive, only rarely would the situation be available.

Generally the NAND chips allow 512byte block writes and 128kB block erase (sizes depend upon chip and manufacturer). And once a 512byte block is written you can't change it without erasing the whole 128kB section.
Post 15 Feb 2010, 21:27
View user's profile Send private message Visit poster's website Reply with quote
Borsuc



Joined: 29 Dec 2005
Posts: 2466
Location: Bucharest, Romania
Borsuc
SSDs don't have a buffer? My HDD has 32MB buffer, I assume it's used exactly for this purpose. Right? (for HDDs it's probably to avoid too many head movements)

_________________
Previously known as The_Grey_Beast
Post 16 Feb 2010, 20:15
View user's profile Send private message Reply with quote
baldr



Joined: 19 Mar 2008
Posts: 1651
baldr
Borsuc,

In the most cases, controller's internal buffer is the only (8/16KiB) protection from wearing. NCQ? Think again.
Post 16 Feb 2010, 21:57
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3 ... 8, 9, 10, 11, 12  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar.

Powered by rwasa.