flat assembler
Message board for the users of flat assembler.

Index > Heap > Reading damaged floppy disk sectors

Goto page 1, 2  Next
Author
Thread Post new topic Reply to topic
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
I was given a floppy disk with damaged sectors, and although I successfully recovered almost all files by creating an image with ddrescue, I'm stuck at three unrecoverable sectors (and as a result a DOC file is damaged).

Are there any chances to force Linux or Windows to ignore CRC errors and get the data as read anyway? Is it even possible if I use Assembly to access the Floppy drive in PIO mode and no OS at all? (Assuming the sector header is not broken, but if you know how to read raw tracks please let me know)

Other possibilities of recovering the files have been exhausted, this floppy disk is really the only copy they have left...
Post 04 Feb 2014, 04:08
View user's profile Send private message Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 8975
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 334455
sleepsleep
maybe, you could name us the file name if it is something exists in public? maybe could try our googlefu?
Post 04 Feb 2014, 04:32
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17350
Location: In your JS exploiting you and your system
revolution
Spinrite can sometimes recover damaged sectors on HDDs and floppies.
Post 04 Feb 2014, 05:14
View user's profile Send private message Visit poster's website Reply with quote
sinsi



Joined: 10 Aug 2007
Posts: 695
Location: Adelaide
sinsi
Have you tried another drive? Floppy drives were notorious for misaligning, quite often formatting in one drive meant that it was unreadable in others.
What size floppy? You might be able to move the actual disk into another cover.
The CRC error comes from the drive controller, not the OS (it just passes it along).

I wrote a formatter for CP/M way back, this let me write everything to the track (address marks, pre- and post-gap etc.). There should be a way of reading a track in the same way.
Post 04 Feb 2014, 08:07
View user's profile Send private message Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
sleepsleep, the information inside the doc file was composed by them, there is **really** no other copy of the data inside.

revolution, the problem is that is saves the recovered data back to the unreliable medium and also might stress the disk too much. I would like to extract the data before as is, given that the FDDs are super dumb and send everything as is to the FDC. Then, I might try Spinrite.

sinsi, yes, at least tried three FDDs of different brand and manufacture year. It is a 3 1/2 1.44 MB floppy disk. I don't think it is a misalignment problem because it is only 3 sectors that can't be recovered and are not evenly spaced. I know the controller takes care of the CRC, but I was wondering if in PIO (or even DMA) mode perhaps bytes are transmitted to the CPU as soon as they are available rather than buffering? In that case, I would receive a copy of what was read followed by an error condition (which contrary to what an OS would normally do, I'd ignore it and perhaps take several samples). I'm afraid that if perhaps that was the case in the past, now modern chipsets buffer everything to both, avoid an unfruitful transfer loop and to allow the FDC to be read at any time since the internal buffer is large enough to hold the entire read request.

My last resort is to see if I can control an FDD with an Arduino Leonardo or risk my precious (and ashamedly not really used yet) Nexys3, which since it has an FPGA with no 5V inputs I'm very worried if the voltage conversion techniques don't work well (also considering that the FDD and the FPGA would have different power supplies and my not broad knowledge on electronics).

Thanks to all.
Post 04 Feb 2014, 18:39
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17350
Location: In your JS exploiting you and your system
revolution
LocoDelAssembly wrote:
... I was wondering if in PIO (or even DMA) mode perhaps bytes are transmitted to the CPU as soon as they are available rather than buffering?
If you use low level I/O commands you can set the controller to deliver all bytes read from the disk surface including the ECC bytes. Boot into something like DOS to give you low level access.

Often for damaged sectors you will find portions of the data change each time you read the sector. You can load and store multiple reads and then post-process the captured data to try and reconstruct the data.

But be aware that not all data is recoverable. Sometimes physical damage on the disk surface is just too great and there is no data left in that spot. The ECC can correct up to a certain point and is generally optimised for a single run of burst error correction. If you have two or more spots with burst errors then the ECC info is mostly going to be useless.
Post 05 Feb 2014, 02:04
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
Aha! I've checked some sources months back and somehow I never heard about a read track command but in fact it does exist!

http://www.isdaman.com/alsos/hardware/fdc/floppy.htm#Commands
http://www.classiccmp.org/dunfield/r/765.pdf (NEC uP765 spec also listing read track as a command)

Also by rechecking a Spanish book (EL UNIVERSO DIGITAL DEL IBM PC, AT Y PS/2) I also see the command explained briefly. Wonder if I actually learned about this command before and later I forgot. Confused

Thanks a lot!
Post 05 Feb 2014, 03:42
View user's profile Send private message Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
A follow up, I found a program to get raw images for WindowsXP ( http://simonowen.com/samdisk/ ).

After getting 4 samples, converting to raw, hex dumped and then ran them through a diff tool I found what it is attached. I'm not sure how much chances I have to recover something from the faulty bits, these samples have very large burst of difference. My next step will probably be using the driver supplied on the same site to do my own low level reading so I can take lots of samples of the affected sectors, and after that, I´ll probably let SpinRite do its thing.
[edit]Console output
Code:
C:\Documents and Settings\Usuario\Escritorio>SAMdisk.exe image.dsk image.raw
Warning: cyl 0 head 1 sector 8 (id=9) has a data CRC error
Warning: cyl 32 head 1 sector 0 (id=1) has a data CRC error
Warning: cyl 33 head 1 sector 0 (id=1) has a data CRC error
Warning: source disk missing 108 sectors from regular format
Wrote 83 cyls, 2 heads, 18 sectors,  512 bytes/sector = 1529856 bytes

C:\Documents and Settings\Usuario\Escritorio>SAMdisk.exe image2.dsk image2.raw
Warning: cyl 0 head 1 sector 8 (id=9) has a data CRC error
Warning: cyl 32 head 1 sector 0 (id=1) has a data CRC error
Warning: cyl 33 head 1 sector 0 (id=1) has a data CRC error
Warning: source disk missing 108 sectors from regular format
Wrote 83 cyls, 2 heads, 18 sectors,  512 bytes/sector = 1529856 bytes

C:\Documents and Settings\Usuario\Escritorio>SAMdisk.exe image3.dsk image3.raw
Warning: cyl 0 head 1 sector 8 (id=9) has a data CRC error
Warning: cyl 32 head 1 sector 0 (id=1) has a data CRC error
Warning: cyl 33 head 1 sector 0 (id=1) has a data CRC error
Warning: source disk missing 108 sectors from regular format
Wrote 83 cyls, 2 heads, 18 sectors,  512 bytes/sector = 1529856 bytes

C:\Documents and Settings\Usuario\Escritorio>SAMdisk.exe image4.dsk image4.raw
Warning: cyl 0 head 1 sector 8 (id=9) has a data CRC error
Warning: cyl 32 head 1 sector 0 (id=1) has a data CRC error
Warning: cyl 33 head 1 sector 0 (id=1) has a data CRC error
Warning: source disk missing 108 sectors from regular format
Wrote 83 cyls, 2 heads, 18 sectors,  512 bytes/sector = 1529856 bytes    
(No errors occur during SAMdisk.exe a: image*.dsk)[/edit]


Description:
Filesize: 583.61 KB
Viewed: 8513 Time(s)

diffuse.png


Post 17 Feb 2014, 19:03
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17350
Location: In your JS exploiting you and your system
revolution
Yeah, a single missed bit change will alter all of the following data. So if you have multiple errored bits in succession things become exponentially harder to decode.

Fortunately you are reading a floppy disk which has a simpler and documented encoding format so recovery of the bulk of the data should be possible. The latest HDDs have considerably more complex and undocumented formats so in that case backups are often the only option.
Post 18 Feb 2014, 10:33
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
Do you mean that since we know floppies are MFM-encoded that knowledge can help me to infer actual values from the samples or that it would be an advantage if I use the FPGA? Would the FPGA give me any advantage at all? (Connected to the FDD, not directly to the head)

About reading all bytes from surface, is that possible on SATA or even older PATA IDE HDDs? Was even possible on ancient HDDs without implementing your own controller (or have a specialized one if such thing existed)?

BTW, when I opened the damaged document using the first image I got something that didn't make Word freeze, and it kinda looks the data are there, but the table layout is severely garbled and (at least to me) it is not obvious in which columns each datum is supposed to fit (except the last few rows where it seems there is no damage there). Perhaps there is hope to recover the data even if I cannot improve the recovery any further.
Post 19 Feb 2014, 00:15
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17350
Location: In your JS exploiting you and your system
revolution
Yes, knowledge of MFM will definitely help in the recovery. I think an FPGA wouldn't help much in comparison to a CPU, just use some software to re-time and recover.

AFAIK all HDD controllers have modes to return all sector and ECC bytes. The only thing you don't get is the actual magnetic changes recorded since the storage format is proprietary and unknown.

Word documents have a lot of redundancy so even partial data can sometimes be enough to recover the information.
Post 19 Feb 2014, 00:41
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
Sorry for taking so long to reply.
Quote:
I think an FPGA wouldn't help much in comparison to a CPU, just use some software to re-time and recover.
Just making sure I'm not missing anything here: Do you mean re-timing from the already decoded BYTES* delivered by the FDC or there is yet an even more raw access that returns every magnetic reversal found by the head? With the FPGA I was hoping that perhaps I could extract "sub bits". Do you think something like that could happen or the FDD already performs some filtering rather than just amplifying and conditioning the signal to produce a square wave without limiting bandwidth?

Quote:
AFAIK all HDD controllers have modes to return all sector and ECC bytes. The only thing you don't get is the actual magnetic changes recorded since the storage format is proprietary and unknown.
I later found by exercising Google correctly that ATA defines READ SECTOR LONG, but has been deprecated since version 4 precisely to let manufacturers store sectors in whatever way they want. However, next time I receive an HDD with damaged sectors I'll check if per chance the command is included in its capabilities.

Thanks again for all the info!

*Not meaning that is impossible here, just wanted to be sure this is the maximum deepness I can get from a computer.
Post 02 Mar 2014, 15:55
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17350
Location: In your JS exploiting you and your system
revolution
For FD data the format is simple. You can write some code (like spinrite does) to try combinations of bits in places where they might have been corrupted. An FPGA won't for that for you.

For HD data you have to rely on the controller since you don't know the encoding format.

Backups are your friend. Razz
Post 02 Mar 2014, 17:23
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
Quote:
For FD data the format is simple.
Yes, but are we in agreement that if we compare this to 10 mbps Ethernet the FDC's Read Track command is giving us layer 2 frames while the FPGA would give us digital samples of the 20 MHz layer 1 Manchester coded signal?

I understand that I can try flipping suspicious bits (and calculate how them would affect subsequent bits) until I match the CRC, but I was wondering if I get raw samples from the FDD (which I'm expecting the MFM data without any decoding) I could perhaps detect quick flux reversals or any hint helping me to reduce the search space. (With the FPGA I would still have to take several samples)

Sorry for being so insistent Embarassed
Post 03 Mar 2014, 02:24
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17350
Location: In your JS exploiting you and your system
revolution
So let me get this straight: You want to use an FPGA as a signal capture device and then use a CPU to analyse and recover? It looks to me as though you want to make a sort of storage oscilloscope thing with the FPGA. If so then the FDD controller is already doing that internally (or at least it is supposed to do that) and give you the best decoding it is able to. It is likely you will just end up with the same data you have in your current captures. But I would be keen to see if you can get something different and usable.
Post 03 Mar 2014, 03:48
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
Yes, the FPGA board (NEXYS 3) as either 1)some sort of digital storage oscilloscope on the FDD output pins while letting the FDC drive the FDD for me and just listen from the "drive B" plug in the ribbon cable (this is possible, right?), or 2)drive the FDD myself (which perhaps would let me read tracks from several positions by violating the time constraints of the STEP signal).

I probably won't go for 2), but I'd like to test 1) if it can give me any advantage. Take look at 8.3.10 READ DATA output signal and Fig. 8.3-3 in TEAC FD-235HF-C829 MICRO FLOPPY DISK DRIVE SPECIFICATION. What I'm expecting in order to have any chance of getting more data is that the FDD does not ensures the t* timings but rather sends all flux reversals it finds with that negative short pulse (and if lucky perhaps some FDDs send an even shorter pulse for weak bits that maybe I'll manage to catch with the FPGA where the FDC won't, so essentially I could detect more reversals).

Do you think there is any chance I can get better decoding than the FDC and/or improve the detection of troublesome bits or this is just a waste of time (and resources if I damage the FPGA in the process)?
Post 03 Mar 2014, 16:35
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17350
Location: In your JS exploiting you and your system
revolution
I have never tried such a thing. But it is never a waste of time if one is learning something in the process.

I think the off-track-stepping thing might be worth a try. FDDs always had trouble with alignment. It would be tricky to time the steps accurately to match the place where the read data is corrupted. For this part I think that driving your own signals, and not using a controller, would give the best chance. But since your can read almost all of the disk with only a few dud sectors then I suspect that the alignment is not too far off. Most probably the disk is low quality and has lost its magnetic coating.

I hope the data is really worth all the effort. Is it the recipe to immortality? :p
Post 03 Mar 2014, 16:55
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
Yes, I also think that head positioning is not the issue here, but maybe by reading from the sides instead of (approximately) the center of the track could recover something. Two of the three damaged sectors are in the same track and sector but on opposing sides. Not sure what kind of physical damage causes that.

Quote:
I hope the data is really worth all the effort. Is it the recipe to immortality? :p
haha, no, but the floppy contains the data from a series of measurements in insect collecting points that started in 1996 and ended in 2002. The broken file is the 2002 data (which I continue to be surprised at how it is possible they had one single copy on a floppy all this long time).

Okay, I guess I could give the FPGA a try then. Do you know if my proposed wiring is correct? Any suggestion of what should I use to adapt the FDD signal to 3.3 V? Would a simple resistor divider be enough? (Don't know how much current can I draw from the FDD yet, though)

The board pins are already protected by 200 Ohm resistors and Zener diodes (and I have an expansion board with unprotected pins if necessary), but to be safe I don't want to trust the board's ability to clamp the voltage (which is actually there to protect from ESD, not to actively run 5V stuff).
Post 03 Mar 2014, 17:53
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17350
Location: In your JS exploiting you and your system
revolution
CMOS 3.3V drivers to the FDD TTL 5V input receivers should be no problem.

For the CMOS receivers being driven from the FDD by open collector outputs means you will need pull-up resistors to the 3.3V supply on the FPGA board. ~200ohms to 1kohms should be good. The lower the value the better the signal.
Post 03 Mar 2014, 18:36
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
Quote:
For the CMOS receivers being driven from the FDD by open collector outputs means you will need pull-up resistors to the 3.3V supply on the FPGA board. ~200ohms to 1kohms should be good. The lower the value the better the signal.
After reading the 82077AA spec, it seems that there are some 3.5" drives that actually use totem-pole drivers. In fact, the spec says **most** 3.5" drives use such drivers.

I honestly don't know what to think here, I have a Cyrix 586 computer (bought in 1995-1996) which has a 5.25" and a 3.5" FDD connected on the same cable and it is not clear to me whether open collector and totem-pole FDDs can coexist without problem or both are open collector. If the FDDs I have here turn out to be totem-pole, I think I'd put the board's Zener diodes to test, something I'd like to avoid. Also, I don't know the consequences of connecting a 3.3V pull-up on a 5V totem-pole driving HIGH.

Would interfacing the FPGA with MOSFET transistors be both, safe and provide a reliable signal? (Not sure if are easy to find nor which part numbers should I look for, though)

Why couldn't my FPGA just support 5V... Sad
Post 04 Mar 2014, 04:32
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page 1, 2  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on YouTube, Twitter.

Website powered by rwasa.