flat assembler
Message board for the users of flat assembler.

Index > OS Construction > file system

Goto page Previous  1, 2, 3, 4  Next
Author
Thread Post new topic Reply to topic
Hayden



Joined: 06 Oct 2005
Posts: 132
Hayden
what about a cacheing style fs. instead of reading in 1 sector at a time then anazlizing it, say read in a whole track/head or multiple heads ( depending on the disk cache size for optimal performance ) and keep an index of which heads is currently stored in memory. the idea is that if you need to read a sector it may have already been read in during the last disk i/o.

OT. for anyone thats interested here is some code i wrote to read multiple sectors a while ago. maybee someone could turn it into a INT13 handler or adopt it into a FS.

Note: there are some code limitations; it's 16-bit realmode code, by default it read's 128 ( 64k ) sectors at a time, if you adjust it to read more you will need to change code so that ES is updated to next segment when ES:BX boundary has been reached, also note that I brute force INT13 if there is error so that means that if code encounter a bad sector it may end up in infinate loop.

anyway if any one use the code please post there mods here thanks.
Code:
; ------------------------------------------------
; BOOT-OS.ASM Build(0620a)
; Created By: Hayden McKay
; ------------------------------------------------
use16

jmp far 007C0h:init_bootsys          ; must be far
align 4

    SystemStack  dd 08000h:0fbffh    ;  1k
    SystemSpace  dd 09000h:00000h    ; 64k

    SectorStart  dw 1                ; lba
    SectorCount  db 128              ; maximum 64k

align 4

init_bootsys:

    ; setup stack ect...

    lss  sp, dword [cs:SystemStack]     ; nmi safe
    pushf
    pusha
    push dx                             ; boot num
    push sp

    ; setup for lba->chs translation...

    mov  ah,08h                     ; drive params
    int  13h
    xor  ah, ah
    mov  al, dh                     ; maximum head
    and  cx,3fh                     ; clear d6->d7
    inc  ax
    mul  cx
    mov  si, cx                     ; sector count
    mov  di, ax                     ; heads x secs

    ; begin the lba->chs, mutli-sector read...

    pop  bp                         ; ptr boot num
    les  bx, dword [cs:SystemSpace]
    align 4

read_sectors:

    mov  ax, word  [cs:SectorStart]
    xor  dx, dx
    div  di
    mov  cl, ah
    mov  ch, al                  ; cylinder d0->d7
    shl  cl, 6d                  ; cylinder d8->d9
    mov  ax, dx
    xor  dx, dx
    div  si
    inc  dl
    mov  dh, al                  ; head
    or   cl, dl                  ; sector d0->d5
    mov  dl, byte  [ss:bp]

    ; read how many sectors...

    mov  al, cl
    and  ax,3fh
    sub  ax, si                     ; - max sec...
    neg  ax                         ; abs
    inc  al                         ; +=1
    cmp  al, byte  [cs:SectorCount] ; more than...
    jbe  @f
    mov  al, byte  [cs:SectorCount] ; less than...
    align 4
    @@:

    ; use the bios int13 i/o...

    pusha                              ; bios safe
    xor  ax, ax                        ; calibrate
    int  13h
    popa
    pusha
    mov  ah,02h                        ; read secs
    stc                                ; bios safe
    int  13h
    popa
    jc   @b                            ; use force

    ; prepare for some more sectors...

    add  word  [cs:SectorStart], ax ; update start
    sub  byte  [cs:SectorCount], al ; update count
    jz   @f                         ; done!
    shl  ax, 9d                     ; *=512
    add  bx, ax                     ; update es:bx
    jmp  near  read_sectors
    align 4
    @@:
    pop  dx                         ; boot num

    ; initiate the system file...

    popa
    popf
    lds  bx, dword [cs:SystemSpace]   ; setup data
    call far dword [cs:SystemSpace]   ; enter code

    ; 157 bytes

    ; Todo list...

    ; * maybee copy the system to the hma
    ; * maybee supply an open file system

    rb 510d - $
    db 055h
    db 0AAh                              ; bootme!
    

_________________
New User.. Hayden McKay.
Post 19 Oct 2007, 14:43
View user's profile Send private message Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
caching hole track or several heads can evectivelly helps

now the problem is not the file system model in drive

it is the linking manner in ram that i want to find
the nearest possible of the µP mecanism

for Real mode
for Protected mode

and for other µP types

exemple: PIC 16F84, 6809, etc

all these have a different memory model
so the ram disk routine is platform dependant

but the global structure need to be the same for all architectures
to make easyer the design by one personn: ME

thanks for the caching idea
making a full head loading seems to be a good idea
the limit of 64k is not a problem
and is simple to use under real mode


wht is the best to index sectors locations?
CHS?, LBA?
chs is a notation that show the geometric position of the sector
assuming the drive is not a flash memory
0 doesn't exists in C/H/S form and can be used like a flag
Lba is the linear location of sector and don't care about geometry

i prefer C/H/S
and you?
Post 19 Oct 2007, 15:52
View user's profile Send private message Visit poster's website Reply with quote
Hayden



Joined: 06 Oct 2005
Posts: 132
Hayden
CHS is fine but i prefer LBA indexing for these reasons;

* sectors can be indexed useing only one variable, ie; for a floppy 16-bits would allow me to index a full 2880 sectors. so no need to keep track of C,H,S etc.. ( my boot code keeps track of sectors in LBA and just translates them to CHS just before the call to INT13 )

* LBA indexing can be translated to CHS/PHS quite easily.

* LBA scheme works better for protected mode.

If you like I'll explain how my code works a little better, it's quite optimized and compact so it might be hard for others to grasp exactly what is going on. anyway i've been meaning to turn it into a r/w int13 handler, maybee i'll get arround to it this weekend

_________________
New User.. Hayden McKay.
Post 19 Oct 2007, 18:02
View user's profile Send private message Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
ok
so it needs to manage CHS and LBA format

CHS is very usefull cause it like the RGB
each component in a part of the DWORD

al=sector
ah=head
ror eax,16
ax=track

with LBA, the problem is to respect drive geometry and limit the heads movements
but the file system functions are there to make it transparent

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

FINALLY:
i'll make it with LBA
but i'm not really happy to make that

the problem is tha hard ware layer, it will waste a lot of time just to convert LBA to CHS, but it is good because it is more like the RAM.
then, HD will be NVRAM


Last edited by edfed on 14 Apr 2008, 02:20; edited 1 time in total
Post 20 Oct 2007, 14:01
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
page 2, file system structure post updated.
you know? the one with the beautyfull JPG driagram Wink
Post 05 Dec 2007, 02:20
View user's profile Send private message Visit poster's website Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 3043
Location: vpcmipstrm
bitRAKE
Hayden wrote:
Code:
shl cl, 6d ; cylinder d8->d9    
This line seems strange to me.
Post 05 Dec 2007, 05:08
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
% Most disk BIOS calls use the following parameter scheme:

AH = function request number
AL = number of sectors (1-128 dec.)
CH = cylinder number (0-1023 dec.)
CL = sector number (1-17 dec.)
DH = head number (0-15 dec.)
DL = drive number (0=A:, 1=2nd floppy, 80h=drive 0, 81h=drive 1)
DL = drive number (0=A:, 1=2nd floppy, 80h=C:, 81h=DSmile
Note that some programming references use (0-3) as the
drive number which represents diskettes only.
ES:BX = address of user buffer


% and return with:
CF = 0 if successful
= 1 if error
AH = status of operation (see INT 13,STATUS)
Post 05 Dec 2007, 05:13
View user's profile Send private message Visit poster's website Reply with quote
LocoDelAssembly
Your code has a bug


Joined: 06 May 2005
Posts: 4633
Location: Argentina
LocoDelAssembly
bitRAKE, but it is OK
Quote:
CL = sector number 1-63 (bits 0-5)
high two bits of cylinder (bits 6-7, hard disk only)


What is "strange" is writting "6" as "6d" Wink (But valid, the "d" subfix means decimal).
Post 05 Dec 2007, 05:26
View user's profile Send private message Reply with quote
bitRAKE



Joined: 21 Jul 2003
Posts: 3043
Location: vpcmipstrm
bitRAKE
LocoDelAssembly wrote:
bitRAKE, but it is OK
Quote:
CL = sector number 1-63 (bits 0-5)
high two bits of cylinder (bits 6-7, hard disk only)


What is "strange" is writting "6" as "6d" Wink (But valid, the "d" subfix means decimal).
Thank you, I was totally blanking on that - was thinking hex, lol. Laughing
(time to sleep)
Post 05 Dec 2007, 05:37
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
ok, cl 0-5 are used for sector now!
in the DOS age, it was limited to 17 sectors in total per head.

and i don't know any good reference about the norm after 1986.

probably, the helppc need to be updated and modernised!

Wink Very Happy
Post 07 Dec 2007, 03:06
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
file system services.

they will be located in boot/fs/
theses services are specialised for this file system.

read file:
a command line give the file name and path
the routine returns the memory location set by the Memory Manager
this location is copied into the ram file system.
if this file is reopen, then, it will point to the first location without reloading.
a modification on an opened file is seen by other applications
file reload can be forced.

write file:
the file doesn't exist on disk, then the disk manager create the file header and prepare the file system to receive the file.
the file exist, then it is simply overwriten.
if the write result in a fragmentation, the disk manager seek an other contiguous area where to move the file.
if no contiguous area found, it test for a possibility to displace the needed sectors, for exemple, a very little file just after the file to save is easy to move.
if no possibility of move, the file is fragmented (zoned)
a command line give the file name and path
if the path doesn't exist, it's created.

file delete:
to delete a file,
is it in ram? if yes, it's erased in ram and ram fs
is it on disk? if yes, the entry is erased in fs
is it a secure delete? if yes file is fully overwriten with 0
if it's nowhere, ne delete is achieved

file creation:
the command line give the path and file name.
the entry in the path is set
if the path doesn't exist, it's created.
the file is a 0 byte lengh file.
no sector is allocated for it.

file system verification:
scan the file system for crossing references.
normally, there cannot be crossing reference.
as a sector cannot be allocated for more than One file.
verification of bad sectors, verification of unreferenced sectors.
if a sector of a file is shown as empty, set it to full.
Post 08 Dec 2007, 00:12
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
one big correction to make...
DO not put the file name in an header.

it's endless.
there is a name in theh folder, and sometimes into the file itself (MP3,WMA,WAV,DOC etc...)
so why having a file name in the header?
just to follow the idea of somebody???
no i disagree, NO FILE NAME IN FILE SYSTEM HEADERS.
but the path in the file header, why not.
not really a path, just a two dwords to give the comming entry from the folder sector, and optional...


the minimum space occupied is then one sector.
the maximum file size is (4G-1)*512+(4G-1) bytes
i think it's enough for the 2008 year applications.
Post 09 Dec 2007, 15:24
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
i relaunch the discussion:

what are the mains functions used by a file system.
can you give me some lists of mostlly used functions.

it's because i never play with files in my codes and i want to know exactlly wich functions you like to call. and wich you dislike to call.

and more deeper, i want to know what you want to obtain by calling such a function.

thanks.
Post 18 Jan 2008, 03:01
View user's profile Send private message Visit poster's website Reply with quote
Hayden



Joined: 06 Oct 2005
Posts: 132
Hayden
Persuedo code: fn = filenum s = string i = integer x = object

fn = open(s,i) -- open file in text or binary mode
close(fn) -- close file
flush(fn) -- force o/s to flush cached file to disk
i = getc(fn) -- fetch char from cache/file
s = gets(fn,i) -- fetch sequence of 'n bytes from cache/file
putf(fn,x) -- put bytes into file

_________________
New User.. Hayden McKay.
Post 06 Feb 2008, 12:18
View user's profile Send private message Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
thanks a lot hayden.
Quote:
Pseudo code: fn = filenum s = string i = integer x = object

fn = open(s,i) -- open file in text or binary mode
close(fn) -- close file
flush(fn) -- force o/s to flush cached file to disk
i = getc(fn) -- fetch char from cache/file
s = gets(fn,i) -- fetch sequence of 'n bytes from cache/file
putf(fn,x) -- put bytes into file


so, only this is realy needed?
ok! let's play with asm coding of this:
Code:
...
filepath db 'usr1/video/dragon ball z - film 1 - à la poursuite de garlik.avi',0
...
file?:
.path dd filepath
.ptr dd ?
.size dd ?
.size2 dd ?
...
filelist:
.size dd @f-$-8
.ptr dd 0  ;to point to the current handle 
.num rd 20    ; limit the application to 20 files open
@@:
    

open file:
Code:
 mov edi,filelist  
 mov esi,file?
 call openf
...
openf:
.path = 0
.ptr = 4
.size = 8
.size2 = 12
...
 mov edi,[edi+4] ; extract the current file handle
 mov [edi+filelist.num],esi ... to link the file to the application?
 mov [esi+.ptr],ebx  ... linear location in ram?
 mov [esi+.size],ecx  ...in bytes
 mov [esi+.size2],edx ...in clusters ( 512, 4096, 1,etc...)bytes
 ret 
    

close file:
Code:
mov edi,
call closef
...
closef:
 mov d[edi],0 
 mov d[esi+.ptr],0  
 mov d[esi+.size],0  
 mov d[esi+.size2],0 
 ret 
    

flush file:
Code:
...
    


am i in the right way about the implementation?
Post 06 Feb 2008, 13:21
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
Hm, why emulate libc at the OS kernel level? Imho text/binary distinction has no place in an OS kernel, that should be up to the user binaries.

And what about the more important things like cache-control and async I/O? Smile
Post 06 Feb 2008, 14:26
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
it's not an emulation, just a way to support multitask functions.
execution context is not a stack, but an array in memory.

about the cache control and async I/O, i want it transparent.
and synchronised by the PIT IRQ0, to make it a bit more synchrone.

the file system services are hard to define without experience. i never play with files, and i'm a little agree in the fact that copying the standard service is not a good idea.
but how can i play with files in this file system?

fodder:
do you have an idea?
Post 06 Feb 2008, 14:40
View user's profile Send private message Visit poster's website Reply with quote
f0dder



Joined: 19 Feb 2004
Posts: 3170
Location: Denmark
f0dder
Ummm, where did "emulation" and "multitask" enter the picture?

Cache control and async I/O cannot, by definition, be transparent. Once a standard "read" call is done, the application expects data to be in the buffer.
Post 06 Feb 2008, 14:48
View user's profile Send private message Visit poster's website Reply with quote
edfed



Joined: 20 Feb 2006
Posts: 4242
Location: 2018
edfed
i've got a shocking idea...
why make a file system depending on granularity?
why not make a linear file system, whithout notion of SECTORS, CLUSTERS, etc...?

i was thinking about it cause managing a linear address space is easy, and managing a granular address space is boring.
this idea can simplify the implementation and the ram mirroring of this file system.
ram exactlly as disk, disk exactlly as ram.
one thing change, the minimal file occupation will become the minimal set of pointers + 1 bytes.
many little files will be writen on one unique sector, and the access to file system will be a linear access with 64bit pointers for drive > 4GB, 32bit pointers for drives > 16MB, 24bit pointers for the drives =< 16MB

no fat, only tables and structures.

and for services, only linear accesses to a linear address space, simple and efficient. no?

the drive will become a simple tree.

there, the paging capabilities of the pentuiums will be usefull, as all linear spaces of drives will be paged in ram.
Post 07 Feb 2008, 17:35
View user's profile Send private message Visit poster's website Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 17663
Location: In your JS exploiting you and your system
revolution
What happens when you want to append bytes to a file one at a time? How do you handle deletions? What happens to free space, how is it reused?

BTW: a FAT is "only tables and structures".

BTW2: RAM is allocated by the OS in 4kbyte sections, not 1 byte sections.
Post 07 Feb 2008, 17:45
View user's profile Send private message Visit poster's website Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3, 4  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2020, Tomasz Grysztar. Also on GitHub, YouTube, Twitter.

Website powered by rwasa.