flat assembler
Message board for the users of flat assembler.

Index > Projects and Ideas > A standard C library

Author
Thread Post new topic Reply to topic
gens



Joined: 18 Feb 2013
Posts: 161
gens 15 Oct 2014, 18:58
had this idea a while ago
i made a couple string functions and a general "framework" for making a shared library and a static library

aim was to have different functions for generic, SSE1234, AVX, and so on builds
with performance as a higher priority then size

i went for 64bit on linux, but it shouldn't be hard to make 32bit and windows, bsd and such versions
(maybe even kolibrios:))


is there any interest in such a thing ?


update: a crude outline
http://speedy.sh/JzeaS/libasm.tar.gz
notes:
no proper test framework (just a general recommendation)
the .asmm functions probably don't work


Last edited by gens on 19 Oct 2014, 20:36; edited 3 times in total
Post 15 Oct 2014, 18:58
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 15 Oct 2014, 19:27
You say a C library. Does that mean it is intended to be called by C? Or to replace an existing C library? Or it is written in C?
Post 15 Oct 2014, 19:27
View user's profile Send private message Visit poster's website Reply with quote
gens



Joined: 18 Feb 2013
Posts: 161
gens 15 Oct 2014, 20:05
standard C library, as defined by POSIX
functions like fread(), sin(), malloc(), memcpy() and so on
written in FASM ofc Smile

funny that you mention replacing an existing one, as there are some 3 of them on linux
well.. there are more, but i know of 2-3 that are complete
GNU's glibc being the most popular one
Post 15 Oct 2014, 20:05
View user's profile Send private message Reply with quote
gens



Joined: 18 Feb 2013
Posts: 161
gens 15 Oct 2014, 20:16
when i thought about it i came to think that stdio(.h) part would be a bit complicated, mostly thanks to threading
string part would be fairly easy
math would be tricky as it has to comply with the standards of precision
locale would be boring
stdlib part would be a bit fun, mostly thanks to malloc

i think that's almost everything
a couple functions and definitions here and there

oh ye, a linker/loader
and the stuff put in by the compiler
Post 15 Oct 2014, 20:16
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 16 Oct 2014, 02:23
So why are you calling it a C library? I'm still confused where C fits in here, Is it your intention that programs written in C will call/link/bind to it?

Also, what is your intended methodology for measuring performance? Do you intent to make special versions of each function for different CPU/cache/RAM combinations?
Post 16 Oct 2014, 02:23
View user's profile Send private message Visit poster's website Reply with quote
gens



Joined: 18 Feb 2013
Posts: 161
gens 16 Oct 2014, 03:17
i mix "standard C library" with "C standard library" in my head
it's a C standard library Smile

for testing most of the stuff a small C (or fasm) program that benchmarks using clock_gettime with CLOCK_MONOTONIC (or CLOCK_REALTIME)
on a quiet system without "turbo boost" and in "performance" mode, ofc
small data sets or one-shot functions in a loop to get a bit more precision, if needed

cache, hmmm
i thought about it when writing memcpy for SSE2
normal movaps slows down a lot when the cache is full so it is needed to switch to movntps for big chunks of data
how fast to switch, well, depends on the size of the cache
could be a "variable" set at assemble time using the sed program

something like a memcpy test would check with big and small copies (or a scale like 1byte-100MB),
aligned dest-src, 1 byte unaligned, 2 byte unaligned, etc

functions per cpu capability
currently i have memcpy_generic.asm and memcpy_SEE.asm, so cpu extensions marked in file name
AMD-INTEL or by architectures only if it proves to be needed (afaik intel can be slower on unaligned things and page boundaries)

bdw, test by overwriting the installed libc functions using LD_PRELOAD


in short
make a function
if there is no, any that conforms with POSIX is fine
make a test, if there isn't any (or make a better one)
if it is faster then the other one, copy the other one to an archive and replace it
(id be doing most of the bookkeeping)

at one time il make a full sweep test using the archive to quick test on different cpu's
i'm sure there will be problems with that, like a standard "results" output


i was thinking of doing it casually (since i'm lazy:), without a time frame
no strings attached


am sure i forgot to say something..

PS in the best scenario every function (and C macro) would have one .c file that does correctness tests (one by one, printing result),
then performance tests (if required for that function)
Post 16 Oct 2014, 03:17
View user's profile Send private message Reply with quote
revolution
When all else fails, read the source


Joined: 24 Aug 2004
Posts: 20299
Location: In your JS exploiting you and your system
revolution 16 Oct 2014, 07:07
gens wrote:
PS in the best scenario every function (and C macro) would have one .c file that does correctness tests
Agree (although I think the filename .c is misleading). But this should not be "best scenario". Instead it should be "a requirement of all scenarios". Wink In many cases the test code will be larger than the function it tests. This is normal and should be encouraged. Having full confidence in the correct operation of one's of code is a wonderful thing.
Post 16 Oct 2014, 07:07
View user's profile Send private message Visit poster's website Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 12737
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 0010456
sleepsleep 16 Oct 2014, 10:08
gens wrote:
had this idea a while ago
i made a couple string functions and a general "framework" for making a shared library and a static library

aim was to have different functions for generic, SSE1234, AVX, and so on builds
with performance as a higher priority then size

i went for 64bit on linux, but it shouldn't be hard to make 32bit and windows, bsd and such versions
(maybe even kolibrios:))


is there any interest in such a thing ?


yeah, i am interested to have it running in raspberry pi
Post 16 Oct 2014, 10:08
View user's profile Send private message Reply with quote
gens



Joined: 18 Feb 2013
Posts: 161
gens 17 Oct 2014, 00:20
revolution wrote:
Agree (although I think the filename .c is misleading). But this should not be "best scenario". Instead it should be "a requirement of all scenarios". Wink In many cases the test code will be larger than the function it tests. This is normal and should be encouraged. Having full confidence in the correct operation of one's of code is a wonderful thing.


sure
decided to have a shell script for testing, for example ./test/string/memcpy/test.sh
and another one to make it easier, in the form of ./test.sh memcpy
or something like that, as guidelines


but i don't wanna force anyone to write these damn things Smile
no coding standard either (except that it has to be one file)
so that experience, time and such things matter less

i updated the build script to have target instruction sets (yes/no variables) and a nicer output
and added basic readme's (main one remaining)

not much demand, but il upload it all anyway in a day or two
Post 17 Oct 2014, 00:20
View user's profile Send private message Reply with quote
gens



Joined: 18 Feb 2013
Posts: 161
gens 19 Oct 2014, 20:33
updated the original post
still don't have everything i want, but its not far
Post 19 Oct 2014, 20:33
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.