flat assembler
Message board for the users of flat assembler.

Index > Linux > my HeavyThing general purpose x86_64 linux library

Goto page Previous  1, 2, 3, 4  Next
Author
Thread Post new topic Reply to topic
redsock



Joined: 09 Oct 2009
Posts: 435
Location: Australia
redsock 15 Sep 2016, 02:47
sleepsleep wrote:
but how webops team could generate those files listing dynamically?
a few different ways... if it were me, I'd modify the webserver's assembly language hooks and make use of the https://2ton.com.au/library_as_html/dir.inc.html directory reading goods from the library to custom generate HTML/CSS/images for a given listing. (making sure to bake in some access controls/permissions).

Your other alternatives might be to use a FastCGI -indexfiles option and code the directory listing generator in your FastCGI/CGI language of choice... (Even other assembly language CGI environments, like @JohnFound's that runs https://fresh.flatassembler.net via rwasa)

Granted that isn't necessarily "out of the box" easy to do I suppose, haha... most of the other webservers I have played with either have templates that ship alongside the webserver, or the HTML/etc is all hardcoded/baked in ... and since I like the idea of rwasa being a standalone binary (without an install package/configuration script, lots of extra required files, etc), adding these sorts of "you are _definitely_ going to want to customise this feature, or that) didn't really go hand in hand.

Open to ideas/commentary though Smile

_________________
2 Ton Digital - https://2ton.com.au/
Post 15 Sep 2016, 02:47
View user's profile Send private message Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 13047
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 0010456
sleepsleep 15 Sep 2016, 09:55
imo, a web server should has a flag whether to list files or no, i think, almost all web server came with this function, Smile

i suggest, it probably could be, if no -indexfiles flag during execute, will turn on listing mode,

not really sure how to integrate, but i am thinking using ls -l?, the representation could be [D] for directory, [F] files, [E] executable, without any images,?

maybe with a capped size (512 kb)? because the worse usually happened when listing linux packages, browser just hang.
Post 15 Sep 2016, 09:55
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 15 Sep 2016, 11:21
If I can find some time, I will write some example implementation of CGI and/or FastCGI directory lister for RWASA.
Post 15 Sep 2016, 11:21
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 13047
Location: ˛                             ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣Posts: 0010456
sleepsleep 15 Sep 2016, 19:27
cross fingers, much appreciate,
Post 15 Sep 2016, 19:27
View user's profile Send private message Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 767
tthsqe 19 Jan 2017, 02:43
Hello. Could you tell me the meaning of line 200 of vdso.inc?
Code:
calign
.foundit:
        movzx   eax, word [rbx+6]   ; ??????????????????
        mov     rcx, [rbx+8]
        add     rcx, [rsp+992]  
        sub     rcx, [rsp+1008] ; the address of our symbol

        mov     [vdso_gettimeofday], rcx

        add     rsp, 1024
        pop     r15 r14 r13 r12 rbx
        epilog    
Post 19 Jan 2017, 02:43
View user's profile Send private message Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 435
Location: Australia
redsock 19 Jan 2017, 06:45
tthsqe wrote:
Hello. Could you tell me the meaning of line 200 of vdso.inc?
Lol good eye! When I wrote it initially, it was a hand-compile of a C++ version I have here somewhere, line 200 is a remnant that didn't get deleted after I got what I wanted Wink Next release of the library will have it commented out thusly. Is it the ELF64_ST_TYPE uint16? something like that from vague recollection. Anyway, you are correct in that it is unused in the HeavyThing implementation.

Cheers and thanks for that!

_________________
2 Ton Digital - https://2ton.com.au/
Post 19 Jan 2017, 06:45
View user's profile Send private message Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 435
Location: Australia
redsock 26 Sep 2017, 02:13
After a long hiatus, I released v1.20 today: https://2ton.com.au/HeavyThing-1.20.tar.gz

From the ChangeLog:
Code:
September 26th, 2017: v1.20
        -- Business has been booming along this last year! Thanks for the
        -- continued support, it is very much appreciated.
        mimelike.inc:
                Fixed a bug dealing with external body handling related to
                rwasa -backpath handling for NodeJS.
        webserver.inc:
                Added -backpath related no-response-modify for NodeJS.
        dataseg_macros.inc:
                A huge shout-out to Tomasz Grysztar (author of fasm itself)
                for modifying the globals macro functionality such that it
                now correctly applies conditionally (all binaries prior to
                this release contained _all_ global variables, now only if
                their corresponding code is actually used). Note that this
                also upped our fasm memory requirements (262144 is no
                longer sufficient to compile HeavyThing programs).
        xmlparser.inc:
                Added pull-style UTF8, UTF16, UTF32 forward and backward
                fast tag parser.
        vector.inc:
                Added simple 64bit resizable "vector" functionality.
        xmlmemnode.inc:
                Added DOM-style XML support.
        string32.inc:
                Fixed return value bug in string$from_utf16 and incorrect
                length error.
                Added a simple placeholder/memcpy for to_utf32.
        sysinfo.inc:
                Got rid of the pathetic /proc/cpuinfo parsing and replaced
                with the correct sched_getaffinity syscall to determine
                CPU count.
        ht.inc:
                Added xmlparser.inc, vector.inc, xmlmemnode.inc to the
                includes section.
        * version update to v1.20
    

_________________
2 Ton Digital - https://2ton.com.au/
Post 26 Sep 2017, 02:13
View user's profile Send private message Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 435
Location: Australia
redsock 22 Jun 2018, 02:06
v1.22 released today, see the ChangeLog for full details.

Most notable in this release aside from bugfixes:

I now support UTF8 string literals in fasm source. Previously, my macro for string literals only widened the '...string...' to UTF32 (or optionally UTF16).

Now the macro correctly parses and validates the conversion of UTF8 -> UTF32/UTF16. See https://2ton.com.au/library_as_html/cleartext.inc.html for the macro itself.

Here is an example bit of code that makes use of the functionality:
Code:
include 'ht/ht_defaults.inc'
include 'ht/ht.inc'

public _start
falign
_start:
        call    ht$init

        ; send our string to stdout (turns it back into UTF8)
        mov     rdi, .utf8
        call    string$to_stdoutln

        ; get the second character from the string and output it as U+hexcode
        mov     rdi, .lpad
        call    string$to_stdout

        mov     rdi, .utf8
        mov     esi, 1
        call    string$charat

        mov     edi, eax                ; second codepoint in string
        mov     esi, 16                 ; base 16 please
        call    string$from_unsigned
        push    rax
        mov     rdi, rax
        call    string$to_stdoutln
        pop     rdi
        call    heap$free

        ; get the # of bytes required for its UTF8 representation
        mov     rdi, .utf8_bytes
        call    string$to_stdout
        mov     rdi, .utf8
        call    string$utf8_length
        mov     edi, eax
        mov     esi, 10                 ; base 10 please
        call    string$from_unsigned
        push    rax
        mov     rdi, rax
        call    string$to_stdoutln
        pop     rdi
        call    heap$free

        ; get the string length
        mov     rdi, .string_length
        call    string$to_stdout
        mov     rdi, .utf8
        call    string$length           ; this isn't really necessary, qword [.utf8] also works
        mov     edi, eax
        mov     esi, 10                 ; base 10 please
        call    string$from_unsigned
        push    rax
        mov     rdi, rax
        call    string$to_stdoutln
        pop     rdi
        call    heap$free

        mov     eax, syscall_exit
        xor     edi, edi
        syscall
cleartext .utf8, '勝利のためのアセンブリ言語'
cleartext .lpad, 'U+'
cleartext .utf8_bytes, 'UTF8 bytes required: '
cleartext .string_length, 'UTF32 length: '

include 'ht/ht_data.inc'    


And when that is run, its output looks like:
Code:
勝利のためのアセンブリ言語
U+5229
UTF8 bytes required: 39
UTF32 length: 13    

_________________
2 Ton Digital - https://2ton.com.au/
Post 22 Jun 2018, 02:06
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 22 Jun 2018, 05:38
Hi, redsock.
Hope you and all your relatives are fine. Smile

I would like to discuss a little the buffering issues with rwasa server. This is virtually the only reason why I am using nginx on my domain and I badly want to migrate to rwasa.

The problem is that I am very often using SSE that needs the server to not buffer the FastCGI output, but to directly pass it to the client. This way the server sent events can reach the client instantly.

In nginx, this problem is solved IMHO in most elegant way, by defining a header "X-Accel-Buffering" - if the server receives this header from the FastCGI server and the value of the header is "no", it switches the buffering off and relays the answer directly to the client.

In Apache and the other servers I know, the solution is not so flexible - through the configuration files.

If you are too busy (and don't mind) I can try to implement this feature by myself, but will probably need some guidance in the source and the internal architecture of the project.
Post 22 Jun 2018, 05:38
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 435
Location: Australia
redsock 22 Jun 2018, 12:17
Thanks @JohnFound, can you send me an offending/example FastCGI program that does what you want? It will [likely] be much faster for me to implement the change you need than walk you through the rwasa/epoll designs.

Happy to help and at the moment have some free time on my hands.

Cheers
Post 22 Jun 2018, 12:17
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 22 Jun 2018, 12:59
redsock wrote:
Thanks @JohnFound, can you send me an offending/example FastCGI program that does what you want? It will [likely] be much faster for me to implement the change you need than walk you through the rwasa/epoll designs.

Happy to help and at the moment have some free time on my hands.

Cheers


Thanks. I don't have compact example right now, but have implemented debugging echo service installed on board.asm32.info. I have written it when reporting the same bug in the Apache HTTP2 module. Smile

For quick illustration of the problem use curl:

1. As it should be (the buffering switched off). The echo output arrives in an even stream:
Code:
curl https://board.asm32.info/0/\!echoevents/    


2. As it is now in rwasa and other servers with buffering:
Code:
curl https://board.asm32.info/1/\!echoevents/    


Hope this will help, but if not, I will try to create small stand alone program that to reproduce the problem.

Regards.

P.S. In the case 2, rwasa seems to never returns data until the request is fully processed by the FastCGI client. Nginx in this case at least sends the data on relatively small chunks.

_________________
Tox ID: 48C0321ADDB2FE5F644BB5E3D58B0D58C35E5BCBC81D7CD333633FEDF1047914A534256478D9
Post 22 Jun 2018, 12:59
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 435
Location: Australia
redsock 23 Jun 2018, 03:10
I spent this morning coding up the necessary changes, it works for my tests but more testing is required Smile Will PM you a link to a test binary most likely tomorrow after I test for backward compatibility (took a bit of effort to keep the old functionality there while also supporting the new way).

If you greenlight will do another formal release with the changes.

Cheers

_________________
2 Ton Digital - https://2ton.com.au/
Post 23 Jun 2018, 03:10
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 23 Jun 2018, 04:26
redsock wrote:
I spent this morning coding up the necessary changes, it works for my tests but more testing is required Smile Will PM you a link to a test binary most likely tomorrow after I test for backward compatibility (took a bit of effort to keep the old functionality there while also supporting the new way).

If you greenlight will do another formal release with the changes.

Cheers


Great as always! Did you made it through X-Accel-Buffering header or another way? I mean, do I need to change something to the back-end?

_________________
Tox ID: 48C0321ADDB2FE5F644BB5E3D58B0D58C35E5BCBC81D7CD333633FEDF1047914A534256478D9
Post 23 Jun 2018, 04:26
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 435
Location: Australia
redsock 23 Jun 2018, 05:20
The way I did it, there are three separate ways to "enable FastCGI direct mode" with rwasa:

1) Use the X-Accel-Buffering: no header
2) Use the text/event-stream Content-Type
3) Pass -fastcgi_direct to rwasa on the command line and _every_ FastCGI response goes directly.

If the FastCGI program does not specify size or its own chunked transfer encoding, for HTTP/1.1 responses rwasa will correctly add chunked transfer encoding to the FastCGI output, and otherwise leaves it alone.

Hopefully it will work with your event-stream stuff too Wink it works in my development sandbox, haha, stay tuned

_________________
2 Ton Digital - https://2ton.com.au/
Post 23 Jun 2018, 05:20
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3499
Location: Bulgaria
JohnFound 23 Jun 2018, 05:31
Brilliant! Smile
Post 23 Jun 2018, 05:31
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 435
Location: Australia
redsock 24 Jun 2018, 23:37
Thanks to JohnFound's request, FastCGI direct (unbuffered, unmodified) support has been added to rwasa. Per my previous comments, it can be enabled in several ways.

New functionality can be found in the v1.23 release Smile https://2ton.com.au/library_as_html/ChangeLog.html

Also, v1.21 for some reason was not announced here, that added several new features to sshtalk. Public/private room listing, buddylist toggle, etc. that generally make sshtalk far more useful.

Code:
$ ssh 2ton.com.au    
to check it out Smile

Cheers as always

_________________
2 Ton Digital - https://2ton.com.au/
Post 24 Jun 2018, 23:37
View user's profile Send private message Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 435
Location: Australia
redsock 21 Oct 2018, 23:05
v1.24 released today, see the ChangeLog for full details.

Notable things aside from minor bugfixes:

Based on some work a while back with Tomasz, added a 'fasm preprocessor' to ease the compilation times for HeavyThing programs. Included in the library is now an optional
Code:
$ ./assemble_all_faster.sh    
that uses the preprocessor to build all of the library and examples. (The original assemble_all.sh is also still present).

Added bas64url support to string handling and latin1 functions, necessary e.g. for the ACME Protocol

_________________
2 Ton Digital - https://2ton.com.au/
Post 21 Oct 2018, 23:05
View user's profile Send private message Reply with quote
ganuonglachanh



Joined: 29 Jul 2019
Posts: 4
ganuonglachanh 29 Jul 2019, 00:42
Hi redsock

Thanks for amazing rwasa server, could you please add support for:

1. url rewrite
2. http/2 (https://tools.ietf.org/html/rfc7540)
3. brotli compression (https://github.com/google/brotli)

Very Happy
Post 29 Jul 2019, 00:42
View user's profile Send private message Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 435
Location: Australia
redsock 29 Jul 2019, 00:46
You're welcome!

I have a nearly-finished rwasa2 sitting here that implements the full compliment of HTTP/2 ... time permitting I hope to release something later this year.

As far as URL rewriting is concerned, this can already be done via the webhook in rwasa (though it is not plainly obvious). Let me know if you'd like some help with code examples for rewriting.

I have added brotli compression to my longer-term "Someday when I'm bored..." list Smile

Cheers!

_________________
2 Ton Digital - https://2ton.com.au/
Post 29 Jul 2019, 00:46
View user's profile Send private message Reply with quote
ganuonglachanh



Joined: 29 Jul 2019
Posts: 4
ganuonglachanh 29 Jul 2019, 01:13
redsock wrote:
You're welcome!

I have a nearly-finished rwasa2 sitting here that implements the full compliment of HTTP/2 ... time permitting I hope to release something later this year.

As far as URL rewriting is concerned, this can already be done via the webhook in rwasa (though it is not plainly obvious). Let me know if you'd like some help with code examples for rewriting.

I have added brotli compression to my longer-term "Someday when I'm bored..." list Smile

Cheers!


Hi redsock

Glad to hear that the support for HTTP/2 is comming Laughing

For the URL rewriting, could you please add some example for wordpress (most famous PHP app) url rewriting? Rolling Eyes

Best wished Razz
Post 29 Jul 2019, 01:13
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3, 4  Next

< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2025, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.