flat assembler
Message board for the users of flat assembler.

flat assembler > Linux > my HeavyThing general purpose x86_64 linux library

Goto page Previous  1, 2, 3
Author
Thread Post new topic Reply to topic
redsock



Joined: 09 Oct 2009
Posts: 283
Location: Australia
sleepsleep wrote:
but how webops team could generate those files listing dynamically?
a few different ways... if it were me, I'd modify the webserver's assembly language hooks and make use of the https://2ton.com.au/library_as_html/dir.inc.html directory reading goods from the library to custom generate HTML/CSS/images for a given listing. (making sure to bake in some access controls/permissions).

Your other alternatives might be to use a FastCGI -indexfiles option and code the directory listing generator in your FastCGI/CGI language of choice... (Even other assembly language CGI environments, like @JohnFound's that runs https://fresh.flatassembler.net via rwasa)

Granted that isn't necessarily "out of the box" easy to do I suppose, haha... most of the other webservers I have played with either have templates that ship alongside the webserver, or the HTML/etc is all hardcoded/baked in ... and since I like the idea of rwasa being a standalone binary (without an install package/configuration script, lots of extra required files, etc), adding these sorts of "you are _definitely_ going to want to customise this feature, or that) didn't really go hand in hand.

Open to ideas/commentary though Smile

_________________
2 Ton Digital - https://2ton.com.au/
Post 15 Sep 2016, 02:47
View user's profile Send private message Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 7337
Location: ˛                              ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣ Posts: 6699
imo, a web server should has a flag whether to list files or no, i think, almost all web server came with this function, Smile

i suggest, it probably could be, if no -indexfiles flag during execute, will turn on listing mode,

not really sure how to integrate, but i am thinking using ls -l?, the representation could be [D] for directory, [F] files, [E] executable, without any images,?

maybe with a capped size (512 kb)? because the worse usually happened when listing linux packages, browser just hang.
Post 15 Sep 2016, 09:55
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3475
Location: Bulgaria
If I can find some time, I will write some example implementation of CGI and/or FastCGI directory lister for RWASA.
Post 15 Sep 2016, 11:21
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
sleepsleep



Joined: 05 Oct 2006
Posts: 7337
Location: ˛                              ⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣⁣ Posts: 6699
cross fingers, much appreciate,
Post 15 Sep 2016, 19:27
View user's profile Send private message Reply with quote
tthsqe



Joined: 20 May 2009
Posts: 714
Hello. Could you tell me the meaning of line 200 of vdso.inc?
Code:
calign .foundit: movzx eax, word [rbx+6] ; ?????????????????? mov rcx, [rbx+8] add rcx, [rsp+992] sub rcx, [rsp+1008] ; the address of our symbol mov [vdso_gettimeofday], rcx add rsp, 1024 pop r15 r14 r13 r12 rbx epilog
Post 19 Jan 2017, 02:43
View user's profile Send private message Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 283
Location: Australia
tthsqe wrote:
Hello. Could you tell me the meaning of line 200 of vdso.inc?
Lol good eye! When I wrote it initially, it was a hand-compile of a C++ version I have here somewhere, line 200 is a remnant that didn't get deleted after I got what I wanted Wink Next release of the library will have it commented out thusly. Is it the ELF64_ST_TYPE uint16? something like that from vague recollection. Anyway, you are correct in that it is unused in the HeavyThing implementation.

Cheers and thanks for that!

_________________
2 Ton Digital - https://2ton.com.au/
Post 19 Jan 2017, 06:45
View user's profile Send private message Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 283
Location: Australia
After a long hiatus, I released v1.20 today: https://2ton.com.au/HeavyThing-1.20.tar.gz

From the ChangeLog:
Code:
September 26th, 2017: v1.20 -- Business has been booming along this last year! Thanks for the -- continued support, it is very much appreciated. mimelike.inc: Fixed a bug dealing with external body handling related to rwasa -backpath handling for NodeJS. webserver.inc: Added -backpath related no-response-modify for NodeJS. dataseg_macros.inc: A huge shout-out to Tomasz Grysztar (author of fasm itself) for modifying the globals macro functionality such that it now correctly applies conditionally (all binaries prior to this release contained _all_ global variables, now only if their corresponding code is actually used). Note that this also upped our fasm memory requirements (262144 is no longer sufficient to compile HeavyThing programs). xmlparser.inc: Added pull-style UTF8, UTF16, UTF32 forward and backward fast tag parser. vector.inc: Added simple 64bit resizable "vector" functionality. xmlmemnode.inc: Added DOM-style XML support. string32.inc: Fixed return value bug in string$from_utf16 and incorrect length error. Added a simple placeholder/memcpy for to_utf32. sysinfo.inc: Got rid of the pathetic /proc/cpuinfo parsing and replaced with the correct sched_getaffinity syscall to determine CPU count. ht.inc: Added xmlparser.inc, vector.inc, xmlmemnode.inc to the includes section. * version update to v1.20

_________________
2 Ton Digital - https://2ton.com.au/
Post 26 Sep 2017, 02:13
View user's profile Send private message Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 283
Location: Australia
v1.22 released today, see the ChangeLog for full details.

Most notable in this release aside from bugfixes:

I now support UTF8 string literals in fasm source. Previously, my macro for string literals only widened the '...string...' to UTF32 (or optionally UTF16).

Now the macro correctly parses and validates the conversion of UTF8 -> UTF32/UTF16. See https://2ton.com.au/library_as_html/cleartext.inc.html for the macro itself.

Here is an example bit of code that makes use of the functionality:
Code:
include 'ht/ht_defaults.inc' include 'ht/ht.inc' public _start falign _start: call ht$init ; send our string to stdout (turns it back into UTF8) mov rdi, .utf8 call string$to_stdoutln ; get the second character from the string and output it as U+hexcode mov rdi, .lpad call string$to_stdout mov rdi, .utf8 mov esi, 1 call string$charat mov edi, eax ; second codepoint in string mov esi, 16 ; base 16 please call string$from_unsigned push rax mov rdi, rax call string$to_stdoutln pop rdi call heap$free ; get the # of bytes required for its UTF8 representation mov rdi, .utf8_bytes call string$to_stdout mov rdi, .utf8 call string$utf8_length mov edi, eax mov esi, 10 ; base 10 please call string$from_unsigned push rax mov rdi, rax call string$to_stdoutln pop rdi call heap$free ; get the string length mov rdi, .string_length call string$to_stdout mov rdi, .utf8 call string$length ; this isn't really necessary, qword [.utf8] also works mov edi, eax mov esi, 10 ; base 10 please call string$from_unsigned push rax mov rdi, rax call string$to_stdoutln pop rdi call heap$free mov eax, syscall_exit xor edi, edi syscall cleartext .utf8, '勝利のためのアセンブリ言語' cleartext .lpad, 'U+' cleartext .utf8_bytes, 'UTF8 bytes required: ' cleartext .string_length, 'UTF32 length: ' include 'ht/ht_data.inc'


And when that is run, its output looks like:
Code:
勝利のためのアセンブリ言語 U+5229 UTF8 bytes required: 39 UTF32 length: 13

_________________
2 Ton Digital - https://2ton.com.au/
Post 22 Jun 2018, 02:06
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3475
Location: Bulgaria
Hi, redsock.
Hope you and all your relatives are fine. Smile

I would like to discuss a little the buffering issues with rwasa server. This is virtually the only reason why I am using nginx on my domain and I badly want to migrate to rwasa.

The problem is that I am very often using SSE that needs the server to not buffer the FastCGI output, but to directly pass it to the client. This way the server sent events can reach the client instantly.

In nginx, this problem is solved IMHO in most elegant way, by defining a header "X-Accel-Buffering" - if the server receives this header from the FastCGI server and the value of the header is "no", it switches the buffering off and relays the answer directly to the client.

In Apache and the other servers I know, the solution is not so flexible - through the configuration files.

If you are too busy (and don't mind) I can try to implement this feature by myself, but will probably need some guidance in the source and the internal architecture of the project.
Post 22 Jun 2018, 05:38
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 283
Location: Australia
Thanks @JohnFound, can you send me an offending/example FastCGI program that does what you want? It will [likely] be much faster for me to implement the change you need than walk you through the rwasa/epoll designs.

Happy to help and at the moment have some free time on my hands.

Cheers
Post 22 Jun 2018, 12:17
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3475
Location: Bulgaria
redsock wrote:
Thanks @JohnFound, can you send me an offending/example FastCGI program that does what you want? It will [likely] be much faster for me to implement the change you need than walk you through the rwasa/epoll designs.

Happy to help and at the moment have some free time on my hands.

Cheers


Thanks. I don't have compact example right now, but have implemented debugging echo service installed on board.asm32.info. I have written it when reporting the same bug in the Apache HTTP2 module. Smile

For quick illustration of the problem use curl:

1. As it should be (the buffering switched off). The echo output arrives in an even stream:
Code:
curl https://board.asm32.info/0/\!echoevents/


2. As it is now in rwasa and other servers with buffering:
Code:
curl https://board.asm32.info/1/\!echoevents/


Hope this will help, but if not, I will try to create small stand alone program that to reproduce the problem.

Regards.

P.S. In the case 2, rwasa seems to never returns data until the request is fully processed by the FastCGI client. Nginx in this case at least sends the data on relatively small chunks.

_________________
Tox ID: 48C0321ADDB2FE5F644BB5E3D58B0D58C35E5BCBC81D7CD333633FEDF1047914A534256478D9
Post 22 Jun 2018, 12:59
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 283
Location: Australia
I spent this morning coding up the necessary changes, it works for my tests but more testing is required Smile Will PM you a link to a test binary most likely tomorrow after I test for backward compatibility (took a bit of effort to keep the old functionality there while also supporting the new way).

If you greenlight will do another formal release with the changes.

Cheers

_________________
2 Ton Digital - https://2ton.com.au/
Post 23 Jun 2018, 03:10
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3475
Location: Bulgaria
redsock wrote:
I spent this morning coding up the necessary changes, it works for my tests but more testing is required Smile Will PM you a link to a test binary most likely tomorrow after I test for backward compatibility (took a bit of effort to keep the old functionality there while also supporting the new way).

If you greenlight will do another formal release with the changes.

Cheers


Great as always! Did you made it through X-Accel-Buffering header or another way? I mean, do I need to change something to the back-end?

_________________
Tox ID: 48C0321ADDB2FE5F644BB5E3D58B0D58C35E5BCBC81D7CD333633FEDF1047914A534256478D9
Post 23 Jun 2018, 04:26
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 283
Location: Australia
The way I did it, there are three separate ways to "enable FastCGI direct mode" with rwasa:

1) Use the X-Accel-Buffering: no header
2) Use the text/event-stream Content-Type
3) Pass -fastcgi_direct to rwasa on the command line and _every_ FastCGI response goes directly.

If the FastCGI program does not specify size or its own chunked transfer encoding, for HTTP/1.1 responses rwasa will correctly add chunked transfer encoding to the FastCGI output, and otherwise leaves it alone.

Hopefully it will work with your event-stream stuff too Wink it works in my development sandbox, haha, stay tuned

_________________
2 Ton Digital - https://2ton.com.au/
Post 23 Jun 2018, 05:20
View user's profile Send private message Reply with quote
JohnFound



Joined: 16 Jun 2003
Posts: 3475
Location: Bulgaria
Brilliant! Smile
Post 23 Jun 2018, 05:31
View user's profile Send private message Visit poster's website ICQ Number Reply with quote
redsock



Joined: 09 Oct 2009
Posts: 283
Location: Australia
Thanks to JohnFound's request, FastCGI direct (unbuffered, unmodified) support has been added to rwasa. Per my previous comments, it can be enabled in several ways.

New functionality can be found in the v1.23 release Smile https://2ton.com.au/library_as_html/ChangeLog.html

Also, v1.21 for some reason was not announced here, that added several new features to sshtalk. Public/private room listing, buddylist toggle, etc. that generally make sshtalk far more useful.

Code:
$ ssh 2ton.com.au
to check it out Smile

Cheers as always

_________________
2 Ton Digital - https://2ton.com.au/
Post 24 Jun 2018, 23:37
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  
Goto page Previous  1, 2, 3

< Last Thread | Next Thread >

Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 2004-2018, Tomasz Grysztar.

Powered by rwasa.