Index
> Linux > my HeavyThing general purpose x86_64 linux libraryGoto page Previous 1, 2, 3, 4 Next |
| Author |
|
|
JohnFound 27 Mar 2016, 11:17
Hi, redsock. I have a question about configuring rwasa for working with my FastCGI script. I am recently playing with FastCGI forum inplementation in assembly language (Demo: http://board.asm32.info). It runs on Apache, but I though it would be great if we have entirely assembly language web solution and tried to make it work with rwasa. But can't configure it properly.
Usually, the web servers start the FastCGI processes and pass the socket where it have to listen in STDIN handler. Usually they spawn several processes according to the load. How I should configure rwasa to do so? |
|||
27 Mar 2016, 11:17 |
|
|
redsock 27 Mar 2016, 23:02
I use the "spawn-fcgi" package that is available with most distros.
Code: # spawn-fcgi -s /dev/shm/php.sock /usr/bin/php-cgi Code: PHP_FCGI_CHILDREN=20 php-cgi -e -b /dev/shm/php.sock Whether it is a unix socket (considerably better for localhost) or IPv4 doesn't really matter obviously, the fastcgi process is responsible for dealing with itself independently of the webserver. When I start rwasa with Code: # ./rwasa -bind 80 -sandbox /var/www/html -fastcgi .php /dev/shm/php.sock -indexfiles index.php,index.html -errsyslog -runas root -foreground As for passing every single request backward via FastCGI (an uncommon thing to do in a normal web enviro due to things like static resources, images, CSS, etc) but nevertheless: ~line 2050 of webserver.inc, if you add something like Code: jmp .fastcgi_doit 
If you have an example FastCGI handler you can post/send me that is spewing the 506's I'll take a closer look for you but since everything I have here (php and HHVM) both work I am unable to test further. Cheers |
|||
|
27 Mar 2016, 23:02 |
|
|
JohnFound 28 Mar 2016, 03:59
redsock wrote: I use the "spawn-fcgi" package that is available with most distros. Not so important. I made the script able to run properly and to create its own unix socket on "./engine.sock" (btw, using relative paths is another thing I would like in rwasa for testing purposes). Quote: As for passing every single request backward via FastCGI (an uncommon thing to do in a normal web enviro due to things like static resources, images, CSS, etc) but nevertheless: ~line 2050 of webserver.inc, if you add something like Well, if you want to run only language interpreters (like PHP) it is normal to pass only some kind of files to them. But in my case, the whole site engine is inside the script and all the content is generated from a database (sqlite). Quote: If you have an example FastCGI handler you can post/send me that is spewing the 506's I'll take a closer look for you but since everything I have here (php and HHVM) both work I am unable to test further. The whole system is open source. The repository is at http://asm32.info/fossil/repo/asmbb - use the branch RecursiveTemplate. But in order to compile it you will need the FreshLibDev branch of FreshLib from http://fresh.flatassembler.net/fossil/repo/fresh/index Here is some snapshot that should work: FreshLibDev.zip; AsmBB Set %lib% to point to unpacket FreshLibDev.zip "freshlib" directory and compile "source/index.asm" The script needs all requests from the server. Best regards. _________________ Tox ID: 48C0321ADDB2FE5F644BB5E3D58B0D58C35E5BCBC81D7CD333633FEDF1047914A534256478D9 |
|||
|
28 Mar 2016, 03:59 |
|
|
redsock 28 Mar 2016, 04:37
JohnFound wrote: ...Here is some snapshot that should work: Code: # lib=../../FreshLibDev/freshlib fasm source/index.asm flat assembler version 1.71.39 (16384 kilobytes memory) error: out of stack space. A quick strace reveals: Code: (...snip...) gettimeofday({1459139698, 509797}, NULL) = 0 open("index.asm", O_RDONLY) = 3 lseek(3, 0, SEEK_END) = 3034 lseek(3, 0, SEEK_SET) = 0 read(3, "; ______________________________"..., 3034) = 3034 close(3) = 0 open("../../FreshLibDev/freshlib/freshlib.inc", O_RDONLY) = 3 lseek(3, 0, SEEK_END) = 691 lseek(3, 0, SEEK_SET) = 0 read(3, "; ______________________________"..., 691) = 691 close(3) = 0 open("../../FreshLibDev/freshlib/macros/allmacros.inc", O_RDONLY) = 3 lseek(3, 0, SEEK_END) = 832 lseek(3, 0, SEEK_SET) = 0 read(3, "; ______________________________"..., 832) = 832 close(3) = 0 open("../../FreshLibDev/freshlib/macros/_uniquelists.inc", O_RDONLY) = 3 lseek(3, 0, SEEK_END) = 1509 lseek(3, 0, SEEK_SET) = 0 read(3, "; list - name of the list.\r\n; li"..., 1509) = 1509 close(3) = 0 open("../../FreshLibDev/freshlib/macros/_executable.inc", O_RDONLY) = 3 lseek(3, 0, SEEK_END) = 1133 lseek(3, 0, SEEK_SET) = 0 read(3, "; ______________________________"..., 1133) = 1133 close(3) = 0 open("../../FreshLibDev/freshlib/macros//_executable.inc", O_RDONLY) = 3 lseek(3, 0, SEEK_END) = 1133 lseek(3, 0, SEEK_SET) = 0 read(3, "; ______________________________"..., 1133) = 1133 close(3) = 0 open("../../FreshLibDev/freshlib/macros///_executable.inc", O_RDONLY) = 3 lseek(3, 0, SEEK_END) = 1133 lseek(3, 0, SEEK_SET) = 0 read(3, "; ______________________________"..., 1133) = 1133 close(3) = 0 open("../../FreshLibDev/freshlib/macros////_executable.inc", O_RDONLY) = 3 Cheers |
|||
|
28 Mar 2016, 04:37 |
|
|
JohnFound 28 Mar 2016, 05:30
Sorry, I forgot about the other needed env variable: "TargetOS=Linux". But on my computer it still can't be compiled with console FASM, giving "out of memory". Usually It needs 128MB to compile, but now it refuses with even much more. I should investigate it further...
|
|||
|
28 Mar 2016, 05:30 |
|
|
redsock 28 Mar 2016, 05:35
Ok so, got it all sorted out.
This is the diff output from my HeavyThing-1.16 version's webserver.inc... Code: 2090,2092d2089 < ; JohnFound mod: < jmp .fastcgi_doit < ; end JohnFound mod 2747d2743 < 2901,2911d2896 < ; JohnFound mod: < ; So, the other language-based C/C++ based FastCGI implementations do not send < ; a Status: header for no-error-condition (200 OK == no error) < ; Since JohnFound's version does indeed send a 200 OK Status header, this was < ; passing the error code 200 to the webservercfg$error handler, which indeed < ; correctly was reporting a 506 unimplemented ERROR CODE, since 200 is not an < ; error code. < ; to remedy this: < cmp rsi, 200 < je .do_checkcontenttype < ; end JohnFound mod If you add those lines and recompile rwasa, it will send ALL requests via the FastCGI handler, just make sure to pass it a bogus .endswith on the command line arguments (it still redirects to the first supplied fastcgi handler, but ignores the endswith with the above modification 3 liner. The longer section is the long-winded explanation of what rwasa was doing. Let me know if that works on your end or if you want me to upload the modified webserver.inc 
Cheers |
|||
|
28 Mar 2016, 05:35 |
|
|
redsock 28 Mar 2016, 05:43
Sorry here is a patch file for you instead, easier to read than diff...
Code: *** HeavyThing-1.16/webserver.inc 2016-01-28 10:29:09.320183337 +1000 --- webserver.inc 2016-03-28 15:30:22.617705132 +1000 *************** *** 2087,2092 **** --- 2087,2095 ---- cmp qword [rbx+_avlofs_next], 0 je .dofilesystem mov rbx, [rbx+_avlofs_next] + ; JohnFound mod: + jmp .fastcgi_doit + ; end JohnFound mod calign .fastcgiwalk: mov rdi, r15 *************** *** 2741,2746 **** --- 2744,2750 ---- mov rdx, rax call epoll$outbound ; if epoll$outbound _failed_ to connect, we get a zero in eax + test eax, eax jz .fastcgi_error ; we return -1 here so that the webserver layer does _nothing_ with our request *************** *** 2894,2899 **** --- 2898,2914 ---- je .do206 mov rdi, [rbx+webserver_config_ofs] pop rsi + ; JohnFound mod: + ; So, the other language-based C/C++ based FastCGI implementations do not send + ; a Status: header for no-error-condition (200 OK == no error) + ; Since JohnFound's version does indeed send a 200 OK Status header, this was + ; passing the error code 200 to the webservercfg$error handler, which indeed + ; correctly was reporting a 506 unimplemented ERROR CODE, since 200 is not an + ; error code. + ; to remedy this: + cmp rsi, 200 + je .do_checkcontenttype + ; end JohnFound mod call webservercfg$error mov rdi, r13 mov r13, rax |
|||
|
28 Mar 2016, 05:43 |
|
|
redsock 28 Mar 2016, 05:45
JohnFound wrote: Sorry, I forgot about the other needed env variable: "TargetOS=Linux". But on my computer it still can't be compiled with console FASM, giving "out of memory". Usually It needs 128MB to compile, but now it refuses with even much more. I should investigate it further... |
|||
|
28 Mar 2016, 05:45 |
|
|
JohnFound 28 Mar 2016, 05:56
Oh, I found it. It is naming problem (I am usually compiling it through WINE/Fresh IDE). Simply rename the file "freshlib/data/uConfig.asm" to "uconfig.asm".
Then the following command compiles it for me (but 3GB of needed memory is still very strange) and places the script where it has to be: Code: TargetOS="Linux" lib="../../FreshLibDev/freshlib" fasm -m 3000000 index.asm ../www/engine In the directory www there are configuration files for apache and lighttpd I use for testing. If you run "engine" in "www" it will create "engine.sock" and will wait on it. |
|||
|
28 Mar 2016, 05:56 |
|
|
JohnFound 28 Mar 2016, 06:11
@redsock - white I typed, you manage to post a solution. Thank I will try it.
|
|||
|
28 Mar 2016, 06:11 |
|
|
JohnFound 28 Mar 2016, 07:51
So, with the patched version of rwasa and with some bugfixes in my code
 it works. Below is a screenshot from the AsmBB running on rwasa. 
But, redsock, why not to include these changes in the official release? I mean not permanent redirection of course, but syntax like the following: Code: rwasa -fastcgi /some_path <sock address> Where all requests to /some_path/* are considered FastCGI. (and if you set "/" it means the all requests to this site are considered FastCGI). This option is available for all web servers I know. Also, all web servers I know allow "Status: 200 OK" to be sent. Also, notice the script running problem. Now I can run the FastCGI script separately, because I am on my own compute. But imagine it was on shared hosting where I can't just start programs on the server. Best regards.
_________________ Tox ID: 48C0321ADDB2FE5F644BB5E3D58B0D58C35E5BCBC81D7CD333633FEDF1047914A534256478D9 |
||||||||||
|
28 Mar 2016, 07:51 |
|
|||||||||
|
redsock 28 Mar 2016, 21:07
JohnFound wrote: ...But, redsock, why not to include these changes in the official release? I mean not permanent redirection of course, but syntax like the following: There are a few items left outstanding before 1.17 comes out but when I get to those, will be sure to include these updates for rwasa.
How does process management for FastCGI backends work with Apache and/or nginx? Fork/exec at startup is certainly doable (even fork/exec during worker process runtime, though more messy)... I have not seen FastCGI coupled to webservers this way in production environments before. Anyway, glad the patch worked for you. Cheers! |
|||
|
28 Mar 2016, 21:07 |
|
|
JohnFound 29 Mar 2016, 03:54
redsock wrote: How does process management for FastCGI backends work with Apache and/or nginx? Fork/exec at startup is certainly doable (even fork/exec during worker process runtime, though more messy)... I have not seen FastCGI coupled to webservers this way in production environments before. In both Apache and lighttpd, the FastCGI processes are started depending on the load. If the response from the process start to delay more than acceptable, new one is started and the requests are sent there, until the previous processes finish their work. If the process hangs it is terminated. If there are no requests for long time, even all child processes can be terminated. You didn't see such use, because usually FastCGI is used only for executing language interpreters where the interpreter is started by the system administrator. This is possible of course only if you own the server. It is not an option for shared hosting web hosting. _________________ Tox ID: 48C0321ADDB2FE5F644BB5E3D58B0D58C35E5BCBC81D7CD333633FEDF1047914A534256478D9 |
|||
|
29 Mar 2016, 03:54 |
|
|
redsock 10 Aug 2016, 04:42
Hey all,
In donating the virtual server for Tomasz and the community here, and in part due to the above discussion with @JohnFound re: rwasa (and that it is now running this board), CGI, FastCGI, et al, I got motivated and did the v1.17 release of my library today. Check out the ChangeLog for the nitty-gritty details: https://2ton.com.au/library_as_html/ChangeLog.html Added lots of new functionality to the library as well as a decent handful of bugfixes. Let me know if you have any issues. Cheers! |
|||
|
10 Aug 2016, 04:42 |
|
|
redsock 30 Aug 2016, 04:48
Another update today... v1.18 released... small Fossil related bugfix for @JohnFound and https://fresh.flatassembler.net in dealing with compressed FastCGI redirect responses...
And I included a simple but fast http://dev.maxmind.com/geoip/geoip2/geolite2/ parser and lookup utility (as HTML: https://2ton.com.au/library_as_html/maxmind.inc.html) Cheers! |
|||
|
30 Aug 2016, 04:48 |
|
|
redsock 13 Sep 2016, 00:06
Another small update today... v1.19 released (https://2ton.com.au/HeavyThing/). A bug I introduced in 1.18 for the way that rwasa was dealing with FastCGI redirection combined with Set-Cookie got fixed up (same one that caused the persistent logins here on the board to malfunction there for a short while, though the board got the 1.19 rwasa straight away I didn't do a new library release til today).
Also included are a few new library examples: TCP outbound example in simple_socket for both IPv4 and hostname-based epoll sockets. A https://download.libsodium.org/doc/ HeavyThing example for crypto_box_easy and crypto_box_open_easy. And last but not least, a SHA3/256 Keccak example similar to the SHA256 one already included. Cheers as usual! |
|||
|
13 Sep 2016, 00:06 |
|
|
sleepsleep 13 Sep 2016, 15:52
thanks redsock,
/me started using rwasa 1.19 question - is it possible to allow it to lists files in sandbox directory? - is it possible to allow it prints client GET, POST etc, when running -foreground? eg Code: [Tue Sep 13 2016 23:47:06 GMT+0000 (GMT)] "GET /" "Mozilla/5.0 (Windows NT 6.1; WOW128; rv:37.0) Gecko/20100101 Firefox/32.0" [Tue Sep 13 2016 23:47:06 GMT+0000 (GMT)] "GET /css/phonon.css" "Mozilla/5.0 (Windows NT 6.1; WOW128; rv:37.0) Gecko/20100101 Firefox/32.0" [Tue Sep 13 2016 23:47:06 GMT+0000 (GMT)] "GET /js/phonon.js" "Mozilla/5.0 (Windows NT 6.1; WOW128; rv:37.0) Gecko/20100101 Firefox/32.0" |
|||
|
13 Sep 2016, 15:52 |
|
|
redsock 13 Sep 2016, 20:56
sleepsleep wrote: thanks redsock, By "list files in sandbox directory" I assume you mean old-school directory listings when requests arrive for /some/directory/ yeah? I debated originally when I designed rwasa whether or not to include such functionality, and concluded then that since there are HTML, CSS, images, etc required for same, those should be left in the hands of the webops teams. As such, you'd have to use some kind of -indexfiles option to specify your own directory listing handler (which could then reasonably produce a styled directory listing w/ HTML, CSS, image links, etc). |
|||
|
13 Sep 2016, 20:56 |
|
|
sleepsleep 14 Sep 2016, 04:20
thanks redsock,
i reassemble back and got http requests now, redsock wrote:
yes, redsock wrote: those should be left in the hands of the webops teams but how webops team could generate those files listing dynamically? |
|||
|
14 Sep 2016, 04:20 |
|
| Goto page Previous 1, 2, 3, 4 Next < Last Thread | Next Thread > |
Forum Rules:
|