flat assembler
Message board for the users of flat assembler.

Index > Projects and Ideas > [IDEA] Commandline download tool

Author
Thread Post new topic Reply to topic
OzzY



Joined: 19 Sep 2003
Posts: 1029
Location: Everywhere
OzzY 14 Feb 2008, 19:54
Do you want it?
I already made it in Pelles C. I'll make an optimized version in ASM.

Usage:
Create a batch file with your downloads:
Quote:

@echo off
echo "Downloading the files..."
quickget http://url1.com file1.ext
quickget http://url2.com file2.ext
quickget http://url3.com file3.ext
echo "Done!"


You let it downloading all over the night with low memory usage and fast downloading.
Maybe I can implement download resume, by using InternetSetFilePointer. Wink

Do you like this idea?
IMHO, why do you need a GUI for letting the computer download large files all over the night? It's just waste of memory. Very Happy
Post 14 Feb 2008, 19:54
View user's profile Send private message Reply with quote
wisepenguin



Joined: 30 Mar 2005
Posts: 129
wisepenguin 14 Feb 2008, 19:58
ill try it ozzy, for sure.

do good work, have fun.
Post 14 Feb 2008, 19:58
View user's profile Send private message Reply with quote
wisepenguin



Joined: 30 Mar 2005
Posts: 129
wisepenguin 14 Feb 2008, 19:59
oh, resuming would be nice Smile
Post 14 Feb 2008, 19:59
View user's profile Send private message Reply with quote
OzzY



Joined: 19 Sep 2003
Posts: 1029
Location: Everywhere
OzzY 14 Feb 2008, 20:00
Hi wisepenguin!
I did it in C and works. Here's the output:
Quote:

C:\>quickget http://www.smorgasbordet.com/pellesc/500/setup.exe pellesc.exe
Starting download...
8711648 of 8711648 bytes downloded from http://www.smorgasbordet.com/pellesc/500/setup.exe.

It downloaded Pelles C without error.

I'll port it to FASM. Very Happy
Post 14 Feb 2008, 20:00
View user's profile Send private message Reply with quote
wisepenguin



Joined: 30 Mar 2005
Posts: 129
wisepenguin 14 Feb 2008, 20:29
good man, C or asm would be fine.

are you planning to add cancelling/pausing/resuming downloads ?
Post 14 Feb 2008, 20:29
View user's profile Send private message Reply with quote
OzzY



Joined: 19 Sep 2003
Posts: 1029
Location: Everywhere
OzzY 14 Feb 2008, 21:08
Yes, I'll try to implement resume with InternetSetFilePointer.
Cancelling and pausing you do with Ctrl+C at the console. Razz

I'll also implement in later versions: Shutdown computer after download finishes.

Currently I'm stuck in getting the parameters from commandline. In C it's easy (argc and argv). I know GetCommandLineA, but still needs a parser.
Post 14 Feb 2008, 21:08
View user's profile Send private message Reply with quote
wisepenguin



Joined: 30 Mar 2005
Posts: 129
wisepenguin 14 Feb 2008, 21:15
thats what we like to hear! - well, that and "i dont have a headache, dear"
Post 14 Feb 2008, 21:15
View user's profile Send private message Reply with quote
OzzY



Joined: 19 Sep 2003
Posts: 1029
Location: Everywhere
OzzY 14 Feb 2008, 21:43
Here is the incomplete C version:
Code:
#include <stdio.h>
#include <windows.h>
#include <wininet.h> // For HTTP functions. Windows-only.

void show_usage(void);
int download_file(char *the_url, char *file_name);

int main(int argc, char *argv[])
{
      if( argc < 3 )
   {
              show_usage();
               return 0;
   }
      else
        {
              if(download_file(argv[1], argv[2]) == -1)
           {
                      printf("Could not download file! Internal error!");
                       return -1;
          }
      }
    return 0;
}

void show_usage(void)
{
    // Show copyright and usage information
     puts("QuickGet - Commandline downloader tool v0.1");
      puts("Written By Frederico Wagner");
      puts("");
 puts("Usage: quickget url file_name");
        puts("Example: quickget http://www.someurl.com/myfile.zip myfile.zip");
   puts("");
 puts("Please call the program with the right parameters from a command prompt. Exiting...");
      Sleep(2500);
}

int download_file(char *the_url, char *file_name)
{
   // Let's get the URL handle
        HINTERNET hInternet = InternetOpen("QuickGet", 0, 0, 0, 0);           // Get internet handle
      if(hInternet == NULL)
               return -1;
  HINTERNET hURL = InternetOpenUrl(hInternet, the_url, 0, 0, 0, 0);       // Open the URL
     if(hURL ==  NULL)
           return -1;

      DWORD file_size = InternetSetFilePointer(hURL, 0, 0, FILE_END, 0);      // Get file size
    if( file_size == -1 )
               return -1;

      InternetSetFilePointer(hURL, 0, 0, FILE_BEGIN, 0);                                      // Go back to start of file

     FILE *fp = fopen(file_name, "wb");                                                                    // Open the file for writting
       if( fp == NULL )
            return -1;


  puts("Starting download...");

 DWORD nbread = 0;
   char *buff = malloc(4096);
  int count=0;
        while( count < file_size )
       {
              InternetReadFile(hURL, buff, 4096, &nbread);
            fwrite(buff, nbread, 1, fp);
                count += nbread;
            printf("\r%d of %d bytes downloded from %s.", count, file_size, the_url);            
    }
      free(buff);
 fclose(fp);
 InternetCloseHandle(hURL); InternetCloseHandle(hInternet);
  return 0;
}    



BTW... Pelles C generates pretty clean code:
Code:
[global _main]
[section .text]
#line 8 "main.c"
[function _main]
_main:
[fpo _main, ..?X_main-_main, 0, 2, 0, 0, 0, 0]
cmp dword [esp+(4)],3
jge @346
call _show_usage
xor eax,eax
ret
@346:
mov eax,dword [esp+(8)]
push dword [eax+(8)]
push dword [eax+(4)]
call _download_file
pop ecx
pop ecx
cmp eax,-1
jne @348
push dword (@350)
call _printf
pop ecx
or eax,-1
ret
@348:
xor eax,eax
@345:
ret
..?X_main:
[global _show_usage]
#line 26 "main.c"
[function _show_usage]
_show_usage:
[fpo _show_usage, ..?X_show_usage-_show_usage, 0, 0, 0, 0, 0, 0]
push dword (@352)
call _puts
pop ecx
push dword (@353)
call _puts
pop ecx
push dword (@354)
call _puts
pop ecx
push dword (@355)
call _puts
pop ecx
push dword (@356)
call _puts
pop ecx
push dword (@354)
call _puts
pop ecx
push dword (@357)
call _puts
pop ecx
push dword 2500
call dword [(__imp__Sleep@4)]
@351:
ret
..?X_show_usage:
[global _download_file]
#line 39 "main.c"
[function _download_file]
_download_file:
push ebp
mov ebp,esp
sub esp,16
push ebx
push esi
push edi
push dword 0
push dword (0)
push dword (0)
push dword 0
push dword (@359)
call dword [(__imp__InternetOpenA@20)]
mov dword [ebp+(-16)],eax
test eax,eax
jne @360
or eax,-1
jmp @358
@360:
push dword 0
push dword 0
push dword 0
push dword (0)
push dword [ebp+(8)]
push dword [ebp+(-16)]
call dword [(__imp__InternetOpenUrlA@24)]
mov dword [ebp+(-8)],eax
test eax,eax
jne @362
or eax,-1
jmp @358
@362:
push dword 0
push dword 2
push dword (0)
push dword 0
push dword [ebp+(-8)]
call dword [(__imp__InternetSetFilePointer@20)]
mov esi,eax
cmp esi,0xffffffff
jne @364
or eax,-1
jmp @358
@364:
push dword 0
push dword 0
push dword (0)
push dword 0
push dword [ebp+(-8)]
call dword [(__imp__InternetSetFilePointer@20)]
push dword (@366)
push dword [ebp+(12)]
call _fopen
pop ecx
pop ecx
mov dword [ebp+(-12)],eax
test eax,eax
jne @367
or eax,-1
jmp @358
@367:
push dword (@369)
call _puts
pop ecx
and dword [ebp+(-4)],0
push dword 4096
call _malloc
pop ecx
mov edi,eax
xor ebx,ebx
jmp @371
@370:
lea eax,[ebp+(-4)]
push dword eax
push dword 4096
push dword edi
push dword [ebp+(-8)]
call dword [(__imp__InternetReadFile@16)]
push dword [ebp+(-12)]
push dword 1
push dword [ebp+(-4)]
push dword edi
call _fwrite
add esp,(16+3)&(~3)
mov eax,ebx
add eax,dword [ebp+(-4)]
mov ebx,eax
push dword [ebp+(8)]
push dword esi
push dword ebx
push dword (@373)
call _printf
add esp,(16+3)&(~3)
@371:
cmp ebx,esi
jb @370
push dword edi
call _free
pop ecx
push dword [ebp+(-12)]
call _fclose
pop ecx
push dword [ebp+(-8)]
call dword [(__imp__InternetCloseHandle@4)]
push dword [ebp+(-16)]
call dword [(__imp__InternetCloseHandle@4)]
xor eax,eax
@358:
pop edi
pop esi
pop ebx
mov esp,ebp
pop ebp
ret
..?X_download_file:
[extern __imp__InternetSetFilePointer@20]
[extern __imp__InternetOpenUrlA@24]
[extern __imp__InternetOpenA@20]
[extern __imp__InternetReadFile@16]
[extern __imp__InternetCloseHandle@4]
[extern _malloc]
[extern _free]
[extern __imp__Sleep@4]
[extern _fwrite]
[extern _puts]
[extern _printf]
[extern _fopen]
[extern _fclose]
[section .rdata]
[alignb 1]
@373:
db 13,'%d of %d bytes downloded from %s.',0
[alignb 1]
@369:
db 'Starting download...',0
[alignb 1]
@366:
db 'wb',0
[alignb 1]
@359:
db 'QuickGet',0
[alignb 1]
@357:
db 'Please call the program with the right parameters from a command prompt. Exiting...',0
[alignb 1]
@356:
db 'Example: quickget http://www.someurl.com/myfile.zip myfile.zip',0
[alignb 1]
@355:
db 'Usage: quickget url file_name',0
[alignb 1]
@354:
db 0
[alignb 1]
@353:
db 'Written By Frederico Wagner',0
[alignb 1]
@352:
db 'QuickGet - Commandline downloader tool v0.1',0
[alignb 1]
@350:
db 'Could not download file! Internal error!',0
[section .drectve]
db " -defaultlib:crt"
[cpu pentium]
    


BTW... I found out that I can tweak Pelles C generated code to assemble with nasm:
Code:
[global _main]
[section .text]
_main:
cmp dword [esp+(4)],3
jge @346
call _show_usage
xor eax,eax
ret
@346:
mov eax,dword [esp+(8)]
push dword [eax+(8)]
push dword [eax+(4)]
call _download_file
pop ecx
pop ecx
cmp eax,-1
jne @348
push dword (@350)
call _printf
pop ecx
or eax,-1
ret
@348:
xor eax,eax
@345:
ret

_show_usage:
push dword (@352)
call _puts
pop ecx
push dword (@353)
call _puts
pop ecx
push dword (@354)
call _puts
pop ecx
push dword (@355)
call _puts
pop ecx
push dword (@356)
call _puts
pop ecx
push dword (@354)
call _puts
pop ecx
push dword (@357)
call _puts
pop ecx
push dword 2500
call dword [(__imp__Sleep@4)]
@351:
ret

_download_file:
push ebp
mov ebp,esp
sub esp,16
push ebx
push esi
push edi
push dword 0
push dword (0)
push dword (0)
push dword 0
push dword (@359)
call dword [(__imp__InternetOpenA@20)]
mov dword [ebp+(-16)],eax
test eax,eax
jne @360
or eax,-1
jmp @358
@360:
push dword 0
push dword 0
push dword 0
push dword (0)
push dword [ebp+(8)]
push dword [ebp+(-16)]
call dword [(__imp__InternetOpenUrlA@24)]
mov dword [ebp+(-8)],eax
test eax,eax
jne @362
or eax,-1
jmp @358
@362:
push dword 0
push dword 2
push dword (0)
push dword 0
push dword [ebp+(-8)]
call dword [(__imp__InternetSetFilePointer@20)]
mov esi,eax
cmp esi,0xffffffff
jne @364
or eax,-1
jmp @358
@364:
push dword 0
push dword 0
push dword (0)
push dword 0
push dword [ebp+(-8)]
call dword [(__imp__InternetSetFilePointer@20)]
push dword (@366)
push dword [ebp+(12)]
call _fopen
pop ecx
pop ecx
mov dword [ebp+(-12)],eax
test eax,eax
jne @367
or eax,-1
jmp @358
@367:
push dword (@369)
call _puts
pop ecx
and dword [ebp+(-4)],0
push dword 4096
call _malloc
pop ecx
mov edi,eax
xor ebx,ebx
jmp @371
@370:
lea eax,[ebp+(-4)]
push dword eax
push dword 4096
push dword edi
push dword [ebp+(-8)]
call dword [(__imp__InternetReadFile@16)]
push dword [ebp+(-12)]
push dword 1
push dword [ebp+(-4)]
push dword edi
call _fwrite
add esp,(16+3)&(~3)
mov eax,ebx
add eax,dword [ebp+(-4)]
mov ebx,eax
push dword [ebp+(8)]
push dword esi
push dword ebx
push dword (@373)
call _printf
add esp,(16+3)&(~3)
@371:
cmp ebx,esi
jb @370
push dword edi
call _free
pop ecx
push dword [ebp+(-12)]
call _fclose
pop ecx
push dword [ebp+(-8)]
call dword [(__imp__InternetCloseHandle@4)]
push dword [ebp+(-16)]
call dword [(__imp__InternetCloseHandle@4)]
xor eax,eax
@358:
pop edi
pop esi
pop ebx
mov esp,ebp
pop ebp
ret

[extern __imp__InternetSetFilePointer@20]
[extern __imp__InternetOpenUrlA@24]
[extern __imp__InternetOpenA@20]
[extern __imp__InternetReadFile@16]
[extern __imp__InternetCloseHandle@4]
[extern _malloc]
[extern _free]
[extern __imp__Sleep@4]
[extern _fwrite]
[extern _puts]
[extern _printf]
[extern _fopen]
[extern _fclose]
[section .rdata]

@373:
db 13,'%d of %d bytes downloded from %s.',0

@369:
db 'Starting download...',0

@366:
db 'wb',0

@359:
db 'QuickGet',0

@357:
db 'Please call the program with the right parameters from a command prompt. Exiting...',0

@356:
db 'Example: quickget http://www.someurl.com/myfile.zip myfile.zip',0

@355:
db 'Usage: quickget url file_name',0

@354:
db 0

@353:
db 'Written By Frederico Wagner',0

@352:
db 'QuickGet - Commandline downloader tool v0.1',0

@350:
db 'Could not download file! Internal error!',0
[section .drectve]
db " -defaultlib:crt"
[cpu pentium]
    

Maybe optimize and assemble!
nasm -fwin32 main.asm
polink main.obj wininet.lib kernel32.lib crt.lib

And, Pelles C now have 64-bit version. So, it can generate 64-bit nasm code:
Code:
[global $main]
[section .text]
#line 8 "main.c"
[function $main]
$main:
push rsi
[pushreg rsi]
push rdi
[pushreg rdi]
sub rsp,40
[allocstack 40]
[endprolog ..?X$main]
mov esi,ecx
mov rdi,rdx
cmp esi,3
jge @334
call $show_usage
xor eax,eax
jmp @333
@334:
mov rdx,qword [rdi+(16)]
mov rcx,qword [rdi+(8)]
call $download_file
cmp eax,-1
jne @336
lea rcx,[(@338) wrt rip]
call $printf
or eax,-1
jmp @333
@336:
xor eax,eax
@333:
add rsp,40
pop rdi
pop rsi
ret
..?X$main:
[global $show_usage]
#line 26 "main.c"
[function $show_usage]
$show_usage:
sub rsp,40
[allocstack 40]
[endprolog ..?X$show_usage]
lea rcx,[(@340) wrt rip]
call $puts
lea rcx,[(@341) wrt rip]
call $puts
lea rcx,[(@342) wrt rip]
call $puts
lea rcx,[(@343) wrt rip]
call $puts
lea rcx,[(@344) wrt rip]
call $puts
lea rcx,[(@342) wrt rip]
call $puts
lea rcx,[(@345) wrt rip]
call $puts
mov ecx,2500
call qword [(__imp_Sleep) wrt rip]
@339:
add rsp,40
ret
..?X$show_usage:
[global $download_file]
#line 39 "main.c"
[function $download_file]
$download_file:
push r15
[pushreg r15]
push r14
[pushreg r14]
push r13
[pushreg r13]
push rbx
[pushreg rbx]
push rsi
[pushreg rsi]
push rdi
[pushreg rdi]
sub rsp,72
[allocstack 72]
[endprolog ..?X$download_file]
mov r15,rcx
mov r14,rdx
xor eax,eax
mov [rsp+(32)],eax
xor eax,eax
mov r9,rax
mov r8,rax
xor edx,edx
lea rcx,[(@347) wrt rip]
call qword [(__imp_InternetOpenA) wrt rip]
mov qword [rsp+(48)],rax
test rax,rax
jne @348
or eax,-1
jmp @346
@348:
xor eax,eax
mov [rsp+(40)],rax
xor eax,eax
mov [rsp+(32)],eax
xor r9d,r9d
xor r8,r8
mov rdx,r15
mov rcx,qword [rsp+(48)]
call qword [(__imp_InternetOpenUrlA) wrt rip]
mov rsi,rax
test rsi,rsi
jne @350
or eax,-1
jmp @346
@350:
xor eax,eax
mov [rsp+(32)],rax
mov r9d,2
xor r8,r8
xor edx,edx
mov rcx,rsi
call qword [(__imp_InternetSetFilePointer) wrt rip]
mov edi,eax
cmp edi,0xffffffff
jne @352
or eax,-1
jmp @346
@352:
xor eax,eax
mov [rsp+(32)],rax
xor r9d,r9d
xor r8,r8
xor edx,edx
mov rcx,rsi
call qword [(__imp_InternetSetFilePointer) wrt rip]
lea rdx,[(@354) wrt rip]
mov rcx,r14
call $fopen
mov qword [rsp+(56)],rax
test rax,rax
jne @355
or eax,-1
jmp @346
@355:
lea rcx,[(@357) wrt rip]
call $puts
mov dword [rsp+(68)],0
mov rcx,4096
call $malloc
mov rbx,rax
xor r13d,r13d
jmp @359
@358:
lea r9,[rsp+(68)]
mov r8d,4096
mov rdx,rbx
mov rcx,rsi
call qword [(__imp_InternetReadFile) wrt rip]
mov r9,qword [rsp+(56)]
mov r8,1
mov eax,dword [rsp+(68)]
mov edx,eax
mov rcx,rbx
call $fwrite
mov eax,r13d
add eax,dword [rsp+(68)]
mov r13d,eax
mov r9,r15
mov r8d,edi
mov edx,r13d
lea rcx,[(@361) wrt rip]
call $printf
@359:
mov eax,r13d
cmp eax,edi
jb @358
mov rcx,rbx
call $free
mov rcx,qword [rsp+(56)]
call $fclose
mov rcx,rsi
call qword [(__imp_InternetCloseHandle) wrt rip]
mov rcx,qword [rsp+(48)]
call qword [(__imp_InternetCloseHandle) wrt rip]
xor eax,eax
@346:
add rsp,72
pop rdi
pop rsi
pop rbx
pop r13
pop r14
pop r15
ret
..?X$download_file:
[extern __imp_InternetSetFilePointer]
[extern __imp_InternetOpenUrlA]
[extern __imp_InternetOpenA]
[extern __imp_InternetReadFile]
[extern __imp_InternetCloseHandle]
[extern $malloc]
[extern $free]
[extern __imp_Sleep]
[extern $fwrite]
[extern $puts]
[extern $printf]
[extern $fopen]
[extern $fclose]
[section .rdata]
[alignb 1]
@361:
db 13,'%d of %d bytes downloded from %s.',0
[alignb 1]
@357:
db 'Starting download...',0
[alignb 1]
@354:
db 'wb',0
[alignb 1]
@347:
db 'QuickGet',0
[alignb 1]
@345:
db 'Please call the program with the right parameters from a command prompt. Exiting...',0
[alignb 1]
@344:
db 'Example: quickget http://www.someurl.com/myfile.zip myfile.zip',0
[alignb 1]
@343:
db 'Usage: quickget url file_name',0
[alignb 1]
@342:
db 0
[alignb 1]
@341:
db 'Written By Frederico Wagner',0
[alignb 1]
@340:
db 'QuickGet - Commandline downloader tool v0.1',0
[alignb 1]
@338:
db 'Could not download file! Internal error!',0
[section .drectve]
db " -defaultlib:crt64"
db " -defaultlib:kernel32"
    


The good thing is nasm syntax is not too different from FASM's. Pelles C turns out to be a good ASM learning tool with it's C->ASM compiling. Very Happy
Post 14 Feb 2008, 21:43
View user's profile Send private message Reply with quote
Vasilev Vjacheslav



Joined: 11 Aug 2004
Posts: 392
Vasilev Vjacheslav 15 Feb 2008, 08:01
another open-source wget?
Post 15 Feb 2008, 08:01
View user's profile Send private message Reply with quote
Raedwulf



Joined: 13 Jul 2005
Posts: 375
Location: United Kingdom
Raedwulf 15 Feb 2008, 12:53
looks like it Smile. It is interesting though with PellesC's 64-bit support Smile. Btw, ozzy, have you tried poasm Wink lol.
Post 15 Feb 2008, 12:53
View user's profile Send private message MSN Messenger Reply with quote
penang



Joined: 01 Oct 2004
Posts: 59
penang 04 May 2008, 14:06
[quote="OzzY"]
I'll port it to FASM. :D[/quote]


Nice !
Post 04 May 2008, 14:06
View user's profile Send private message Reply with quote
penang



Joined: 01 Oct 2004
Posts: 59
penang 04 May 2008, 14:38
[quote="OzzY"]Do you want it?
I already made it in Pelles C. I'll make an optimized version in ASM.

Usage:
Create a batch file with your downloads:
[quote]
@echo off
echo "Downloading the files..."
quickget http://url1.com file1.ext
quickget http://url2.com file2.ext
quickget http://url3.com file3.ext
echo "Done!"
[/quote]

You let it downloading all over the night with low memory usage and fast downloading.
Maybe I can implement download resume, by using InternetSetFilePointer. ;)

Do you like this idea?
IMHO, why do you need a GUI for letting the computer download large files all over the night? It's just waste of memory. :D[/quote]


Bear with me, a short scenario here:

Let's suppose I get on the net from a very unreliable line. Signals get dropped all the time. Bandwidth isn't great either, gets only 40KB a second.

And let's supposed I'm downloading a large file, 80MB size, and the line dropped. No problem, I can always use download managers to do the "resume" the download, and I don't need to do the download from scratch.

But suppose if the file resides on file-sharing services such as RapidShare. Files residing on RapidShare can't be downloaded using download manager, so when the line get dropped, the download has to be done all over again.

And if the line got dropped again, the download has to be started all over, again ... and so the loop begins.

Could you add the following features --- make it so that it can do "resume download" even for files residing on RapidShare.

Thank you !
Post 04 May 2008, 14:38
View user's profile Send private message Reply with quote
Display posts from previous:
Post new topic Reply to topic

Jump to:  


< Last Thread | Next Thread >
Forum Rules:
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Copyright © 1999-2024, Tomasz Grysztar. Also on GitHub, YouTube.

Website powered by rwasa.