overlapped operations, errors.

Index > Windows > overlapped operations, errors.

Author

Thread

b1528932

Joined: 21 May 2010
Posts: 287

b1528932 17 Jul 2010, 14:44

i have a problem.
from my understanding, overlapped functions (iocompletion, getoverlappedresult, getqueuedcompletionstatus) return exactly same results as functions that initiated them.

for example, EOF can be reported in readfile, but if it return -1/997, its reported in overlapped notyfication.

if WSARecv detect connection close, situation should be exactly same.

following scheme -1(or 0)/997 -> GetLastError() of notyfication will hold specyfic error.

when i issue WSARecv() in blocking mode, and close connection - it report connection reset status (or nothing meaning gracefull close).

But if i use it in overlapped manner, it return -1/997, and GetOverlappedResult/GetQueuedCompletionStatus return ERROR_NETNAME_DELETED (64)!

WTF! how can i handle this, what can i do? i do not mean this particular error, but rather diffrence between errors reported by normal and notyfication functions.

17 Jul 2010, 14:44

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20445
Location: In your JS exploiting you and your system

revolution 17 Jul 2010, 15:30

Winsock sends a message to your window to indicate I/O status updates. You set it up like this:

Code:

invoke WSAAsyncSelect,[sock],[hwnd],MY_SOCKET_EVENT,FD_READ+FD_WRITE+FD_CONNECT+FD_CLOSE

And process it in your WindowsProc

Code:

cmp      [wmsg],MY_SOCKET_EVENT
jz    .socket_event

The lParam has the event and error codes. wParam identifies the socket.

For file I/O it is a little bit trickier. You need to use an alertable wait function to receive the callback of I/O status changes. MsgWaitForMultipleObjectsEx and SleepEx are the most commonly used. The former is Win2K+ and the latter is Win95+ if that is important to you.

17 Jul 2010, 15:30

b1528932

Joined: 21 May 2010
Posts: 287

b1528932 17 Jul 2010, 19:44

instead of + you mean | ?

im not asking about winsock, but fundamental design of overlapped io.

and i dont know why they included this notyfication thing, really.
receiving anything other than FD_CONNECT and FD_ACCEPT is useless.
and even those 2 can be used in overlapped manner on windows xp and later.

17 Jul 2010, 19:44

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20445
Location: In your JS exploiting you and your system

revolution 17 Jul 2010, 19:55

| is a logical operator in fasm. If you are unsure about overlapping bits then use or for binary operations.

As for the fundamental design of overlapped I/O, it depends upon which sub-system you are using. The sockets operate differently from the files so you have to treat each in a different manner.

If you don't have any notifications, then how do you know when a background operation needs attention?

I don't understand why you say "anything other than FD_CONNECT and FD_ACCEPT is useless". Why do you think that? When using Overlapped I/O in the proper manner in a single threaded application it is possible to have 1000s of simultaneous open sockets and files without ever blocking just by processing the notifications as they arrive.

17 Jul 2010, 19:55

b1528932

Joined: 21 May 2010
Posts: 287

b1528932 17 Jul 2010, 20:10

Quote:

I don't understand why you say "anything other than FD_CONNECT and FD_ACCEPT is useless".

because non-blocking more is worse than overlapped. in overlapped mode everything is synchronous, you dont have to do arbitary functions upon other data, like whenb you receive fd_close, or fd_obb.

i prefer having 1 main thread, and use io completion ports.
- *Ex apc method is broken by design - deadlock.
- WaitForMultiple - limit 63 handles, each requiring event object. Events are not cheap. So only 63 operations could execute simultaniusly, no matter what - suck.

when using iocp i can create worker thread(s) and do all processing there, nothing useless in main thread. main thread could process window messages, and block on waitformultiple.

aha, overlapped != nonblocking. perhaps you didnt know that.
overlapped is superior, nonblocking is legacy from < win2000 (and from win2000 becaus eit lack overlapped connect/accept)

17 Jul 2010, 20:10

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20445
Location: In your JS exploiting you and your system

revolution 18 Jul 2010, 01:12

b1528932: I don't understand your problem. Do you have some code that you are having trouble with?

18 Jul 2010, 01:12

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20445
Location: In your JS exploiting you and your system

revolution 18 Jul 2010, 01:27

BTW: You are incorrect about the 63 operation limit. With overlapped I/O the file system uses the APC which has no limit. And winsock used in AsyncSelect mode, which I described above, uses the message queue which also has no limit. The MsgWaitForMultipleEx does not use any event handles, you can wait on a single handle, your own process handle which never signals to yourself, and let the APC complete the file I/O and the message queue receive winsock events. With win95, you use SleepEx with a wait time of zero to allow the APC to complete any outstanding file I/O notifications and can setup a timer, or other method, to periodically break your message loop and trigger more SleepEx's. If, instead, you open separate threads for each operation to do the work then you increase the resource usage dramatically. Plus you have the same problem with synchronisation, you still need to send notifications back to your main thread anyway.

18 Jul 2010, 01:27

Tyler

Joined: 19 Nov 2009
Posts: 1216
Location: NC, USA

Tyler 18 Jul 2010, 02:32

AFAIK, MS always aligns those values on powers of 2 boundaries, so adding is fine.

18 Jul 2010, 02:32

b1528932

Joined: 21 May 2010
Posts: 287

b1528932 18 Jul 2010, 12:51

revolution wrote:

BTW: You are incorrect about the 63 operation limit. With overlapped I/O the file system uses the APC which has no limit. And winsock used in AsyncSelect mode, which I described above, uses the message queue which also has no limit. The MsgWaitForMultipleEx does not use any event handles, you can wait on a single handle, your own process handle which never signals to yourself, and let the APC complete the file I/O and the message queue receive winsock events. With win95, you use SleepEx with a wait time of zero to allow the APC to complete any outstanding file I/O notifications and can setup a timer, or other method, to periodically break your message loop and trigger more SleepEx's. If, instead, you open separate threads for each operation to do the work then you increase the resource usage dramatically. Plus you have the same problem with synchronisation, you still need to send notifications back to your main thread anyway.

apc and event are diffrent ways of doing this.
i wont use apc, because it will deadlock my thread (if i queue apc inside apc, when apc return it will run again instead of code in main thread). And deviceiocontrol cant be done in apc way, so it suck.
waitformultipleobjects has a limit of 63 handles, so it can be used on client, but is unacceptable for server.

the perfect solution is io completion port. if not avaiable, i can use other methods, like waitformultiple, sleepex, or even threads if os doesnt support anything more. i have to check whats avaiable on windows 95 and 3.11, and then create universal framework for any app i will ever want to write, so i have to care about code.

18 Jul 2010, 12:51

f0dder

Joined: 19 Feb 2004
Posts: 3175
Location: Denmark

f0dder 19 Jul 2010, 07:42

Tyler wrote:

AFAIK, MS always aligns those values on powers of 2 boundaries, so adding is fine.

Sure thing, except if you add the same value multiple times - not much of an issue in source code, but if you're combining flags at runtime it can happen. Might as well do the proper thing and always use bitwise OR/AND when dealing with flags.

revolution: ugh, WSAAsyncSelect? Smile

- that's only really suitable for very low-traffic applications.

b1528932 wrote:

WaitForMultiple - limit 63 handles, each requiring event object. Events are not cheap. So only 63 operations could execute simultaniusly, no matter what - suck.

This is less of a problem than one might think, actually. Sure, IOCP is better, but WSAEventSelect (with a limited amount of events) can work pretty well. µtorrent uses this method, balancing connections between n events. When an event fires, it runs through all sockets associated with the event and calls WSAEnumNetworkEvents on each to figure out where the action is.

Yup, it requires more processing than IOCP, but it works pretty well in practice - I think most people will agree that µtorrent isn't a bad resource hog Smile

19 Jul 2010, 07:42

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20445
Location: In your JS exploiting you and your system

revolution 19 Jul 2010, 08:48

f0dder wrote:

revolution: ugh, WSAAsyncSelect? - that's only really suitable for very low-traffic applications.

I found that the message queue is very efficient. I am currently finalising a 1Gbps app that opens up 700+ connections to the slave sensors. So far no problem in single threaded mode. Usually ~80% CPU usage and the UI is nicely responsive. And that is on a slow P4.

Actually I've always preferred queues to events. Multiple event detection is always tricky to code efficiently, but queues are simple to understand and code. I can't see the advantage of events, the work still needs to be done whether it comes from a queue or an event. There is no shortcut to reduce the CPU usage.

19 Jul 2010, 08:48

rugxulo

Joined: 09 Aug 2005
Posts: 2341
Location: Usono (aka, USA)

rugxulo 19 Jul 2010, 11:08

revolution wrote:

So far no problem in single threaded mode. Usually ~80% CPU usage and the UI is nicely responsive. And that is on a slow P4.

Kids today. Laughing

19 Jul 2010, 11:08

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20445
Location: In your JS exploiting you and your system

revolution 19 Jul 2010, 11:21

rugxulo wrote:

revolution wrote:
slow P4.

Kids today.

Slow for a P4. There were faster P4s made. How else should I say it? "A P4 CPU running at the lower end of the speed spectrum when compared to other P4s"? It's a bit wordy though. Wink

19 Jul 2010, 11:21

f0dder

Joined: 19 Feb 2004
Posts: 3175
Location: Denmark

f0dder 19 Jul 2010, 12:22

revolution wrote:

f0dder wrote:
revolution: ugh, WSAAsyncSelect? - that's only really suitable for very low-traffic applications.
I found that the message queue is very efficient. I am currently finalising a 1Gbps app that opens up 700+ connections to the slave sensors. So far no problem in single threaded mode. Usually ~80% CPU usage and the UI is nicely responsive. And that is on a slow P4.

I don't know what other CPU processing your application is doing, so can't really judge the efficiency - but the stats I've seen for AsyncSelect have been pretty horrible once you start having a lot of connections and/or high traffic. A quick google turned up this post.

revolution wrote:

I can't see the advantage of events, the work still needs to be done whether it comes from a queue or an event. There is no shortcut to reduce the CPU usage.

Of course there's CPU usage with every model, and you can't avoid some kernel/user roundtrips... but the Windows threadpump isn't known to be �ber-efficient, and it shows in the table in the post I've linked.

_________________

- carpe noctem

19 Jul 2010, 12:22

b1528932

Joined: 21 May 2010
Posts: 287

b1528932 19 Jul 2010, 14:58

wsaeventselect require 1 event for everything.
only issue here is calling WSAEnumNetworkEvents for every single socket in my thread. so if i have milion clients and 1 of them sends data, i have to loop milion tmies calling WSAEnumNetworkEvents.

wsaasyncselect - i have to call WSAEnumNetworkEvents, check if msg is real, and do other securoty checks.
same as event version, and data are unreliable, because user might just use postmessage to send me his own data.

19 Jul 2010, 14:58

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20445
Location: In your JS exploiting you and your system

revolution 19 Jul 2010, 15:12

b1528932 wrote:

and data are unreliable, because user might just use postmessage to send me his own data.

Why is your code fighting the user? You appear to be treating the user as a hostile element. This thread might be asking a question I don't want to answer. Confused

19 Jul 2010, 15:12

b1528932

Joined: 21 May 2010
Posts: 287

b1528932 19 Jul 2010, 15:17

Quote:

Why is your code fighting the user? You appear to be treating the user as a hostile element.

user can have access only to desktop on wich i have this queue. and if it does, it can compromise my app if i dont do error checks.
using PostMessage is 100% legal, and writing memory is illegal, so if user will behave in documented way, it will crach my app. if i use event version - he can set event ofc, but when i check for events and dont find any i just do nothing.
async version doesnt have that capability, and even if it does - its less efficiend because information passed to me in wparam/lparam are useless.

when user has access to desktop, i maybe dont want do give him access to socket and my code.

using async version is just wrong.

19 Jul 2010, 15:17

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20445
Location: In your JS exploiting you and your system

revolution 19 Jul 2010, 15:23

b1528932 wrote:

Quote:
Why is your code fighting the user? You appear to be treating the user as a hostile element.

user can have access only to desktop on wich i have this queue. and if it does, it can compromise my app if i dont do error checks.
using PostMessage is 100% legal, and writing memory is illegal, so if user will behave in documented way, it will crach my app. if i use event version - he can set event ofc, but when i check for events and dont find any i just do nothing.
async version doesnt have that capability, and even if it does - its less efficiend because information passed to me in wparam/lparam are useless.

when user has access to desktop, i maybe dont want do give him access to socket and my code.

Didn't answer my Q though Sad

b1528932 wrote:

using async version is just wrong.

Well if your usage situation is such then I agree.

But I still don't understand your query here. What do you need help with? Is there something not working that you need to fix?

19 Jul 2010, 15:23

revolution
When all else fails, read the source

Joined: 24 Aug 2004
Posts: 20445
Location: In your JS exploiting you and your system

revolution 19 Jul 2010, 15:29

f0dder wrote:

A quick google turned up this post.

Interesting. I've never had the need for so many connections. But their data rates are shockingly low. I wonder what packet size they were using? There appears to be a lot of overhead (or something) sucking up the theoretical 100Mb bandwidth.

19 Jul 2010, 15:29

f0dder

Joined: 19 Feb 2004
Posts: 3175
Location: Denmark

f0dder 19 Jul 2010, 16:00

b1528932 wrote:

wsaeventselect require 1 event for everything.
only issue here is calling WSAEnumNetworkEvents for every single socket in my thread. so if i have milion clients and 1 of them sends data, i have to loop milion tmies calling WSAEnumNetworkEvents.

The idea is to set up N queues, with one event per queue - when WaitForMultipleObjectSex returns, you know which queue to loop. Yes, there can potentially be a lot of sockets to check, but dividing into multiple queues does help. Works just fine for µtorrent with a pretty fair amount of (active) connections. And you can create a thread per CPU core if you want to. Unless you want Win9x support, you might as well just use IOCP though, since it is the best-performing method. Perhaps a bit more complex than other methods, but you only have to get the framework right once Smile

revolution wrote:

Interesting. I've never had the need for so many connections. But their data rates are shockingly low. I wonder what packet size they were using? There appears to be a lot of overhead (or something) sucking up the theoretical 100Mb bandwidth.

The test is an echo server, so it has to both send and receive. Perhaps the network setup was half duplex? At least then the best figures add up to ~100mbps total.

_________________

- carpe noctem

19 Jul 2010, 16:00

< Last Thread | Next Thread >

Forum Rules:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum