I was trying to find (and perhaps I haven't did it properly), info about this subject. I know that firewall vendors tend to use this method but when I try to find info about this in the networking doc from the DDK I found nothing, only filter drivers that are not on top of an existing TDI (like TCP/IP).

Additionally, Vista has a new framework for doing this called Windows Filtering Platform (WFP), and I have read there that firewall vendors should port their TDI hook drivers to this new platform but that Vista emulates the old way but future Windows versions won't support this anymore (I have read that it is implemented though emulation in a MSDN blog that I don't have the link to it right now).

So, is the "TDI filter driver for TCP connection or User Datagram Protocol (UDP) traffic management." documented somewhere or this is just an abuse of the driver architecture that Microsoft is aware of?


And why I need to know about this? I'm planning to log all the HTTP headers from both request and response messages for further analysis. I'm trying to analyze the impact of the transparent proxies of my ISP to see how much benefit we receive over all the inconveniences that them present (excessive and very abusive caching, RFC2616 violations, etc).

I don't want to use a opaque local proxy because besides that some software may not allow you to configure a proxy, normally when you make your software aware of the existence of a proxy it will try to bypass the cache and that would contaminate the samples. Removing the cache control headers is not an option because if a user press Ctrl+F5 I must respect that and since I have no way to know when the user really requested cache bypass then I can't use this method (even though it seems to be the simplest of all the methods I had thought of).

I also investigated a bit about Layered Service Provider, but it looked somewhat hard to implement and kernel sockets bypass it (though, this is not a big problem but an small inconvenience).

Well, I wrote too much already

I'll appreciate any info about this.

Thanks,
Loco

Perhaps you might consider a different method. Ethereal (google it, there are others also but I haven't used them) can log all the traffic for you and you can then analyse the log file in any manner you need.

Ultra heavy and since it analyze at layer-2 of OSI model it is way more harder to analyze (out of order packets, fragmentation, lots of etc.). Perhaps the HTTP dissector could help me to create a pcap file that has all frames in order and all duplicated packets out but still it has some limitations (if it does not understand very well some headers like Content-Length it does not consider the stream as a HTTP one). Additionally the best filter I can set for capturing is "tcp port 80", but this means that it will capture the whole stream while I'm only interested in the headers.

Finally it is really unpractical, I'm planning to use it in a 24/7 basis for no least than a month and install it on all my computers at home and perhaps some friends using the same ISP will use it to increase the amount of data.

What I'm gonna do is dumping the headers on a TXT, later import it to a SQLite database for manipulation and statistical data acquisition.

PS: BTW, it is called WireShark since long time ago, I've even reported a bug an year ago when it had this name already

Perhaps you can modify and re-compile Ethereal (or WireShark, or whatever) and do any analysis you require before saving anything to the log file. Since you only want headers then I expect in most cases they would not get fragmented (being usually small enough to fit into one packet).

I mean IP level fragmentation and out of order packets handling. If I hook TDI calls all will be in order and with no duplication. Handling TCP/IP yourself is not very easy and although WireShark have some routines to do this (at least it realizes when a duplicate ACK and data arrives), it is very heavy anyway and I'm wasting CPU time in processing TCP/IP twice (once Wireshark and once tcpip.sys). I also augment the number of ring level transitions because Wireshark does the TCP/IP analysis in user-mode since WinPcap does very few analysis (only apply filters to packets like "tcp port 80" or "udp" or "host x", etc), but cannot give me a stream as the representing socket sees it.

BTW, the TDI hook is somewhat documented, it can be done with IoAttachDevice (and family) which is a perfectly valid way to redirect I/O to your driver (and this is how documented filters work). However, explicit mentions of attaching your object device on top of a TDI is something that I still see no official documentation. Also exists a way in which you alter the dispatch table of the existing TDI, but I'm unsure if this way is even documented outside the networking DDK documentation.

Thanks revolution for your suggestion anyway.

For a program that is only intended to run for one month I would not think that ring transition overhead would be of a great concern to optimise away. A few extra percent (worst case I expect, and usually less) of CPU usage would not be of much importance when browsing the web.

But, of course, if you get your TDI filter working don't forget to post it here.

LocoDelAssembly,
you mean some sort of documentation like this?

Something more or less like http://msdn.microsoft.com/en-us/library/ms800889.aspx but specific to TDI. I suppose that the IoAttachDevice documentation is all what is needed because of the way TDI works and then, it is not said explicitly because people with more IQ (just normal IQ) than me can derive the documentation themselves.