I propose creation of universal pluggable components which run as separate processes and communicate with each other via cross-connected stdin and stdcout. For example HTTP proxy server would consist of two TCP plugs and a Logic unit that processes data, like TCP<--->LogicUnit<--->TCP
The idea behind this construction is to be able to assemble simple components (written in different languages) to get required functionality. In another words, its replacement of calling API with message exchange.
This approach is opposite to modern tendensy to integrate everything into huge and unreliable monsters.

any suggestions to be in the hierarchy center?

Why stdin/stdout? That introduces a fair amount of overhead compared to other methods. And what do you expect to gain? Message exchange is just another form of API where you only have two exported functions (send+recvmsg), but a - potentially - totally arbitrary message format... how is this better?

- Each component runs as a separate process in its own address space. They can not harm each other.
- Names of the entries, lists of parameters, data formats, and calling conventions are not less arbitrary then messages.
- Easy debugging and troubleshooting - you can log all inter-component communications.
- Simple way to make components backward compatible.
- Better security (components can't reach each other internals).
- Stdin/Stdout is chosen because they are supported by every programming "engine", i.e. anybody can make component with anything he wants. Almost any command-line tool may be used as is (starting from batch scripts).
- There is a well known way to start process redirecting its stdin/stdout to a pipes available to the calling process. That allows creation of various topologies of interconnected components.

Once again, the main idea is to build a project from completely independent components.

If you're serious about this and want to build something that works well and is comfortable to work with, you have a lot of work ahead of you. I suggest you take a look at COM - with out-of-process mode, it fits a lot of your goals.

I did such things with VB Script and batch files

Thing with COM is that it already meets most of your goals, can be used from multiple languages (a bit less support than stdin/stdout would have, but you'll have to consider just how useful supporting "everything" would be ), and is relatively efficient.

can someone explain this to a noob like me.

After a brief look at how COM works I have an impression that it's an attempt to apply OOP / API approach to client-server communications.
My goal is opposite - to switch from OOP / API to client-server messaging.
Yes, message exchenge is in fact send/receive API, but enforcement of API with just one message parameter requires some communication protocol to be defined externally. You have to have it documented. Communications could be dumped and then studied to verify their complience with the protocol.
On the other hand, COM embeds definitions of the communications. It also complex, requires specific OS functionality (which in fact is optional), and worse of all it requires installation/registration of interfaces in the registry breaking application portability. COM is an evil invention When I discovered that Virtual Box heavily relies on COM (which is an optional high-level OS feature) I uninstalled it and even after that had hard time cleaning up all its garbage in the registry).
There is a big difference between "using COM in your application" and using any application that already uses stdin/stdout communications.
Obviously, reading/writing stdin/stdout in your application is much simpler then using COM, why bother?
Of course, parsing/interpreting messages can also become complex, but that's what you do in your application yourself, instead of guiding some monstrous engine trying to make it do what you want.
What about bugs in COM implementation, version incompatibilities, etc.?

booter,

Yes, COM appears to be designed with C++ implementation in mind. Interface pointer could actually be pointer to C++ class instance (interface hacking, heh), with virtual functions' table pointer being it's first (hidden) member, for in-process COM server. UUIDs (and registry implementation of mapping them to COM servers) may look unwieldy, can you propose another scheme to uniquely identify something?

COM looks like great technique to abstract handle further: with interface pointer you get only applicable functions together (though any of them can give you E_NOTIMPL face-off). Mozilla guys even develop XPCOM — cross-platform COM implementation.

You may implement IPC via standalone files, file mapping, shared sections, mailslots, pipes, localhost TCP or UDP, local RPC, WM_COPYDATA, DDE, even clipboard (user goes to hell ), probably something else I forgot or don't know. All of them have their own merits and deficiencies. Choose your weapon.

No offence, though. I'm, too, looking for some plug-in functionality for my content-aware download program (e.g. to download only needful files from 50+ MiB driver package, or skip it altogether if it doesn't support my vendor-device-subsystem combo — call me byte-wise-Shylock ).

_____
Borsuc,

Do you really need somebody to tutor you in COM?

baldr
The problem with COM is that it needs installation and makes the system "dirty" by writing to the registry. By my book it's unacceptable
However, I don't mind using existing COM-aware applications (if they installed and COM is available on the particular computer).
In my opinion, stdin/stdout is the most universal way for IPC with command-line applications.
Prticularly, I'm going to write a "stdin/stdout server" in vbscript , which would invoke IE and directly control "inner HTML" to provide WEB-like GUI to the client written in FASM or anything else. It's going to be several control messages that client can send to modify HTML and detect user actions (such as clicking button).
That's way much simpler then using COM for the same functionality.
Everybody would be welcome to implement the same "server" in more efficient/economical, though more laborous way. The client executable would stay unchanged.
Finally, it seems to be possible to write "connector" that would provide COM invocation via stdin/stdout messaging.
I wrote such "connector" for TCP/IP (in FASM). It works, though I have to polish messaging interface to provide a simple way for interpretation of server responces on client side.

booter, this might be useful for you?

Ok is this something like, an app can send stdout to a server (process), which gets it and sends it then to the target app's stdin?

I still don't get why you want to make this switch - as you acknowledge yourself, you still need to define an API with your scheme... and if you don't standardize the message format, then you have both the "disadvantage" of defining an API as well as the big disadvantage of applications choosing their own ad-hoc ways to do things. And unlike a "traditional" calling scheme, you probably aren't going to do parameter validation at compile-time?

COM might not be perfect (proprietary and not so portable), but it has the following properties that I think are pretty important:
1) intra-process as well as inter-process method calls; intra-process when you need things to be fast, inter-process when you need the security/stability properties of having the method hosted in a separate process (or are doing distributed programming).
2) a well-defined way of programming against it.
3) "functionality discovery" through TLB files.
4) a registry for components (registry doesn't have to imply the Windows registry, but just a "nameserver service" - this is important if you want to re-use components between applications).
5)



Because COM has a well-defined interface, and already offers a lot of functionality you'd otherwise have to duplicate... and even if you insist on doing something from the ground up, there's things to be learned from other people's ideas

So instead of gathering the complex functionality in one engine, you'd rather have each component re-implementing it?

With your approach, you'd scatter the bugs around in multiple components - is that better? As for version incompabilities, COM actually handles that pretty well through interface querying.

I really don't get your obsession with stdin/stdout - and the method calling mechanism is something the applications shouldn't really have to know about, anyway. And client-side, method calls are pretty much always going to be preferable to more generic message-sending schemes.

IMHO you should think of this project in two separate parts: interface and implementation.

The interface defines how programs interact at the "high" level - basics: how do you look up a component, how do you get it to execute a method, how do you register yourself as a component.

The [/b]implementation[/b] deals with the nitty-gritty details like data marshalling (LocoDelAssembly's link is good), transport (intra-process with a CALL, inter-process via pipes or sockets, ...), how the registry is implemented (config file for portable apps, windows registry for reusable components, nameserver for distributed programming, ...).

This split would still let you do the actual communication via stdin/stdout if you insist, but have a reasonable interface for programming it.

And yes, defining a proper interface might exclude you from using some languages... but outside of doing things for the heck of it, do you reckon you're ever going to write an application server as a .BAT file?