Hello,

I'm working out the theorethical background of making a neural nets program in FASM. The program will be very simple, but it will need an enourmous amount of working memory to store states of millions of neurons.

So far I only know how to address memory if it's dw-ed in the program itself. I can't do that here, because the program will be something like taking up ALL available system memory (or close to that) meaning something like a 3 gigabyte program in a high-end PC. I wanted to have a program I can with fewest (or none) modificiations use in either Windows or DOS. Is there any platform-independent way to allocate memory to the program outside of what can be done with dw-s?

I assume such an operation must be relatively simple.. so can I get an example program that uses that?

Links to online resources describing this would be fine as well.

Thanks for any help.

I'm also interested in NN, but I can't do Menuet, FASM and 3D at the same time

So little time - but I can help you with some tips:
http://www.itcollege.ee/~mkalme/progemine.html

look for PowerMul and search for line 149 in the source. You will give it the bytes you need and you get back the pointer to the beginning of the region.
You will need other API for Linux of course, but for Windows - I settled with this. Maybe malloc() would suit you!?

Neural nets are fairly easy to implement. There are some things to keep in mind though:
1) You definately need floats. It won't be easy without them.
2) Each net structure takes (in 32-bit world) 8*IN_Conn_Neurals+4*OUT_Conn_Neurals+4 bytes
3) It would be relatively random memory access so swapping would make it increadibly slow
4) The structure mostly looks like a linked list

So a 10 neuron full mesh network would already need about 560bytes and the size it needs is x² times bigger when the neuron count is x times bigger. You should make some macrostructures and maybe pack them somehow so you can use SSE or other floating point help to achive the fastest speeds and lowest memory requirements. With any memory access the relativity is crucial. If you find a way to process tightly arranged NN-s together then you will gain enormous speed benefits.

I think something like malloc() would do yes, dynamic allocation of memory by the OS, as simple as that... So how do I actually use this in FASM?

...and I don't understand you in this exactly...

I'm new to the whole neural net buissnes (atho I have 'worked' in the area of AI for years), I read lots of online documentation describing neural nets and wondered if I could make one myself. My intention is not to attempt to emulate real neurons or to use elaborate mathematical equasions in an attemept to artificially force the structure into problem-solving (forcing it to progress towards the desired solution)... My neural net would be a simple set of numeric nodes that all follow a simple set of rules for interaction with each other node, which in time turns the node gird into a node circut which is capable of mimicking it's desired output (supervised learning model). Each neuron (or node) only contains basic information such as how much input it got (so it can be checked if passed treshold, etc) and where in the grid it is located.

What I don't understand here is where do the "8*IN_Conn_Neurals+4*OUT_Conn_Neurals+4 bytes" come into play, why do I need float-accuracy as opposed to say byte-accuracy, and why is this a random access model if all neurons need to be processed and their data lies togather, sequentially?

My prog would of course not be swapped, it would use RAM as I have no requirement to make it exactly N neurons big as long as it's a grid (or a 3D grid).

The all need to be processed, but the problem is that when your 5th node is connected to 137th and 25th is connected to the 3rd then in each step you have to calculate the inputs from other neurons and to read the inputs, you need to read their output so this makes it rather chaotic.

The formula was rather from the back of my head, but here is what I took into account:
1) In full mesh every neuron is connected to every other neuron
2) Half of the neurons act as inputs, the other half as output to...

In 10-neuron network you have each of the 10 neurons connected to 9 other neurons. Lets say 5 of them input and 4 of them ouput. Each pointer to another neuron takes 4 bytes as well the weight it has. This makes 8 bytes for every input neuron. You need 4 bytes for each output neuron pointer + 4 bytes for your "own" ID.

As you can see, this is rather worst case, but ± something like that.

I will try to explain why I chose floats. The floats are good because they are scalable and you don't have to worry about fairly large (130000000) and quite small numbers (0.0125) because the weights can vary that much. That said, the integers are just too linear to be used in NN. When you go to SSE world, you have the possibility to add/sub/mul/div 4 floats in one instructions and this is neccessary when you will have something like 9 inputs in a full-mesh NN.
I can't defend myself much more so you're gonna have to trust me on this

I will try to make some example code that emulates NN-s, but I'll have to dig up some old sources in BASIC for that.

There is something I want to mention about memory allocation. If you have something like NNs, then you have to allocate only the current amount of memory and this could be fairly static. If you are going to evolve your NNs runtime, THEN you will need dynamic allocation. And what I noticed in LocalAlloc, Windows still does its own thing on RAM allocation, so if you really allocate 10MB of space then Windows says in taskmanager that you have ~100KB or so...only when you access the data near the 10MB boundary it will *really* allocate it.

One last thing before I finish - the swapping is not up to you to decide. I don't know about other OS-s but Windows tends to swap anyway. And if you think you allocated the whole RAM, the next time you look, Windows has swapped some other programs and you have some more free memory

For windows you would need to use: VirtualAlloc / VirtualLock / VirtualUnlock / VirtualFree.
For DOS you would need a dpmi driver, either microsofts, or djgpp (csdpmi5b.zip):
Interrupt 31 Function 0501 - Allocate Memory Block
Interrupt 31 Function 0502 - Free Memory Block

I've created a new topic with relatively easy-to-find name in:
http://board.flatassembler.net/topic.php?p=34607#34607