I want to make a simple program on neural networks in Lisp but I think my knowledge in programming is not enough. It will be a great help if you can help me making a simple one. Can you tell me the basics of neural networks? Can you give me some good references?
Thanks a lot!
Neural networks are computer programs that solve a class of problems called "combinatorial optimisation problems". They solve these problems by using a very very very simplified model of how the brain works.
Combinatorial optimisation problems are problems where you have a lot of inputs and you want to work something out from these inputs. An example would be face recognition: you have a picture of a face made up of thousands of pixels, and you want to match this face to a database of 500 faces that you know of. Lets say you have 6000 pixels in your image. You might think of storing all the possible combinations of images that could be displayed in these pixels, but this would take a huge amount of memory space - more memory than is in every computer every built. This is because the number of different images that can be stored in a number of pixels increases exponentially as the number of pixels increases. This is called a "combinatorial explosion", hence the name "combinatorial optimisation problem". There are too many combinations of pixel vaules (and hence images) to store information about each one.
This is where neural networks come in. They take in all the inputs and feed them in to a network of artificial nerve cells. These nerve cells behave like very very simplified nerves in the brain. They look at how many inputs they are getting, and if they have an input over a given threshold value, then the "fire", meaning they send out a signal on their output. The output of one nerve cell is connected to the input of another, making a network of nerve cells (hence the name "neural network").
In the traditional type of neural network, all the connections "point in the same way", so that the network is a "directed, acyclic graph". All the inputs are fed in at one end of the network (like the values of each pixel in the image), and at the other end, an output is produced (there would be one output value for each person in the database). Then in our face recognition example, the neural network would be wired up so that when the face of someone in the database was fed in, the network would go to work and only one of the outputs would fire: the one corresponding to the person whose face was fed in.
In practice, neural networks tend to be wired up in a very simple way and the main problem is to decide what weights to put on each connection between nerve cells. Each connection has a weight to say how much this connection contributes to the cell that it is feeding into. There are fancy algorithms that can "learn" the best weights to put on each connection, so that the network does not have to be designed by hand. Some examples of input data and correct answers can be fed into the algorithm, and it will change the weights on the connections to make the network produce the right answers.
This learning is where neural networks shouw their strength: they can be shown some limited "training examples" to set their weights, and then they will still work on inputs that are different from that they were trained on, but similar enough. So for example in the face recognition system, the network may be able to recognise a face that it has been trained on in a different lighting condition.
I hope that helps a little. There is so much to neural networks, and also different types of neural network that I can't begin to describe here. These links should help you with further reading:
http://en.wikipedia.org/wiki/Neural_network
http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html
http://www.cs.stir.ac.uk/~lss/NNIntro/InvSlides.html
This is a matlab extension for simulating neural networks. I haven't used it myself but it looks interesting:
http://www.mathworks.com/products/neuralnet/
Good luck with your program. If you have any more questions about neural networks or your program, email me and I will help you out. I've not used Lisp but I have used Scheme.
___________________
Hope that helps, Adam
My online games website: http://www.cygnetgames.co.uk
powered by Yahoo Answers
trained neural network that drives a car along a track credits: Ogre3d OgreNewt Track by Pierre Blanc …
A project I did to train neural networks to play Puyo Puyo. I needed a platform to test the agents on as well, so I wrote the simple Puyo Puyo clone you see here. This demo shows two agents competing after being trained for about 20 generations.