Fast Artificial Neural Network – FANN 2.1

fannExplorer and fannKernel

The graphical user interface fannExplorer and the multithreaded fannKernel now include the features present in the FANN 2.1 C library. Lots of tooltips have been added to fannExplorer's Algorithm, Training, Testing and Cascade windows. A version of the unreleased (available from CVS and quite stable) FANN 2.1 C source code is included with a bugfix for the 'Out of Bound' problem that prevented setting activation function and steepness on individual neurons.

.Net Languages and Mathematica Samples

Samples for C#, VB and SML.Net are included. Samples for Mathematica 5 via NetLink and Soap are also available. A bug fix for the the 'text/xml expected' problem in the gsoap 2.7.6d source code is included for compatibility with .Net 2.0.

Includes GCC compatible makefile

Compilation of fannKernel on Linux and Mac OS X requires the Boost C++ library 1.33 but is now possible – tested on Linux 2.4. Note that fannExplorer requires Flash Player version 7 or 9. It is a very simple but hopefully effective makefile. Solution file for Visual Studio C++ 2005 (requires native compilation not for the Express edition) is included for Windows.
Large view of tooltips

Large view of cascade training
Source and Executables
Documentation for C++
Source for fannExplorer

This is what you need to get started. Executables included for Windows and makefile for GCC compilers and source code. Includes .Net and Mathematica samples. [1.4Mb]

The NaturalDocs documentation for the FANN C library and the C++ wrapper in html form. [0.2Mb]

You only need this if you have Flash 8 Professional and wishes to change fannExplorer. [2.2Mb]
fannExplorer21.zip
fann21Docs.zip
fannExplorerSource.zip

Overview
fannExplorer is a portable graphical environment for developing, training and testing neural networks. It supports animation of the training process. The fannExplorer provides an easy-to-use browser based interface to the fast artificial neural network (fann) library. It requires a web browser with JavaScript, Flash version 7 or later and fannKernel.

fannKernel provides the neural network calculation engine. It is multi-threaded so multiple neural networks can be trained at the same time by one or more users. It uses Soap to communicate with client applications such as the fannExplorer. Other clients can be developed using programming languages that support Soap directly such as Adobe Flex/Flash ActionScript and Microsoft .Net languages or indirectly via Soap toolkits.

fannDotNet is a proxy dll giving languages that support .Net an interface to the fannKernel. It can be compiled with COM interoperability enabled to allow access from applications that only support COM. Samples are included for C#, VB, SML.Net and Mathematica.

Installation
Unzip the fann21.zip file anywhere you want. fannExplorer and fannKernel do not use or change registry settings or menus. Install and enable Flash Player and JavaScript in the web browser you want to use. Flash Player version 7 is required, version 9 is recommended. To uninstall simply delete the fann directory and subdirectories where you installed it.
Starting up
Double-click fannKernel.exe in the fann\fannSoap\fannKernel\Release subdirectory or start it from a console/commandline. Your firewall may display a warning at this point, you need to allow fannKernel to communicate locally with fannExplorer.

Then start your favourite web browser and type in or copy/paste the web browser link shown with your local machine name instead of SonyMobile which is the machine this is written on. If you have JavaScript and Flash 7 or later enabled then you will be greeted with the fannExplorer main window. You can save this as a shortcut/favourite/bookmark in your browser. You can run multiple fannExplorer clients simultaneously, the fannKernel is multi-threaded, so you only need to run one fannKernel when you train several neural networks at the same time. Note that using multiple threads is only faster if your processor has multiple CPU cores.
Loading a neural network
In the fannExplorer menu choose the menu items indicated with numbers - in any order. For example to load the xor sample: Load (1) xor.net with the Load Neural Network, load (2) xor.train with Load Training Data, load (3) xor.test with Load Test Data, finally (4) select Show All.

In the Controller you can click on the header lines to access different functionalities: Topology gives basic information about the neural network. Algorithm allows you to choose the exact parameters used by the neural network. Training contains functions related to training. Testing lets you evaluate the error and output of the neural network.
Training a neural network
To try out the xor sample click on the training pane header. First click on Randomize or Initialize. You can see the relative weights change in the Weight Graph window. Then click on Train. You can see the Mean Square Error Plot (MSE) being updated with a measure of how well the neural network approximates the desired output. It should rapidly approach 0.001 for this sample. If not then click Randomize and Train again.
Testing a neural network
Click on the testing pane header then click on Execute. In the Output Plot you can now see the values the neural network was trained to reproduce as green dots and the current output that the neural network produces as red dots.
Animating the training
Switch back to the Training pane. Click on Randomize to reset the neural network then click on Update output plot while animating. Now use the Animate button instead of the Train button. While this slows down the training process dramatically, it can be very instructive to view the weights, MSE and output.

You can try this again with a the BesselJ sample. First load besselj.net, besselj.train and besselj.test as before. Since the BesselJ function is more difficult for a neural network to approximate, adjust the network parameters: On the Algorithm pane set the training algorithm to Incremental. Back on the Training pane set Maximum number of epochs to 6000, set Stop when mse falls below to 0, set Epochs between mse reports to 100, and set Epochs between display updates to 500. Click on Randomize and Animate.

Preparing your own data
First you have to convert or create training and test data files in the FANN file format. It is a simple format that you can create with a text editor or a script. The first line contains three integers: Number of data pairs, number of inputs and number of outputs. Next follows the data pairs: One line with the input data values and one line with the output data values. All numbers are separated by blank space characters. See the FANN C fann_read_train_from_file and the example files if you need more details. You can not create new data files in fannExplorer, only edit simple ones.

To get good initial results normalize your input/output data in a -1 to 1 interval for use with the symmetric activation functions or 0 to 1 for the rest. Also use a 0 to 1 interval for the cascade training with the default settings. You can do this with the large number of scaling functions in FANN 2.1 or simply use a script or spreadsheet. Note that the floating point format depends on your C compiler if you use exponential notation (see printf/scanf and the formats in the FANNPRINTF/FANNSCANF macros for your compiler).

Working with files
Neural network files are placed in the net subdirectory below the fannKernel's startup directory. The files have extensions named according to their content. Fann neural network definitions have extension .net, training data have .train and test data have .test.

If you follow the same naming convention and place copies of your own fann files in the net subdirectory, then they become available in the fannExplorer's Load dialogs, so you can train your own neural networks.

Creating your own neural network
Load your data files with fannExplorer then either create a New Neural Network from the File menu or try enabling Cascade Training on the Training pane to get a network created for you. If you want to use Cascade Training then you need to read the FANN C documentation and pay attention to the tooltips.

To get started try loading the xor.train and xor.test files in fannExplorer - there is no need to load a neural network file for this sample - one will be created with cascade training. Edit the data files by pressing the Edit buttons on the Train and Test panes replacing the -1 values with 0. Press Apply when you are done. You can save the data files with new names from the File menu - you could also have used a text editor to do this. On the Training pane turn Cascade training mode on. Change 'Maximum number of neurons' to 5 and 'Neurons between mean square error tests' to 1. Then Reset the network and press Train. Use the Execute button on the Testing pane to try the little marvel or save it from the File menu.

The time series predicting sunspot sample is in a form suitable for cascade training if you want to try a more demanding example. The time series prediction works by presenting as input the values for time t-1, t-2, t-4 and t-8 and as output the value for t. The data was generated and normalized in a 0.2 to 0.8 range with a script using almost 300 years of observed sunspot activity. The test data contains the data for recent years that were excluded from the training data set. You will need a little patience and have cascade training generate about 15-20 hidden neurons for this sample to work. After you have generated a network you can turn off cascade training and see if you can improve the results with normal training and by adjusting the parameters on the Algorithm pane.

Performance
If you have a neural network with many neurons or connections then close the Weight Graph Window. It will not be informative and updating it may take so long that you will experience timeouts. If you have a large test data set then do not use the Output Plot with animation. The grid component that you can access from the Weight Graph, MSE Plot and Edit Test/Training Data is only suitable for small data sets. Use a text editor or script for larger data sets. The reason behind the fannExplorer/fannKernel design was to make it very simple to run one kernel on a machine in a classroom and have students access it via a browser, this design implies the outlined performance limitations.
Scripting
Scripting samples with C#, VB and SML.Net are included in the fannDotNet directory. You can also find samples on using Mathematica via .Net if you are using Windows or via web services otherwise. If you want to integrate and distribute a neural network that you have created with an application written in a .Net language, then you can obtain a precompiled Windows DLL from the FANN website and use Platform/Invoke to access it (see MSDN for details). If your neural network has been trained then you only need to declare functions for loading and running it in your C# or VB code.
Where to go from here
While you are using fannExplorer you can access its online help page which has more detailed information. If you are not familiar with neural networks then you will need a good text book to get satisfactory results. You can also find information on neural networks and backpropagation on Wikipedia. For more details see the FANN C or C++ documentation.

Is it time for a break? Then see a preview of what is happening in the freegoldbar programming corner. Or have a go at the DeathStacks game.

Security notice
Use a firewall when you are connected to the internet and do not run fannKernel on a webserver. fannKernel is intended to be used in a friendly environment such as on a local machine or in an intranet behind a firewall. fannKernel uses SOAP over tcp/ip by default on port 2718 (first four digits of the exponential constant) to communicate with fannExplorer clients. fannKernel does not implement authentication or encryption. If you want absolute security then you can unplug your network cable while using fannKernel. Otherwise use a good firewall to disallow incoming connections.
License
Your neural networks are your own property. The FANN C Library and C++ wrapper are distributed under LGPL. fannKernel and fannExplorer are copyright 2007/2550 by freegoldbar at yahoo dot com. All rights reserved. For private and educational use only. Not for sale, commercial or military use. No warranties given or implied. No suitability for any purpose.