SourceForge.net Logo

Subsections

1 Introduction

This report focuses on the process of implementing a fast artificial neural network library. Many performance enhancement techniques will be used and documented, to allow the library to be as fast as possible.

The report is written in English to be able to reach a broader audience. It is my hope that people interested in implementing an application in the field of artificial intelligence and computerized learning, will be able to read this report as an introduction to programming with an artificial neural network library. And that people who have already written an application using neural networks, can use this report as a guide on how to increase the performance of their application using the fann library.


1.1 Motivation

In [Nissen et al., 2002] I participated in building and programming of an autonomous robot, based on an Compaq iPAQ with a camera attached to it. In [Nissen et al., 2003] I participated in rebuilding this robot and adding artificial neural networks (ANN) for use in the image processing. Unfortunately the ANN library that we used [Heller, 2002] was too slow and the image processing on the iPAQ was not efficient enough.

The iPAQ does not have a floating point processor and for this reason we had written a lot of the image processing using fixed point arithmetic. From this experience I have learned that rewriting code to fixed point arithmetic, makes a huge difference on the performance of programs running on the iPAQ.

This experience gave me the idea, that it might be useful to develop a fast ANN library with support for fixed point arithmetic. I did however not think that writing another project based on the robot would be a good idea, so I started speculating on what I wanted to do for my master thesis.

For my master thesis I would like to make an autonomous agent, which learns from experience. To allow the agent to operate in a virtual environment, I have chosen the field of computer games. The agent should be able to function as a virtual player in a computer game and learn while doing so. Many different games could be used for this purpose, but I am leaning towards Quake III Arena [IDS, 2000], because it already has artificial agents called game bots [van Waveren, 2001]. Part of the code for these bots are public domain and it should be possible to write a new game bot on the basis of this code.

One way of building a learning game bot is to use reinforcement learning algorithms [Kaelbling et al., 1996]. Reinforcement learning can benefit from having an ANN core, which makes it useful to have a fast ANN library that can be modified for use as a reinforcement learning core. Quake III Arena is written in ANSI C and for this reason the ANN library used as the reinforcement learning core should also be written in ANSI C.

This research lead back to the need for a fast ANN library. So I decided to implement a fast ANN library which could be not only be used by me, but also by other developers.

1.2 Requirements For the Library

The primary aim of this project, is to implement a fast ANN library. However, a fast library is an ambiguous term and although it is the primary aim, it is not the only aim. Therefore the aim will be discussed and specified further.

The primary aim is for the library to be fast at executing the ANN, while training the ANN is not as time critical. The execution of the library should be fast on both systems which has a floating point processor and on systems which does not have a floating point processor.

Although I have some real needs for this library, there really is no point in developing a library which is not used by other developers. For this reason the library should be easy to use, versatile, well documented and portable.

Since the library should be used in Quake III Arena, it should be written in ANSI C. This also supports the requirement for portability, since ANSI C libraries can be used from a wide variety of programming languages and operating systems.

1.3 The Structure of This Report

This report have two main purposes:

The report is divided into five main sections:

Neural Network Theory
Describes the neural network theory needed in order to understand the rest of the report.
Analysis
Analyzes what the library should be able to do and which methods should be used in order to reach this goal.
Design and Implementation
Lowlevel API, architectural and algorithmic design.
User's Guide
A guide to using the library. This guide is partly written for people wanting to use the library and partly written to document the versatility and userfriendliness of the library.
Benchmarks
Documentation of the quality and performance of the library.

Steffen Nissen 2003-11-07