Evolving Connectionist Systems

An interesting family of neural networks is Evolving Connectionist Systems (ECoS). These were invented by Professor Nik Kasabov around 1998. ECoS are constructive networks, that is, they do not start with a fixed structure but instead grow (add neurons) as training data is presented to them. The advantages of this are:

1. they are fast learning, as they learn the data as it presented, rather than iteratively
2. they are hard to over-train, as new data is accommodated by adding new neurons to the network

This makes ECoS networks very well suited to so-called "online" learning, where a stream of data is incoming and must be modeled as it arrives. Neurons are added when the current training example is either novel to the network (it has not seen something similar) or the network is not able to accurately model it.

The first ECoS was the Evolving Fuzzy Neural Network EFuNN. Later ECoS include the Simple Evolving Connectionist System SECoS (which is really an EFuNN with the fuzzy logic elements removed) and the Evolving Clustering Method ECM. EFuNN and SECoS both have rule extraction algorithms associated with them, by which fuzzy rules can be extracted from a trained EFuNN or SECoS network. This makes ECoS very useful for data mining, especially in an online application area.

I wrote a review of ECoS technology a couple of years ago, in this paper. An online reprint is available here. I also maintain a website of resources on ECoS networks at: ecos.watts.net.nz.

Research on ECoS networks is continuing, especially at Prof. Kasabov's lab KEDRI. Nowadays, ECoS research is focused on spiking neuron models, that is, neurons that include a temporal aspect to their activation, much as biological neurons do.