Researchers: Jouko Lampinen, Arto Selonen, Leena Ikonen, Harri Hakkarainen and Mika Kataikko
The project is a subproject in the MENES project described above.
One of the most important properties of neural networks is generality, as the same network can be trained to solve rather different tasks, depending on the training data. This is also one of the most prominent problems when practical real world problems are solved by neural networks, as existing domain knowledge is difficult to incorporate into the models.
The goal of the research project is to develop methods for adding prior knowledge to neural network modeling. The approach we are studying is based on training the knowledge on the network instead of hard-coding the knowledge in advance to the connections or weights. The knowledge is specified as target values or constraints for different order partial derivatives of the network. This approach can be viewed as a flexible regularization method that controls directly the characteristics of the resulting mapping.
A concrete goal of the project is to develop a neural network training and simulating system that will support modular network design and domain knowledge representation with fuzzy-like terms. Together with the industrial partners this prototype system will be further developed to a process modeling and control tool.
Our research group has developed a prototype neural modeling tool that supports advanced regularization by background knowledge and modular network design, where the networks are built from smaller blocks, and also using committees of networks to improve generalization. The developed tool supports following model construction principles:
The modules are connected with adaptive layers and then the whole model can be trained together, so that the interconnection layers compensate the errors of the individual modules. This facilitates use of inaccurate simplified models for the subprocesses.
If the system is in state , then effect of input xk to output oi is
where the state and target Rik can be specified with uncertain, "fuzzy-like" terms. Thus the rules contain the target values and the membership functions that are one for acceptable derivative values and zero for forbidden values.
The theoretical background of the system, together with some experiments with real process data are reported in  and .