Modular neural network

A modular neural network is an artificial neural network characterized by a series of independent neural networks moderated by some intermediary. Each independent neural network serves as a module and operates on separate inputs to accomplish some subtask of the task the network hopes to perform.^[1] The intermediary takes the outputs of each module and processes them to produce the output of the network as a whole. The intermediary only accepts the modules' outputs—it does not respond to, nor otherwise signal, the modules. As well, the modules do not interact with each other.

As artificial neural network research progresses, it is appropriate that artificial neural networks continue to draw on their biological inspiration and emulate the segmentation and modularization found in the brain. The brain, for example, divides the complex task of visual perception into many subtasks.^[2] Within a part of the brain, called the thalamus, lies the lateral geniculate nucleus (LGN), which is divided into layers that separately processes color and contrast: both major components of vision.^[3] After the LGN processes each component in parallel, it passes the result to another region to compile the results.

Some tasks that the brain handles, like vision, employ a hierarchy of sub-networks. However, it is not clear whether some intermediary ties these separate processes together. Rather, as the tasks grow more abstract, the modules communicate with each other, unlike the modular neural network model.

Unlike a single large network that can be assigned to arbitrary tasks, each module in a modular network must be assigned a specific task and connected to other modules in specific ways by a designer. In the vision example, the brain evolved (rather than learned) to create the LGN. In some cases, the designer may choose to follow biological models. In other cases, other models may be superior. The quality of the result will be a function of the quality of the design.

Modular neural networks reduce a single large, unwieldy neural network to smaller, potentially more manageable components.^[1] Some tasks are intractably large for a single neural network. The benefits of modular neural networks include:

The possible neuron (node) connections increase quadratically as nodes are added to a network. Computation time depends on the number of nodes and their connections, any increase has drastic consequences for processing time. Assigning specific subtasks to individual modules reduce the number of necessary connections.

A large neural network attempting to model multiple parameters can suffer from interference as new data can alter existing connections or just serve to confuse. Each module can be trained independently and more precisely master its simpler task. This means the training algorithm and the training data can be implemented more quickly.

Regardless of whether a large neural network is biological or artificial, it remains largely susceptible to interference at and failure in any one of its nodes. By compartmentalizing subtasks, failure and interference are much more readily diagnosed and their effects on other sub-networks are eliminated as each one is independent of the other.

• Azam, Farooq (2000). "Biologically Inspired Modular Neural Networks. PhD Dissertation". Virginia Tech. hdl:10919/27998.
• Happel, Bart; Murre, Jacob (1994). "The Design and Evolution of Modular Neural Network Architectures" (PDF). Neural Networks. 7 (6–7): 985–1004. doi:10.1016/s0893-6080(05)80155-8.^{[permanent dead link‍]}
• Hubel, DH; Livingstone, MS (1990). "Color and contrast sensitivity in the lateral geniculate body and primary visual cortex of the macaque monkey". Journal of Neuroscience. 10 (7): 2223–2237. doi:10.1523/JNEUROSCI.10-07-02223.1990. PMC 6570379. PMID 2198331.
• Tahmasebi, P.; Hezarkhani, A. (2011). "Application of a Modular Feedforward for Grade Estimation". Natural Resources Research. 20 (1): 25–32. doi:10.1007/s11053-011-9135-3. S2CID 45997840.
• Clune, Jeff; Mouret, Jean-Baptiste; Lipson, Hod (2013-01-30). "The evolutionary origins of modularity". Proceedings of the Royal Society B: Biological Sciences. 280 (1755): 20122863. arXiv:1207.2743. doi:10.1098/rspb.2012.2863. ISSN 0962-8452. PMC 3574393. PMID 23363632.
• Tahmasebi, Pejman; Hezarkhani, Ardeshir (2012). "A fast and independent architecture of artificial neural network for permeability prediction". Journal of Petroleum Science and Engineering. 86: 118–126. Bibcode:2012JPSE...86..118T. doi:10.1016/j.petrol.2012.03.019.