We are in the midst of a scientific and technological revolution. The computers of today use artificial intelligence to learn from examples and execute sophisticated functions that until recently were thought impossible. These smart algorithms can recognize faces and even drive autonomous vehicles. Deep learning networks, which are responsible for many of these technological advances, are based on the same principles that form the structure of our brain: they are composed of artificial nerve cells that are connected through artificial synapses; these cells send signals to one another via these synapses.

Our basic understanding of neural function dates back to the 1950s. Based on this elementary understanding, present-day artificial neurons that are used in deep learning operate by summing their synaptic inputs linearly and generating in response one of two output states—“0” (OFF) and “1” (ON). In recent decades, however, the field of neuroscience has discovered that individual neurons are, built from complex branching system that contains many functional sub-regions. Indeed, the branching structure of neurons and the many synapses that contact it over its distributed surface area implies that single neurons might behave as an extensive network whereby each sub-region it's own local, that is, nonlinear input-output function. 

New research at the Hebrew University of Jerusalem (HU) seeks to understand the computing power of a neuron systematically. If one maps the input-output of a neuron for many synaptic inputs (many examples), then one may be able to examine how “deep” an analogous network should be to replicate the I/O characteristics of the neuron. Ph.D. student, David Beniaguev, along with Professors Michael London and Idan Segev, at HU’s Edmond and Lily Safra Center for Brain Science (ELSC), have undertaken this challenge and have published their findings in the prestigious journal Neuron.

The objective of the study is to understand how individual nerve cells, the building blocks of the brain, translate synaptic inputs to their electrical output. In doing so, the researchers seek to create a new kind of deep learning artificial infrastructure, that will act more like the human brain and produce similarly impressive capabilities as the brain does. “The new deep learning network that we propose is built from artificial neurons whereby each of them is already 5-7 layers deep. These units are connected, via artificial synapses, to the layers above and below it,” Segev explained.

In the present state of deep neuronal networks, every artificial neuron responds to input data (synapses) with a “0” or a “1”, based on the synaptic strength it receives from the previous layer. Based on that strength, the synapse either sends (excites) —or withholds (inhibits) —a signal to neurons in the next layer. The neurons in the second layer then process the data that they received and transfer the output to the cells in the next level etc. For example, in a network that is supposed to respond to cats (but not to other animals), this network should respond for a cat with a “1” at the last (deepest) output neuron, and with a “0” otherwise. Present-state deep neuronal networks demonstrated that they can learn this task and perform it extremely well.

This approach allows computers in driverless cars, for example, to learn when they have arrived at a traffic light or a pedestrian crossing – even if the computer has never before seen that specific crosswalk. “Despite the remarkable successes that are defined as a ‘game changer’ for our world, we still don’t completely appreciate how deep learning is capable of doing what it does and many people across the world are trying to figure it out,” Segev shared.

The ability of each deep-learning network is also limited to the specific task that it’s being asked to perform. A system that was taught to identify cats isn’t able to identify dogs. Furthermore, a dedicated system needs to be in place to detect the connection between meow and cats. While the success of deep learning is amazing for specific tasks, these systems lag far behind the human brain in their ability to multi-task. “We don’t need more than one driverless car accident to realize the inherent dangers in these limitations,” Segev quipped.

Currently, significant research is being focused on providing artificial deep learning with more intelligent and all-encompassing abilities, such as the ability to process and correlate between different stimuli and to relate to different aspects of the cat (sight, hearing, touch, etc.) and to learn how to translate those various aspects into meaning. These are capabilities at which the human brain excels and those which deep learning has not yet been able to achieve.

“Our approach is to use deep learning capabilities to create a computerized model that best replicates the I/O properties of individual neurons in the brain” Beniaguev explained. To do so, the researchers relied on mathematical modeling of single neurons, a set of differential equations that were developed by Segev and London. This allows them to accurately simulate the detailed electrical processes taking place in different regions of the simulated neuron and to best map the complex transformation for the barrage of synaptic inputs and the electrical current that they produce through the tree-like structure (dendritic tree) of the nerve cell. The researchers used this model to seek a deep neural network (DNN) that replicated the I/O of the simulated neuron. They found that this task is achieved by a DNN of 5-7 layers deep.

The team hopes that building deep-learning networks based closely on real neurons which, as they have shown, are already quite deep on their own, will enable them to perform more complex and more efficient learning processes, which are more similar to the human brain. “An illustration of this would be for the artificial network to recognize a cat with fewer examples and to perform functions like internalizing language meaning. However, these are processes that we still have to prove possible by our suggested DNNs with continued research,” Segev stressed. Such a system wouldn’t just mean changing the representation of single neurons in the respective artificial neuronal network but also combine in the artificial network the characteristics of different neuron types, as is the case in the human brain. “The end goal would be to create a computerized replica that mimics the functionality, ability, and diversity of the brain – to create, in every way, true artificial intelligence,” Segev added.

This study also offered the first chance to map and compare the processing power of the different types of neurons. “For example, to simulate neuron A, we need to map seven different levels of deep learning from specific neurons, while neuron B may need nine such layers” Segev explained. “In this way, we can quantitatively compare the processing power of the nerve cell of a mouse with a comparable cell in a human brain, or between two different types of neurons in the human brain.”

On an even more basic level, the development of a computer model based on a machine learning approach that so accurately simulates brain function is likely to provide a new understanding of the brain itself. “Our brain developed methods to build artificial networks that replicate its own learning capabilities and this in return allows us to better understand the brain and ourselves,” Beniaguev concluded.