A Brain-to-Brain Interface for Real-Time Sharing of Sensorimotor Information.
A brain-to-brain interface (BTBI) enabled a real-time transfer of behaviorally meaningful sensorimotor information between the brains of two rats. In this BTBI, an “encoder” rat performed sensorimotor tasks that required it to select from two choices of tactile or visual stimuli. While the encoder rat performed the task, samples of its cortical activity were transmitted to matching cortical areas of a “decoder” rat using intracortical microstimulation (ICMS). The decoder rat learned to make similar behavioral selections, guided solely by the information provided by the encoder rat’s brain. These results demonstrated that a complex system was formed by coupling the animals’ brains, suggesting that BTBIs can enable dyads or networks of animal’s brains to exchange, process, and store information and, hence, serve as the basis for studies of novel types of social interaction and for biological computing devices.
In his seminal study on information transfer between biological organisms, Ralph Hartley wrote that “in any given communication the sender mentally selects a particular symbol and by some bodily motion, as his vocal mechanism, causes the receiver to be directed to that particular symbol”1. Brain-machine interfaces (BMIs) have emerged as a new paradigm that allows brain-derived information to control artificial actuators2 and communicate the subject’s motor intention to the outside world without the interference of the subject’s body. For the past decade and a half, numerous studies have shown how brain-derived motor signals can be utilized to control the movements of a variety of mechanical, electronic and even virtual external devices3, 4, 5, 6. Recently, intracortical microstimulation (ICMS) has been added to the classical BMI paradigm to allow artificial sensory feedback signals7, 8, generated by these brain-controlled actuators, to be delivered back to the subject’s brain simultaneously with the extraction of cortical motor commands9, 10.
In the present study, we took the BMI approach to a new direction altogether and tested whether it could be employed to establish a new artificial communication channel between animals; one capable of transmitting behaviorally relevant sensorimotor information in real-time between two brains that, for all purposes, would from now on act together towards the fulfillment of a particular behavioral task. Previously, we have reported that specific motor11, 12 and sensory parameters13, 14 can be extracted from populations of cortical neurons using linear or nonlinear decoders in real-time. Here, we tested the hypothesis that a similar decoding performed by a “recipient brain” was sufficient to guide behavioral responses in sensorimotor tasks, therefore constituting a Brain-to-Brain Interface (BTBI)15 (Figure 1). To test this hypothesis, we conducted three experiments in which different patterns of cortical sensorimotor signals, coding a particular behavioral response, were recorded in one rat (heretofore named the “encoder” rat) and then transmitted directly to the brain of another animal (i.e. the “decoder” rat), via intra-cortical microstimulation (ICMS). All BTBI experiments described below were conducted in awake, behaving rats chronically implanted with cortical microelectrode arrays capable of both neuronal ensemble recordings and intracortical microstimulation16. We demonstrated that pairs of rats could cooperate through a BTBI to achieve a common behavioral goal.
All animal procedures were performed in accordance with the National Research Council’s Guide for the Care and Use of Laboratory Animals and were approved by the Duke University Institutional Animal Care and Use Committee. Long Evans rats weighing between 250–350 g were used in all experiments.
Motor brain-to-brain interface
The behavioral motor task consisted of a dark operant chamber equipped with two levers, one LED (Light Emitting Diode) above each lever and, on the opposite wall, a water reward port. Animals were trained to press one of two levers, cued by an LED turned on at the beginning of each trial. A correct choice opened the reward port and allowed brief access to water (300 ms). When animals reached stable performances above 80% correct choices they were assigned either to an encoder or decoder group. The operant chamber configuration remained similar in both the encoder and the decoder groups. Animals assigned to the encoder group were implanted with recording arrays of 32 microelectrodes in the primary motor cortex and after recovery resumed the initial training scheme. Animals assigned to the decoder group were implanted with arrays of 4 to 6 microstimulation electrodes in the primary motor cortex and were further trained to associate the presence of electrical microstimulation pulses with the correct lever press. Extra training followed, with a sequence of 60 to 100 pulses indicating a correct choice in the right lever while the absence of microstimulation pulses (1 pulse) indicated a correct left lever choice. During the electrical microstimulation training phase a trial started with a brief period of white noise, followed by the electrical microstimulation cue. Immediately after this cue both LEDs were turned on. If a correct choice was made the reward port would open and the animal was allowed a brief period of access to water (300 ms), otherwise both LEDs were turned off and the intertrial interval started.
When decoder animals reached stable performances above chance, brain-to-brain interface (BTBI) sessions with motor activity were performed. These sessions were composed of three different phases: 1) identification of cells accurately encoding the motor action performed, 2) data collection for template trials during a rat-to-brain mode and, 3) brain-to-brain interface using real-time analysis followed by electrical microstimulation. There were no setup differences between the rat-to-brain mode and the brain-to-brain interface. The only difference between these two conditions was that in the rat-to-brain mode the patterns of microstimulation depended on the behavior of the encoder animal, while during the brain-to-brain interface mode the patterns of microstimulation depended solely on the neural activity of the encoder rat.
Neural activity was first studied in encoder animals performing the behavioral task to identify units that accurately encoded for the motor activity associated with each lever press. During the rat-to-brain mode the operant chambers remained as in the training phase, however the presence of a correct choice in the encoder operant chamber activated the pattern of electrical microstimulation cue corresponding to the same lever on the decoder animal operant chamber. A correct choice by the decoder rat was signaled by a tone and both encoder and decoder animals were allowed a brief period of access to water. The goal of this phase was to create a template trial based on the neural activity of the encoder rat pressing one of the levers in several trials.
During the brain-to-brain interface mode the number of action potentials in each trial was compared to the template trial and the Zscore of the difference between them was used to determine the number of pulses present in the pattern of microstimulation. A sigmoid function was used to transfer the Zscore value to the number of pulses present in the microstimulation pattern. A higher Zscore was associated with an increased number of pulses in the microstimulation cue (i.e. right lever) while a lower Zscore was associated with a decreased number of pulses in the microstimulation cue (e.g. left lever).
Active tactile brain-to-brain interface
Mildly water deprived animals were trained to perform a behavioral discrimination task as previously described17. Briefly, this task required animals to discriminate between a wide or narrow aperture in order to receive a water reward. The animal was placed in the behavioral box compartment where it waited for the central door to open and allow access to the second compartment, the inner chamber. After the animal entered the inner chamber, it had to pass through the variable width discrimination bars and then nose poke the center of the front wall. The nose poke in the inner chamber opened two water reward pokes located in the outer chamber from which the animal had to select one. The reward poke on the right corresponded to the wide aperture, whereas the poke on the left corresponded to the narrow aperture. As the animal chose from one of the reward pokes, the door separating the inner and outer chambers closed. Correct responses were rewarded by 50 μl water rewards. Incorrect responses were followed by immediate closing of the reward pokes. The percent of trials performed correctly was used as a measure of tactile discrimination.
Animals were then evenly assigned to encoder or decoder groups. Encoder animals (n = 2) were implanted with recording arrays of 32 microelectrodes in the right S1 and after recovery resumed the initial training scheme. Animals assigned to the decoder group were implanted with arrays of six microstimulation electrodes in the right S1 (n = 7). In six decoder animals, recording electrode arrays were also implanted either in the right (N = 1) or in the left S1 (N = 5). After recovering from surgery decoder animals were further trained to associate the presence of electrical microstimulation pulses with the correct lever press. Extensive training followed, with a sequence of 50 pulses indicating a correct choice in the left reward poke while the absence of microstimulation pulses (1 pulse) indicated a correct choice in the right reward poke. Decoder animals were required to identify the microstimulation cue and associate it with a behavioral response in one of the reward pokes. A brief tone indicated the beginning of the trial immediately followed by the microstimulation cue. After a period of 500 ms both reward pokes would open and the rat was required to make a response in one of the photo beams. A correct choice was followed by a brief tone and access to water. When decoder animals reached stable performances of > 65% correct trials for 3 consecutive sessions, tactile BTBI sessions began.
Neural activity was first studied in encoder animals performing the behavioral task to identify units that accurately encoded for the tactile stimuli associated with sampling the width between bars. During the rat-to-brain mode the operant chambers remained as in the training phase, however the microstimulation cue presented to the decoder animal always matched the stimulus presented to the encoder animal. After a correct response by the encoder rat, a brief tone followed by a microstimulation cue of 1 or 50 pulses was sent to the decoder animal and both reward ports in the second chamber would open. If the decoder rat accurately discriminated the microstimulation cue both rats were rewarded. During the brain-to-brain interface mode the neural activity of the encoder rat was analyzed from the moment that the rat broke the discrimination bars photo beam to the moment that the rat broke the photo beam in the center poke. The number of action potentials found in this interval was then counted and compared to the distribution of the Zscores relative to the spikes present in all the previous Wide trials. A Zscore was determined and transferred using a sigmoid function, into the number of pulses present in the pattern of microstimulation.
Passive tactile brain-to-brain interface
The encoder animal was anesthetized and remained head fixed in one room, while the decoder rat was in an open field in a different location with the neural activity also being recorded. The encoder rats’ whiskers were then stimulated by a set of moving bars that accurately reproduce the dynamics observed in the active tactile width discrimination task14. The bars were set up for Wide or Narrow widths and, for each width, the head fixed animal was stimulated for approximately 6 minutes at 0.3 Hz. Neural activity was first analyzed in real time and units with clear whisker related activity were used for the session. To establish a baseline distribution; an initial group of 100 wide stimulus trials was recorded. Then the passive brain-to-brain interface mode started. The encoders’ whiskers were stimulated by the moving aperture corresponding to a Wide stimulus. Meanwhile the number of action potentials recorded from the encoder animal was counted. The number of action potentials was compared to the distribution of action potentials found at the baseline at the beginning of the session and a Zscore was calculated. This Zscore was transferred into the number of pulses to be used in the microstimulation using a sigmoid function. The decoder rat then received the pattern of microstimulation derived from the sigmoid function. Immediately after the encoder animal had been passively stimulated with both Wide and Narrow widths, the decoder animal was anesthetized, head fixed, and its whiskers were passively stimulated with the same Wide and Narrow stimuli as the encoder animal.
Surgery for microelectrode array implantation
Fixed or movable microelectrode bundles or arrays of electrodes were implanted in the M1 and S1 of rats. Craniotomies were made and arrays lowered at the following stereotaxic coordinates for each area: S1 [(AP) −3.0 mm, (ML), +5.5 mm (DV) −0.7 mm], M1 [(AP) +2.0 mm, (ML) +2.0 mm, (DV) −1.5 mm].
A Multineuronal Acquisition Processor (64 channels, Plexon Inc, Dallas, TX) was used to record neuronal spikes, as previously described27. Briefly, differentiated neural signals were amplified (20000–32,000×) and digitized at 40 kHz. Up to four single neurons per recording channel were sorted online (Sort client 2002, Plexon inc, Dallas, TX ). Online sorting was validated offline using Offline Sorter 2.8.8 (Plexon Inc, Dallas, TX).
Intracortical electrical microstimulation
Intracortical electrical microstimulation cues were generated by an electrical microstimulator (Master 8, AMPI, Jerusalem, Israel) controlled by custom Matlab script (Nattick, USA) receiving information from a Plexon system over the internet. Patterns of 1–100 (bipolar, biphasic, charge balanced; 200 μsec) pulses at 400 Hz (motor BTBI) or 250 Hz (tactile BTBI) were delivered to the cortical structures of interest (M1 and S1 respectively). Current intensity varied from 38–200 μA (motor BTBI) and 30–240 μA (tactile BTBI).
For both behavioral tasks the number of correct responses was used as a measure of behavioral performance. We also analyzed the animals’ response latency as a measure of independency between the performance of each animal alone or in a dyad.
Neuronal data were processed and analyzed using Neuroexplorer (version 3.266, NEX Technologies) and custom scripts written in Matlab (7.9.0, Mathworks, Natick, MA). Statistical significance of neural responses was evaluated using a method based on cumulative-summed spike counts28, 29. Comparisons of characteristics of neural responses for different conditions were performed using non-parametric tests (Mann-Whitney-Wilcoxon or Kruskal-Wallis). Signal-to-noise ratio of neural responses was calculated as the proportion of responses, occurring after a correct or incorrect decoder response, that presented Zscore absolute values above 0.3 standard deviations. This specific value was used because it corresponded to the midpoint of the sigmoid curve). Statistical significance was determined using a chi square test for proportions.
- Hartley, R. V. L. Transmission of Information. Bell Technical Journal, 535–564 (1928).
- Jackson, A. & Zimmermann, J. B. Neural interfaces for the brain and spinal cord-restoring motor function. Nat Rev Neurol 8, 690–699 (2012).
- Ethier, C., Oby, E. R., Bauman, M. J. & Miller, L. E. Restoration of grasp following paralysis through brain-controlled stimulation of muscles. Nature 485, 368–371 (2012).
- Koralek, A. C., Jin, X., Long, J. D., 2nd, Costa, R. M., Carmena, J. M. Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills. Nature 483, 331–335 (2012).
- Lebedev, M. A. et al. Future developments in brain-machine interface research. Clinics (Sao Paulo) 66 Suppl 1, 25–32 (2011).
- Moritz, C. T., Perlmutter, S. I. & Fetz, E. E. Direct control of paralysed muscles by cortical neurons. Nature 456, 639–642 (2008).
- O’Doherty, J. E., Lebedev, M. A., Li, Z. & Nicolelis, M. A. Virtual active touch using randomly patterned intracortical microstimulation. IEEE Trans Neural Syst Rehabil Eng 20, 85–93 (2012).
- Venkatraman, S. & Carmena, J. M. Active sensing of target location encoded by cortical microstimulation. IEEE Trans Neural Syst Rehabil Eng 19, 317–324 (2011).
- O’Doherty, J. E. et al. Active tactile exploration using a brain-machine-brain interface. Nature 479, 228–231 (2011).
- Vato, A. et al. Shaping the dynamics of a bidirectional neural interface. PLoS Comput Biol 8, e1002578 (2012).
- Chapin, J. K., Moxon, K. A., Markowitz, R. S. & Nicolelis, M. A. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nat. Neurosci. 2, 664–670 (1999).
- Laubach, M., Wessberg, J. & Nicolelis, M. A. Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task. Nature 405, 567–571 (2000).
- Ghazanfar, A. A., Stambaugh, C. R. & Nicolelis, M. A. Encoding of tactile stimulus location by somatosensory thalamocortical ensembles. J. Neurosci. 20, 3761–3775 (2000).
- Krupa, D. J., Wiest, M. C., Shuler, M. G., Laubach, M. & Nicolelis, M. A. Layer-specific somatosensory cortical activation during active tactile discrimination. Science 304, 1989–1992 (2004).
- Nicolelis, M. A. Beyond Boundaries: The New Neuroscience of Connecting Brains with Machines and How It Will Change Our Lives. (Times Books, 2010).
- Fitzsimmons, N. A., Drake, W., Hanson, T. L., Lebedev, M. A. & Nicolelis, M. A. Primate reaching cued by multichannel spatiotemporal cortical microstimulation. J. Neurosci. 27, 5593–5602 (2007).
- Krupa, D. J., Matell, M. S., Brisben, A. J., Oliveira, L. M. & Nicolelis, M. A. Behavioral properties of the trigeminal somatosensory system in rats performing whisker-dependent tactile discriminations. J. Neurosci. 21, 5752–5763 (2001).
- Hasson, U., Ghazanfar, A. A., Galantucci, B., Garrod, S. & Keysers, C. Brain-to-brain coupling: a mechanism for creating and sharing a social world. Trends Cogn Sci 16, 114–121 (2012).
- Mattout, J. Brain-computer interfaces: a neuroscience paradigm of social interaction? A matter of perspective. Front Hum Neurosci 6, 114 (2012).
- Rebesco, J. M. & Miller, L. E. Stimulus-driven changes in sensorimotor behavior and neuronal functional connectivity application to brain-machine interfaces and neurorehabilitation. Prog. Brain Res. 192, 83–102 (2011).
- Rebesco, J. M., Stevenson, I. H., Kording, K. P., Solla, S. A. & Miller, L. E. Rewiring neural interactions by micro-stimulation. Front Syst Neurosci 4 (2010).
- Talwar, S. K. et al. Rat navigation guided by remote control. Nature 417, 37–38 (2002).
- Wiest, M. C., Thomson, E., Pantoja, J. & Nicolelis, M. A. Changes in S1 neural responses during tactile discrimination learning. J. Neurophysiol. 104, 300–312 (2010).
- Recanzone, G. H., Merzenich, M. M. & Dinse, H. R. Expansion of the cortical representation of a specific skin field in primary somatosensory cortex by intracortical microstimulation. Cereb. Cortex 2, 181–196 (1992).
- Shannon, C. E. The Mathematical Theory of Communication. (University of Illinois Press, 1949).
- Siegelmann, H. T. Computation beyond the turing limit. Science 268, 545–548 (1995).
- Nicolelis, M., Stambaugh, C., Brisben, A. & Laubach, M. in Methods for Neural Ensemble Recordings (ed M. A. Nicolelis) 121–156 (CRC Press, 1999).
- Gutierrez, R., Carmena, J. M., Nicolelis, M. A. & Simon, S. A. Orbitofrontal ensemble activity monitors licking and distinguishes among natural rewards. J. Neurophysiol. 95, 119–133 (2006).
- Wiest, M. C., Bentley, N. & Nicolelis, M. A. Heterogeneous integration of bilateral whisker signals by neurons in primary somatosensory cortex of awake rats. J. Neurophysiol. 93, 2966–2973 (2005).