Gradual progression from sensory to task-related processing in cerebral cortex
Scott L. Brincat*, Markus Siegel*, Constantin von Nicolai, Earl K. Miller
doi: https://doi.org/10.1101/195602
Abstract
Somewhere along the cortical hierarchy, behaviorally relevant information is distilled from raw sensory inputs. We examined how this transformation progresses along multiple levels of the hierarchy by comparing neural representations in visual, temporal, parietal, and frontal cortices in monkeys categorizing across three visual domains (shape, motion direction, color). Representations in visual areas MT and V4 were tightly linked to external sensory inputs. In contrast, prefrontal cortex (PFC) largely represented the abstracted behavioral relevance of stimuli (task rule, motion category, color category). Intermediate-level areas — posterior inferotemporal (PIT), lateral intraparietal (LIP), and frontal eye fields (FEF) — exhibited mixed representations. While the distribution of sensory information across areas aligned well with classical functional divisions — MT carried stronger motion information, V4 and PIT carried stronger color and shape information — categorical abstraction did not, suggesting these areas may participate in different networks for stimulus-driven and cognitive functions. Paralleling these representational differences, the dimensionality of neural population activity decreased progressively from sensory to intermediate to frontal cortex. This shows how raw sensory representations are transformed into behaviorally relevant abstractions and suggests that the dimensionality of neural activity in higher cortical regions may be specific to their current task.
A review of how the prefrontal cortex and high-level visual cortex interact during perception.
Kornblith, S., & Tsao, D. Y. (2017). How thoughts arise from sights: inferotemporal and prefrontal contributions to vision. Current Opinion in Neurobiology, 46, 208-218.
“The Functional Architecture of Cognition Is Rhythmic”. Indeed.
Randolph F. Helfrich, Robert T. Knight, Oscillatory Dynamics of Prefrontal Cognitive Control, In Trends in Cognitive Sciences, Volume 20, Issue 12, 2016, Pages 916-930, ISSN 1364-6613, https://doi.org/10.1016/j.tics.2016.09.007.
Rhythmic coupling across the cortex underlies perception.
Randolph F. Helfrich, Melody Huang, Guy Wilson, and Robert T. Knight
Prefrontal cortex modulates posterior alpha oscillations during top-down guided visual perception
PNAS 2017 114 (35) 9457-9462; published ahead of print August 14, 2017, doi:10.1073/pnas.1705965114
Distributed networks, not functional modules, are where it’s at in the 21st century.
This paper argues that decisions come from repeated computations that are distributed across many brain regions. This fits with the distributed nature of neural coding.
Nat Rev Neurosci. 2017 Feb 17;18(3):172-182. doi: 10.1038/nrn.2017.7.
A distributed, hierarchical and recurrent framework for reward-based choice.
Hunt LT, Hayden BY
For further reading:
Siegel, M., Buschman, T.J., and Miller, E.K. (2015) Cortical information flow during flexible sensorimotor decisions. Science. 19 June 2015: 1352-1355. View PDF »
Neuron populations, not individual neurons, are where it’s at in the 21st century.
Iñigo Arandia-Romero, Ramon Nogueira, Gabriela Mochol, Rubén Moreno-Bote, What can neuronal populations tell us about cognition?, In Current Opinion in Neurobiology, Volume 46, 2017, Pages 48-57, ISSN 0959-4388, https://doi.org/10.1016/j.conb.2017.07.008.
Come see Pavlov’s Dogz at Songbyrd DC on Sunday Nov 12 of the SFN meeting.
9:30pm Songbyrd DC 11/12/17
http://www.songbyrddc.com/shows/2017-11-12-pavlolvs-dogz
Charan Ranganath
Jess Grahn
Mick Rugg
Andy Lee
We show how limitations in cognitive capacity (how many thoughts you can think at the same time – very few) may be due to changes in rhythmic coupling between cortical areas. More specifically, feedback coupling breaks down when capacity is exceeded.
Working Memory Load Modulates Neuronal Coupling Dimitris A Pinotsis, Timothy J Buschman, Earl K Miller
doi: https://doi.org/10.1101/192336
New manuscript submitted to bioRxiv:
Neuronal rhythms orchestrate cell assembles to distinguish perceptual categories
Morteza Moazami Goudarzi, Jason Cromer, Jefferson Roy, Earl K. Miller
doi: https://doi.org/10.1101/191247
Abstract
Categories are reflected in the spiking activity of neurons. However, how neurons form ensembles for categories is unclear. To address this, we simultaneously recorded spiking and local field potential (LFP) activity in the lateral prefrontal cortex (lPFC) of monkeys performing a delayed match to category task with two independent category sets (Animals: Cats vs Dogs; Cars: Sports Cars vs Sedans). We found stimulus and category information in alpha and beta band oscillations. Different category distinctions engaged different frequencies. There was greater spike field coherence (SFC) in alpha (~8-14 Hz) for Cats and in beta (~16-22 Hz) for Dogs. Cars showed similar differences, albeit less pronounced: greater alpha SFC for Sedans and greater beta SFC for Sports Cars. Thus, oscillatory rhythms can help coordinate neurons into different ensembles. Engagement of different frequencies may help differentiate the categories.
A very interesting theory from Randy O’Reilly and crew. I don’t know how to summarize it better than they did in their abstract:
How does the neocortex learn and develop the foundations of all our high-level cognitive abilities? We present a comprehensive framework spanning biological, computational, and cognitive levels, with a clear theoretical continuity between levels, providing a coherent answer directly supported by extensive data at each level. Learning is based on making predictions about what the senses will report at 100 msec (alpha frequency) intervals, and adapting synaptic weights to improve prediction accuracy. The pulvinar nucleus of the thalamus serves as a projection screen upon which predictions are generated, through deep-layer 6 corticothalamic inputs from multiple brain areas and levels of abstraction. The sparse driving inputs from layer 5 intrinsic bursting neurons provide the target signal, and the temporal difference between it and the prediction reverberates throughout the cortex, driving synaptic changes that approximate error backpropagation, using only local activation signals in equations derived directly from a detailed biophysical model. In vision, predictive learning requires a carefully-organized developmental progression and anatomical organization of three pathways (What, Where, and What * Where), according to two central principles: top-down input from compact, high-level, abstract representations is essential for accurate prediction of low-level sensory inputs; and the collective, low-level prediction error must be progressively and opportunistically partitioned to enable extraction of separable factors that drive the learning of further high-level abstractions. Our model self-organized systematic invariant object representations of 100 different objects from simple movies, accounts for a wide range of data, and makes many testable predictions.
O’Reilly, R. C., Wyatte, D. R., & Rohrlich, J. (2017). Deep Predictive Learning: A Comprehensive Model of Three Visual Streams. arXiv preprint arXiv:1709.04654.
Paper showing different, yet complementary, effects of attention and value on alpha vs gamma oscillations in posterior cortex.
Marshall, T. R., den Boer, S., Cools, R., Jensen, O., Fallon, S. J., & Zumer, J. M. (2017). Occipital Alpha and Gamma Oscillations Support Complementary Mechanisms for Processing Stimulus Value Associations. Journal of Cognitive Neuroscience.
And they show the same independence between the visual hemifields that we saw in primates.
Balakhonov, D., & Rose, J. (2017). Crows Rival Monkeys in Cognitive Capacity. Scientific Reports, 7.
For further reading:
Buschman,T.J., Siegel, M., Roy, J.E. and Miller, E.K. (2011) Neural substrates of cognitive capacity limitations. Proceedings of the National Academy of Sciences. 108(27):11252-5. View PDF »
Miller, E.K. and Buschman, T.J. (2015) Working memory capacity: Limits on the bandwidth of cognition. Daedalus, Vol. 144, No. 1, Pages 112-122. View PDF »
Kornblith, S., Buschman, T.J., and Miller, E.K. (2015) Stimulus load and oscillatory activity in higher cortex. Cerebral Cortex. Published online August 18, 2015 doi: 10.1093/cercor/bhv182. View PDF »
Driscoll et al tracked parietal cortex neurons over one month after mice learned and practiced a navigation task. The activity of individual neurons changed but information on the population level was stable. This is a nice demonstration of “mixed selectivity” in individual neurons and further evidence that the functional unit of the brain is neural ensembles, not individual neurons.
Driscoll, L. N., Pettit, N. L., Minderer, M., Chettih, S. N., & Harvey, C. D. (2017). Dynamic reorganization of neuronal activity patterns in parietal cortex. Cell.
For further reading:
Fusi, S., Miller, E.K., and Rigotti, M. (2016) Why neurons mix: High dimensionality for higher cognition. Current Opinion in Neurobiology. 37:66-74 doi:10.1016/j.conb.2016.01.010. View PDF »
Rigotti, M., Barak, O., Warden, M.R., Wang, X., Daw, N.D., Miller, E.K., & Fusi, S. (2013) The importance of mixed selectivity in complex cognitive tasks. Nature, 497, 585-590, doi:10.1038/nature12160. View PDF »
When you search for something do you simply hold a static template of it in mind. Apparently not. Your search template waxes and wanes, waxing with the anticipated moment of search. When the template is strong, your eyes move less.
Olmos-Solis, K., van Loon, A. M., Los, S. A., & Olivers, C. N. (2017). Oculomotor measures reveal the temporal dynamics of preparing for search. Progress in Brain Research.
A study using a combination of TMS and FMRI to asses functional connectivity.
Hawco, C., Armony, J. L., Daskalakis, Z. J., Berlim, M. T., Chakravarty, M. M., Pike, G. B., & Lepage, M. (2017). Differing Time of Onset of Concurrent TMS-fMRI during Associative Memory Encoding: A Measure of Dynamic Connectivity. Frontiers in Human Neuroscience, 11, 404.
Working memory for different items in a sequence is prioritized by how much attention is paid to the item at encoding.
Jafarpour, A., Penny, W., Barnes, G., Knight, R. T., & Duzel, E. (2017). Working Memory Replay Prioritizes Weakly Attended Events. eNeuro, 4(4), ENEURO-0171.
Context-dependent attractor dynamics can underlie mental flexibility.
Tajima, S., Koida, K., Tajima, C. I., Suzuki, H., Aihara, K., & Komatsu, H. (2017). Task-dependent recurrent dynamics in visual cortex. eLife, 6, e26868.
Marc Howard reviews “time cells” in the brain. Time cells show Weber-fraction like decreases in accuracy the further in the past you go. Interestingly, these cells keep track of time even when tasks do not require it. You can’t escape time.
Howard, M. W. Memory as perception of the past: Compressed time in mind and brain.
Increased theta synchrony between the prefrontal cortex and hippocampus when subjects encoded unexpected study items. This is further evidence that theta-band (6-10 Hz) oscillations orchestrate communication between these brain areas.
Gruber, M. J., Hsieh, L. T., Staresina, B., Elger, C., Fell, J., Axmacher, N., & Ranganath, C. (2017). Theta Phase Synchronization Between The Human Hippocampus And The Prefrontal Cortex Supports Learning Of Unexpected Information. bioRxiv, 144634.
For further reading:
Brincat, S.L. and Miller, E.K. (2015) Frequency-specific hippocampal-prefrontal interactions during associative learning. Nature Neuroscience. Published online 23 Feb 2015 doi:10.1038/nn.3954. View PDF »
Brincat, S.L. and Miller, E.K (2016) Prefrontal networks shift from external to internal modes during learning Journal of Neuroscience. 36(37): 9739-9754, 2016 doi: 10.1523/JNEUROSCI.0274-16.2016. View PDF
Alpha-band oscillations in visual cortex (area 4) link sites that encode the location of a stimulus before and after an eye movement. This shows how brain rhythms can construct a stable representation of a visual scene as our eyes move,
Neupane, S., Guitton, D., & Pack, C. C. (2017). Coherent alpha oscillations link current and future receptive fields during saccades. Proceedings of the National Academy of Sciences, 114(29), E5979-E5985.
Loops between the basal ganglia and the cerebral cortex allow the basal ganglia to control functional connectivity in the cortex by synchronizing its rhythms.
Pouzzner, D. (2017). Control of Functional Connectivity in Cerebral Cortex by Basal Ganglia Mediated Synchronization. arXiv preprint arXiv:1708.00779.
For further reading:
Antzoulatos, E.G. and Miller, E.K. (2014) Increases in functional connectivity between the prefrontal cortex and striatum during category learning. Neuron, 83:216-225. View PDF »
Selected as one of Neuron’s Best of 2014-2015
Miller, E.K. and Buschman, T.J. (2013) Cortical circuits for the control of attention. Current Opinion in Neurobiology. 23:216–222. View PDF »
Buschman,T.J. and Miller, E.K. (2010) Shifting the Spotlight of Attention: Evidence for Discrete Computations in Cognition. Frontiers in Human Neuroscience. 4(194): 1-9. View PDF »
Your eyes dart about rhythmically sampling different parts of a scene in little bites. Your memory system papers this over to create a illusion of seamless perception. Let Parr and Friston break it down for you:
Parr, T., & Friston, K. J. (2017). The active construction of the visual world. Neuropsychologia.
For further reading:
Buschman,T.J. and Miller, E.K. (2010) Shifting the Spotlight of Attention: Evidence for Discrete Computations in Cognition. Frontiers in Human Neuroscience. 4(194): 1-9. View PDF »
Buschman, T.J. and Miller, E.K. (2009) Serial, covert, shifts of attention during visual search are reflected by the frontal eye fields and correlated with population oscillations. Neuron, 63: 386-396. View PDF
Miller Lab Alumnus, Wael Asaad, shows that neurons in the prefrontal cortex can figure out which prior events get credit for the consequences of our actions.
Asaad, W. F., Lauro, P. M., Perge, J. A., & Eskandar, E. N. (2017). Prefrontal Neurons Encode a Solution to the Credit-Assignment Problem. Journal of Neuroscience, 37(29), 6995-7007.
More evidence for mixed selectivity. Mixed selectivity is “a neural encoding scheme in which different task variables and behavioral choices are combined indiscriminately in a non-linear fashion within the same population of neurons. This scheme generates a high-dimensional non-linear representational code that allows for a simple linear readout of multiple variables from the same network of neurons” (Fusi et al., 2016). It adds computational horsepower to the brain. In this case, the evidence is from human parietal cortex.
Zhang, C. Y., Aflalo, T., Revechkis, B., Rosario, E. R., Ouellette, D., Pouratian, N., & Andersen, R. A. (2017). Partially Mixed Selectivity in Human Posterior Parietal Association Cortex. Neuron.
For further reading:
Rigotti, M., Barak, O., Warden, M.R., Wang, X., Daw, N.D., Miller, E.K., & Fusi, S. (2013) The importance of mixed selectivity in complex cognitive tasks. Nature, 497, 585-590, doi:10.1038/nature12160. View PDF »
Fusi, S., Miller, E.K., and Rigotti, M. (2016) Why neurons mix: High dimensionality for higher cognition. Current Opinion in Neurobiology. 37:66-74 doi:10.1016/j.conb.2016.01.010. View PDF »
A recent review by the late, great Howard Eichenbaum. You’ll be missed, Howard.
Eichenbaum, H. (2017). Memory: organization and control. Annual review of psychology, 68, 19-45.