Show Reference: "Deep Hierarchies in the Primate Visual Cortex: What Can We Learn For Computer Vision?"

Deep Hierarchies in the Primate Visual Cortex: What Can We Learn For Computer Vision? Vol. PP, No. 99. (2012), pp. 1-1, doi:10.1109/tpami.2012.272 by Norbert Kr├╝ger, Peter Janssen, Sinan Kalkan, et al.
    abstract = {Computational modeling of the primate visual system yields insights of potential relevance to some of the challenges that computer vision is facing, such as object recognition and categorization, motion detection and activity recognition or vision-based navigation and manipulation. This article reviews some functional principles and structures that are generally thought to underlie the primate visual cortex, and attempts to extract biological principles that could further advance computer vision research. Organized for a computer vision audience, we present functional principles of the processing hierarchies present in the primate visual system considering recent discoveries in neurophysiology. The hierarchal processing in the primate visual system is characterized by a sequence of different levels of processing (in the order of ten) that constitute a deep hierarchy in contrast to the flat vision architectures predominantly used in today's mainstream computer vision. We hope that the functional description of the deep hierarchies realized in the primate visual system provides valuable insights for the design of computer vision algorithms, fostering increasingly productive interaction between biological and computer vision research.},
    author = {Kr\"{u}ger, Norbert and Janssen, Peter and Kalkan, Sinan and Lappe, Markus and Leonardis, Ale\v{s} and Piater, Justus and Rodriguez-S\'{a}nchez, Antonio J. and Wiskott, Laurenz},
    doi = {10.1109/tpami.2012.272},
    institution = {University of Southern Denmark},
    issn = {0162-8828},
    keywords = {bottom-up, brain, cortex, model, top-down, visual},
    number = {99},
    pages = {1},
    posted-at = {2013-03-19 08:28:37},
    priority = {2},
    publisher = {IEEE},
    title = {Deep Hierarchies in the Primate Visual Cortex: What Can We Learn For Computer Vision?},
    url = {},
    volume = {PP},
    year = {2012}

See the CiteULike entry for more info, PDF links, BibTex etc.

LIP is retinotopic and involved in gaze shifts.

The visual cortex is hierarchically organized.

55% of neocortex are visual.

Neurons at low stages in the hierarchy of visual processing extract simple, localized features.

Color opponency and center-surround oppenency arise first in LGN.

Hierarchical architectures for information processing have the benefit of being able to re-use processing primitives for different purposes.

The visual system (of primates) contains a number of channels for different types of visual information:

  • color
  • shape
  • motion
  • texture
  • 3D

Separating visual processing into channels by the kind of feature it is based on is beneficial for efficient coding: feature combinations can be coded combinatorially.

There are very successful solutions to isolated problems in computer vision (CV). These solutions are flat, however in the sense that they are implemented in a single process from feature extraction to information interpretation. A CV system based on such solutions can suffer from redundant computation and coding. Modeling a CV

Nearly all projections from the retinae go through LGN.

All visual areas from V1 to V2 and MT are retinotopic.

The ventral pathway of visual processing is weakly retinotopically organized.

The complexity of features (or combinations of features) neurons in the ventral pathway react to increases to object level. Most neurons react to feature combinations which are below object level, however.

The dorsal pathway of visual processing consists of areas MST (motion area), and visual areas in the posterior parietal cortex (PPC).

The complexity of motion patterns neurons in the dorsal pathway are responsive to increases along the pathway. This is similar to neurons in the ventral pathway which are responsive to progressively more complex feature combinations.

Receptive fields in the dorsal pathway of visual processing are less retinotopic and more head-centered.

Parvocellular ganglion cells are color sensitive, have small receptive fields and are focused on foveal vision.

Magnocellular ganglion cells have lower spatial and higher temporal resolution than parvocellular cells.

There are shortcuts between the levels of visual processing in the visual cortex.

Certain neurons in V1 are sensitive to simple features:

  • edges,
  • gratings,
  • line endings,
  • motion,
  • color,
  • disparity

Simple cells are sensitive to the phase of gratings, whereas complex cells are not and have larger receptive fields.

Some cells in V1 are sensitive to binocular disparity.

LIP has been suggested to contain a saliency map of the visual field, to guide visual attention, and to decide about saccades.

Receptive fields in some LIP neurons shift just before a saccade to where their usual receptive field will be after the saccade.