Show Tag: efficient-coding

Select Other Tags

The `efficient coding principle' states that a neural ensemble should encode as much information as possible in its response.

Separating visual processing into channels by the kind of feature it is based on is beneficial for efficient coding: feature combinations can be coded combinatorially.

There are very successful solutions to isolated problems in computer vision (CV). These solutions are flat, however in the sense that they are implemented in a single process from feature extraction to information interpretation. A CV system based on such solutions can suffer from redundant computation and coding. Modeling a CV

To estimate optimally, it is necessary to take into account the rate of each stimulus value. This is neglected by the efficient coding approach, which is recognized by the opponents.

The pure efficient coding hypothesis does not take into account noise which may corrupt signals.

The efficient coding hypothesis describes only one aspect of neural sensory processing. It is a good assumption that neurons are used efficiently to code for stimuli, but other requirements may make redundant coding necessary.

The efficient coding hypothesis does not take into account any task neural processing is supposed to accomplish. Some redundancy may make a code more suitable for a particular task. This is true especially when the values being represented are not equally distributed, when there is noise, and when responding correctly to some values yields higher utility than for others.

In an efficient population code, neural responses are statistically independent.

By optimizing sparseness (or coding efficiency) of functions for representing natural images, one can arrive at tuning functions similar to those found in in simple cells. They are

  • spatially localized
  • oriented
  • band-pass filters with different spatial frequencies.

LGN cells respond whitened---ie. efficiently---to natural images, but they respond non-white to white noise, eg. They are thus well-adapted to natural images from the efficient coding point of view.

One hypothesis about early visual processing is that it tries to preserve (and enhance) as much information about the visual stimuli (with as little effort) as possible. Findings about efficiency in visual processing seem to validate this hypothesis.

A complete theory of early visual processing would need to address more aspects than coding efficiency, optimal representation and cleanup. Tasks and implementation would have to be taken into account.

Non-primates often see only one or two, or more than three primary colors. This is probably, because their visual system is used for different tasks (like hunting in the dark). Since efficient coding does not take into account the task, implementation, and base rate, it cannot explain this variability.

Redundancy reduction, predictive coding, efficient coding, sparse coding, and energy minimization are related hypotheses with similar predictions. All these theories are reasonably successful in explaining biological phenomena.