Show Reference: "Attention as Reward-Driven Optimization of Sensory Processing"

@article{chalk-et-al-2013,
    abstract = {Attention causes diverse changes to visual neuron responses, including alterations in receptive field structure, and firing rates. A common theoretical approach to investigate why sensory neurons behave as they do is based on the efficient coding hypothesis: that sensory processing is optimized toward the statistics of the received input. We extend this approach to account for the influence of task demands, hypothesizing that the brain learns a probabilistic model of both the sensory input and reward received for performing different actions. Attention-dependent changes to neural responses reflect optimization of this internal model to deal with changes in the sensory environment (stimulus statistics) and behavioral demands (reward statistics). We use this framework to construct a simple model of visual processing that is able to replicate a number of attention-dependent changes to the responses of neurons in the midlevel visual cortices. The model is consistent with and provides a normative 
explanation for recent divisive normalization models of attention (Reynolds \& Heeger, 2009).},
    author = {Chalk, Matthew and Murray, Iain and Seri\`{e}s, Peggy},
    citeulike-article-id = {12681404},
    citeulike-linkout-0 = {http://dx.doi.org/10.1162/neco\_a\_00494},
    citeulike-linkout-1 = {http://www.mitpressjournals.org/doi/abs/10.1162/NECO\_a\_00494},
    day = {18},
    doi = {10.1162/neco\_a\_00494},
    journal = {Neural Computation},
    keywords = {ann, attention, divisive-normalization, model, perception},
    month = jun,
    pages = {1--30},
    posted-at = {2013-10-07 14:43:41},
    priority = {2},
    publisher = {MIT Press},
    title = {Attention as Reward-Driven Optimization of Sensory Processing},
    url = {http://dx.doi.org/10.1162/neco\_a\_00494},
    year = {2013}
}

See the CiteULike entry for more info, PDF links, BibTex etc.

Chalk et al. hypothesize that biological cognitive agents learn a generative model of sensory input and rewards for actions.

In Chalk et al.'s model, low-level sensory neurons are responsible for calculating the probabilities of high-level hidden variables given certain features being present or not. Other neurons are then responsible for predicting the rewards of different actions depending on the presumed state of those hidden variables.

In Chalk et al.'s model, neurons update their parameters online, ie. during the task. In one condition of their experiments, only neurons predicting reward are updated, in others, perceptual neurons are updated as well. Reward prediction was better when perceptual responses were tuned as well.

Divisive normalization models have explained how attention can facilitate or suppress some neurons' responses.

Some models view attentional changes of neural responses as the result of Bayesian inference about the world based on changing priors.

Chalk et al. argue that changing the task should not change expectations—change the prior—about the state of the world. Rather, they might change the model of how reward depends on the state of the world.