Bergan et al. show that interaction with the environment can drive multisensory learning. However, Xu et al. show that multisensory learning can also happen if there is no interaction with the multisensory world.⇒
A simple MLP would probably be able to learn optimal multi-sensory integration via backprop⇒
Using a space-coded approach instead of an MLP for learning multi-sensory integration has benefits:
Mixing Hebbian (unsupervised) learning with feedback can guide the unsupervised learning process in learning interesting, or task-relevant things.⇒
Classical models assume that learning in cortical regions is well described in an unsupervised learning framework while learning in the basal ganglia can be modeled by reinforcement learning.⇒
Representations in the cortex (eg. V1) develop differently depending on the task. This suggests that some sort of feedback signal might be involved and learning in the cortex is not purely unsupervised.⇒
Some task-dependency in representations may arise from embodied learning where actions bias experiences being learned from.
Conversely, the narrow range of disparities reflected in disparaty-selective cells in visual cortex neurons might be due to goal-directed feature learning.⇒
Unsupervised learning models have been extended with aspects of reinforcement learning.⇒
The algorithm presented by Weber and Triesch borrows from SARSA.⇒
SOMs can be used for preprocessing in reinforcement learning, simplifying their high-dimensional input via their winner-take-all characteristics.
However, since standard SOMs do not get any goal-dependent input, they focus on globally strongest features (statistically most predictive latent variables) and under-emphasize features which would be relevant for the task.⇒
The model due to Weber and Triesch combines SOM- or K-Means-like learning of features with prediction error feedback as in reinforcement learning. The model is thus able to learn relevant and disregard irrelevant features.⇒
Fujita presents a supervised ANN model for learning to either generate a continuous time series from an input signal, or to generate a continuous function of the continuous integral of a time series.⇒
Backpropagation was discovered at least four times within one decade.⇒
If we know which kind of output we want to have and if each neuron's output is a smooth function of its input, then the change in weights to get the right output from the input can be computed using calculus.
Following this strategy, we get backpropagation⇒
If we want to learn classification using backprop, we cannot force our network to create binary output because binary output is not a smooth function of the input.
Instead we can let our network learn to output the log probability for each class given the input.⇒
One problem with backpropagation is that one usually starts with small weights which will be far away from optimal weights. Due to the size of the combinatorial space of weights, learning can therefore take a long time. ⇒
Backprop needs a lot of labeled data to learn classification with many classes.⇒
It is unclear how neurons could back-propagate errors in their inputs. Thus, the biological validity of backpropagation is limited⇒
Hinton argues that backpropagation is such a good idea that nature must have found a way to implement it somehow. ⇒
In the wake-sleep algorithm, (at least) two layers of neurons are fully connected to each other.
In the wake phase, the lower level drives the upper layer through the bottom-up recognition weights. The top-down generative weights are trained such that they will generate the current activity in the lower level given the current activity in the output level.
In the sleep phase, the upper layer drives activity in the lower layer through the generative weights and the recognition weights are learned such that they induce the activity in the upper layer given the activity in the lower layer.⇒
Hinton proposes building deep belief networks by stacking RBMs and training them unsupervised and in ascending order. After that, the network goes into feed-forward mode and backprop can be used to learn the actual task. Thus, some of the problems of backprop are solved by initializing the weights via unsupervised learning.⇒
According to Zhang et al., ViSOM (and other unsupervised methods) do not take data labels and their intrinsic structure into account if they are present.⇒
SOMs treat all their input dimensions as observables of some latent variable. It is possible to give data points a dimension containing labels. These labels will not have a greater effect on learning than the other dimensions of the data point. This is especially true if the true labels are not good predictors of the actual latent variable. ⇒
Optimizing (ie. training) an estimator with input data will result in different results depending on the distribution of data points: wherever there is a high density of data points, the optimizer will reduce the error there, possibly incurring greater error where the density of data points is lower.⇒