Show Tag: deep-learning

Select Other Tags

Convolutional neural networks make assumptions on the input:

  • stationarity of image statistics (the same statistics are true for all segments of an image),
  • locality of dependencies between pixels.

With a few strong but empirically correct assumptions on the input, CNNs buy us a reduction of the number of parameters and thus better training performance compared to standard feed-forward ANNs.

Advances in computing hardware and fast implementations of the necessary operations have made training of convolutional neural networks on very large data sets practical.

It took Kriszhevsky et al. five to six days to train their network on top-notch hardware available in 2012.

Krizhevsky et al. demonstrate that large, deep convolutional neural networks can have very good object classification performance.

Hinton proposes building deep belief networks by stacking RBMs and training them unsupervised and in ascending order. After that, the network goes into feed-forward mode and backprop can be used to learn the actual task. Thus, some of the problems of backprop are solved by initializing the weights via unsupervised learning.