Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revision Both sides next revision
complex_parameters [2018/04/11 10:35]
admin
complex_parameters [2018/06/12 19:15]
admin
Line 182: Line 182:
  
 We introduce flipout, an efficient method for decorrelating the gradients within a mini-batch by implicitly sampling pseudo-independent weight perturbations for each example. Empirically,​ flipout achieves the ideal linear variance reduction for fully connected networks, convolutional networks, and RNNs. We find significant speedups in training neural networks with multiplicative Gaussian perturbations. We introduce flipout, an efficient method for decorrelating the gradients within a mini-batch by implicitly sampling pseudo-independent weight perturbations for each example. Empirically,​ flipout achieves the ideal linear variance reduction for fully connected networks, convolutional networks, and RNNs. We find significant speedups in training neural networks with multiplicative Gaussian perturbations.
 +
 +https://​eng.uber.com/​differentiable-plasticity/​
 +
 +https://​arxiv.org/​abs/​1711.01297v1 Implicit Weight Uncertainty in Neural Networks
 +
 +http://​mdolab.engin.umich.edu/​sites/​default/​files/​Martins2003CSD.pdf The Complex-Step Derivative Approximation
 +
 +https://​github.com/​facebookresearch/​QuaterNet QuaterNet: A Quaternion-based Recurrent Model for Human Motion
 +
 +
 +