Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
random_matrix [2018/11/06 09:42]
admin
random_matrix [2018/11/06 11:00] (current)
admin
Line 110: Line 110:
  
 https://​openreview.net/​pdf?​id=SJeFNoRcFQ TRADITIONAL AND HEAVY TAILED SELF REGULARIZATION https://​openreview.net/​pdf?​id=SJeFNoRcFQ TRADITIONAL AND HEAVY TAILED SELF REGULARIZATION
-IN NEURAL NETWORK MODELS+IN NEURAL NETWORK MODELS ​. https://​www.youtube.com/​watch?​v=_Ni5UDrVwYU
  
 https://​arxiv.org/​abs/​1810.01075v1 Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning https://​arxiv.org/​abs/​1810.01075v1 Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning
Line 119: Line 119:
 we introduce a random matrix-based framework to analyze the learning dynamics of a single-layer linear network on a binary classification problem, for data of simultaneously large dimension and size, trained by gradient descent. Our results provide rich insights into common questions in neural nets, such as overfitting,​ early stopping and the initialization of training, thereby opening the door for future studies of more elaborate structures and models appearing in today'​s neural networks. we introduce a random matrix-based framework to analyze the learning dynamics of a single-layer linear network on a binary classification problem, for data of simultaneously large dimension and size, trained by gradient descent. Our results provide rich insights into common questions in neural nets, such as overfitting,​ early stopping and the initialization of training, thereby opening the door for future studies of more elaborate structures and models appearing in today'​s neural networks.
  
 +https://​papers.nips.cc/​paper/​6857-nonlinear-random-matrix-theory-for-deep-learning.pdf Nonlinear random matrix theory for deep learning
 +
 +https://​stats385.github.io/​assets/​lectures/​Understanding_and_improving_deep_learing_with_random_matrix_theory.pdf