Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
hyper-parameter_tuning [2018/03/15 11:34]
admin
hyper-parameter_tuning [2018/10/31 10:52] (current)
admin
Line 298: Line 298:
  
 Our approach combines a novel hierarchical genetic representation scheme that imitates the modularized design pattern commonly adopted by human experts, and an expressive search space that supports complex topologies. Our algorithm efficiently discovers architectures that outperform a large number of manually designed models for image classification,​ obtaining top-1 error of 3.6% on CIFAR-10 and 20.3% when transferred to ImageNet, which is competitive with the best existing neural architecture search approaches. ​ Our approach combines a novel hierarchical genetic representation scheme that imitates the modularized design pattern commonly adopted by human experts, and an expressive search space that supports complex topologies. Our algorithm efficiently discovers architectures that outperform a large number of manually designed models for image classification,​ obtaining top-1 error of 3.6% on CIFAR-10 and 20.3% when transferred to ImageNet, which is competitive with the best existing neural architecture search approaches. ​
 +
 +https://​arxiv.org/​abs/​1803.07055 Simple random search provides a competitive approach to reinforcement learning
 +
 +
 +https://​arxiv.org/​pdf/​1805.07440.pdf AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
 +
 +AlphaX also generates the training date for Meta-DNN. So, the learning of Meta-DNN is end-to-end. In searching for NASNet style architectures,​ AlphaX found several promising architectures with up to 1% higher accuracy than NASNet using only 17 GPUs for 5 days, demonstrating up to 23.5x speedup over the original searching for NASNet that used 500 GPUs in 4 days
 +
 +https://​arxiv.org/​abs/​1808.05377 Neural Architecture Search: A Survey
 +
 +We provide an overview of existing work in this field of research and categorize them according to three dimensions: search space, search strategy, and performance estimation strategy.
 +
 +https://​arxiv.org/​abs/​1809.04270 Rapid Training of Very Large Ensembles of Diverse Neural Networks
 +
 +Our approach captures the structural similarity between members of a neural network ensemble and train it only once. Subsequently,​ this knowledge is transferred to all members of the ensemble using function-preserving transformations. Then, these ensemble networks converge significantly faster as compared to training from scratch.
 +
 +https://​arxiv.org/​abs/​1810.05749v1 Graph HyperNetworks for Neural Architecture Search
 +
 +GHNs model the topology of an architecture and therefore can predict network performance more accurately than regular hypernetworks and premature early stopping. To perform NAS, we randomly sample architectures and use the validation accuracy of networks with GHN generated weights as the surrogate search signal. GHNs are fast -- they can search nearly 10 times faster than other random search methods on CIFAR-10 and ImageNet. ​
 +
 +https://​ai.googleblog.com/​2018/​10/​introducing-adanet-fast-and-flexible.html?​m=1