Deep Neural Network Patterns In Training And Transfer Learning
Deep neural nets follow predictable training patterns & can transfer learning between architectures. Research analyzes impact of data distribution, network width & hyperparameters on training dynamics.
This is a Plain English Papers summary of a research paper called Deep Neural Networks Follow Predictable Training Patterns and Can Transfer Learning Between Different Architectures. If you like these kinds of analysis, you should join AImodels.fyi or follow us on Twitter. Overview Research examines training dynamics of deep linear neural networks from random initialization Analyzes impact of data distribution, network width, depth, and hyperparameters Focuses on transfer learning capabilities between different architectures Introduces novel theoretical framework for understanding ne...