Designing neural networks by hand is tedious and suboptimal. NAS automates the search: define a search space of architectures, use a controller (evolutionary, bandit, or RL) to propose and evaluate architectures, select the best. NAS has discovered architectures that outperform hand-designed ones.
Search Space
What can vary?
- Number of layers
- Layer type (conv, pooling, skip)
- Layer dimensions (filters, kernel size)
- Skip connections
Controller-Based Search
# Controller: RNN that proposes architectures
# Evaluator: Train proposed architecture on data # Reward: Accuracy on validation set
for epoch in range(num_epochs): # Controller proposes architecture arch = controller.propose()
# Train and evaluate model = build_model(arch) train(model) accuracy = evaluate(model)
# Reward controller for high accuracy controller.reward(accuracy)
Practical Tools
- AutoKeras: AutoML for images and text
- Ray Tune: Hyperparameter tuning and NAS
- ENAS: Efficient NAS using weight sharing
Conclusion
NAS automates architecture design, discovering models that outperform hand-crafted designs. Practical tools make NAS accessible for practitioners. Understanding NAS informs the future of automated machine learning. Next: optimization techniques—how to train models efficiently.