Neural Architecture Search as Program Transformation Exploration

Created by W.Langdon from gp-bibliography.bib Revision:1.8051

@Article{Turner:2024:CACM,

• author = "Jack Turner and Elliot J. Crowley and Michael F. P. O'Boyle",
• title = "Neural Architecture Search as Program Transformation Exploration",
• journal = "Communications of the ACM",
• year = "2024",
• volume = "67",
• number = "10",
• pages = "92--100",
• month = oct,
• keywords = "genetic algorithms, genetic programming, genetic improvement, Artificial Intelligence and Machine Learning, deep neural networks (DNNs), ANN, NAS, LLM, TVM, CIFAR-10, Fisher Potential, parallel computing, GPU, BlockSwap",
• publisher = "ACM",
• address = "New York, NY, USA",
• ISSN = "0001-0782",
• size = "9 pages",
• abstract = "representational capacity, new tensor convolutions, reduce inference time three fold",
• notes = "Not GP but apply automatic search to get better program.

  Loop interchange, interchange, bottleneck

  'bottleneck(B). This is shown in row 4 of Figure 1. From a NAS point of view, this is a standard operation reducing complexity with a minimal effect on representational capacity. From a program transformation point of view, this is illegal as the computed values are changed.'

  search 'cell design space' polyhedral Program Transformations (Strip-Mining. Tiling. interchange)

  Legality 'lexicographical ordering of the iteration instances is preserved' eh? must be? Is guaranteed? How? By whome? Does it matter in ANN where data flows are layer to layer?

  'Fisher Potential as a Legality Check' (Turner, J. et al. BlockSwap: Fisher-guided block substitution for network compression on a budget. ICLR 2020) https://www.pure.ed.ac.uk/ws/portalfiles/portal/132577775/Blockswap_Fisher_Guided_TURNER_DOA01012020_AFV.pdf

  'Fisher Potential of a proposed network architecture is the sum of the Fisher Information for each layer ...' 'Many good networks are also discarded' 'the space is large and densely populated'

  ARM A57, Nvidia 128-Core Maxwell Jetson Nano, Intel Core i7, Nvidia 1080Ti. ResNet-34, ResNext-29-2x64d, DenseNet-161

  Only 1000 points tested (in five minutes)? How is search guided?

  FBNet

  ImageNet

  Pareto optimal 'The original version of this paper was published in ASPLOS 2021'",

}

Genetic Programming entries for Jack Turner Elliot J Crowley Michael F P O'Boyle

Citations