Teacher-Guided Fitness Approximation Speeds TinyML Architecture Search

Summary

Expensive evolutionary architecture search often does not require an exact fitness value for every candidate; it mainly needs reliable rankings of which candidates are better. This paper introduces Teacher-Guided Learning NSGA-II (TGL-NSGA-II), a low-fidelity framework for constrained TinyML neural architecture search. A pretrained teacher groups samples by class and difficulty, while each candidate receives a short, capped KD-Lite knowledge-distillation training run and is scored on a separate stratified evaluation set. The resulting score is combined with a Gaussian-process surrogate to decide which candidates receive full evaluation. On keyword spotting and bird-call classification, measured Kendall's tau reached 0.74 and 0.62, above predicted lower bounds of 0.60 and 0.46. Joint stratification reduced proxy-score variance by 41% versus random evaluation, while selective teacher mismatch lowered Kendall's tau to 0.41. Under a constrained budget, TGL-NSGA-II produced the best mean hypervolume and generational distance on keyword spotting, the lowest mean false-positive rate on BirdCLEF, and ran 2.2 times faster than full NSGA-II. The results concern population-level low-fidelity evaluation and do not prove convergence of the complete evolutionary search trajectory.