CE-NAS: An End-to-End Carbon-Efficient Neural Architecture Search Framework

This work presents a novel approach to neural architecture search (NAS) that aims to increase carbon efficiency for the model design process. The proposed framework CE-NAS addresses the key challenge of high carbon cost associated with NAS by exploring the carbon emission variations of energy and energy differences of different NAS algorithms. At the high level, CE-NAS leverages a reinforcement-learning agent to dynamically adjust GPU resources based on carbon intensity, predicted by a time-series transformer, to balance energy-efficient sampling and energy-intensive evaluation tasks. Furthermore, CE-NAS leverages a recently proposed multi-objective optimizer to effectively reduce the NAS search space. We demonstrate the efficacy of CE-NAS in lowering carbon emissions while achieving SOTA results for both NAS datasets and open-domain NAS tasks. For example, on the HW-NasBench dataset, CE-NAS reduces carbon emissions by up to 7.22X while maintaining a search efficiency comparable to vanilla NAS. For open-domain NAS tasks, CE-NAS achieves SOTA results with 97.35% top-1 accuracy on CIFAR-10 with only 1.68M parameters and a carbon consumption of 38.53 lbs of CO2. On ImageNet, our searched model achieves 80.6% top-1 accuracy with a 0.78 ms TensorRT latency using FP16 on NVIDIA V100, consuming only 909.86 lbs of CO2, making it comparable to other one-shot-based NAS baselines.

Paper

References (100)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer QYii9/10 · confidence 4/52024-07-07

Summary

The paper proposes a carbon-efficient neural architecture search (NAS), CE-NAS, to reduce the carbon emitted during the NAS process by dynamically allocate GPU resources based on the predicted future carbon intensity. CE-NAS leverages reinforcement learning for generating the probability of choosing which model of NAS (one-shot or vanilla) based on the predicted carbon intensity by a time-series transformer. One-shot model would be chosen to run in high carbon intensity periods and vanilla NAS would be chosen otherwise. Alongside emitting less carbon, CE-NAS achieves a competitive accuracy against state-of-the art models for CIFAR-10 and ImageNet datasets

Strengths

1. Focusing on reducing the carbon emitted by ML tasks that has great environmental impact. 2. Designed a high-accuracy carbon intensity prediction model compared to previous models. 3. Using various baseline models in the experiments. 4. Leveraging real data for the experiments makes it more applicable to real-world scenarios.

Weaknesses

1. The experiments are only conducted in California, it would be good if authors show their results for multiple locations. 2. The baseline models for comparing the proposed time-series carbon intensity predictor could use one more model proposed by Zhang et al. (Title: A GNN-based Day Ahead Carbon Intensity Forecasting Model for Cross-Border Power Grids) which is also a strong model for prediction. 3. The overhead of switching between the models has not been considered considering carbon intensity variations.

Questions

1. How would the CE-NAS perform if we have a deadline for used GPU hours? e.g., we can only use 2k hours of GPU and we can't exceed that amount. 2. How robust is CE-NAS in regions that we have high carbon intensity prediction errors?

Rating

9

Confidence

4

Soundness

4

Presentation

3

Contribution

4

Limitations

1. The authors can discuss the scalability limitations of their model when the search space becomes more complex

Reviewer d3FP7/10 · confidence 5/52024-07-11

Summary

Neural architecture search (NAS) is known to be extremely compute intensive, which also increases the corresponding energy consumption and the carbon costs due to energy production. In this work, the authors attempt to reduce the carbon cost of performing NAS. To this end, they propose a reinforcement learning (RL) agent that switches between energy intensive evaluation of architectures (standard NAS) and energy-efficient sampling (few-shot NAS), during high carbon intensive and low carbon intensity regimes, respectively. To predict the carbon intensity for efficiency scheduling of the two NAS regimes, this work uses a time series transformer which predicts the carbon intensity of energy production using historical information. Using a combination of RL agent for scheduling based on the time series transformer based carbon intensity prediction, the work shows considerable reduction in the overall carbon costs of NAS while still achieving architectures that are comparable to SOTA.

Strengths

* Carbon costs of developing deep learning models is not sufficiently addressed in the literature. This work takes up a carbon efficient approach to NAS, and presents a clear motivation for looking at costs such as energy consumption and carbon emissions. * The proposed end-to-end carbon efficient NAS that performs considerably well across benchmark and open domain tasks. * The strategy of balancing between energy efficient sampling of architecture spaces (using few-shot NAS like approaches) and more energy intensive evaluations to explore search space better, during low and high carbon intensity times, respectively, is a strong contribution of this work. That a trade-off between these strategies is formulated using as a multi-objective optimization that aims to optimize for carbon costs is an elegant formulation of NAS than only one-shot or vanilla NAS. * The time series predictor, although is a straightforward use of an existing transformer model, achieves good predictive performance on carbon intensity predictions. * Experimental evaluation is sufficiently convincing. The use of real carbon intensity data from electricitymaps also provides useful insights into the real-world applications of this method.

Weaknesses

* **Spatial carbon intensity variations**: This work mainly addresses the temporal variations of carbon intensity. While it can be argued that it is the first step, and also an easier mode of control, it would also be interesting to see how the scheduling can take different geographical locations into account. We can assume that large scale DL compute happens on cloud instances, and this could improve the broader usefulness of this proposed method. For instance, could the scheduler take the carbon intensity of the three regions considered in this work, and further optimize the carbon costs, as is commonly done in carbon-aware scheduling literature [4]? * **NasBench101**: In L.256 authors state that NasBench101 does not provide accuracy. I am not sure what they mean by this. NasBench101 does provide training and validation accuracy for all the 423k architectures considered in the search space. Clarification can be useful. * **Beyond CNNs**: This is a general critique of most NAS works, as we are exploring only CNN architectures that are by now very efficient. Why did the authors not consider other open domains? For instance, something that takes a hybrid space of CNNs and transformers [1] into account? * **Dominance number o(a)**: If _a_ is a unique architecture (L 175), what are the _number of samples that dominate_ a given architecture _a_ (L 178)? Or is _a_ a proxy for a sub-space that is being explored? This requires some clarification. And also, on where exactly is the dominance number being used? Is it within the MOO algorithms? #### Other comments * L 47: Authors seem to allude that low latency is always corresponding to low energy/carbon costs. This is not always the case as has been shown in many works [5,3] * Authors are missing some key references pertaining to carbon-aware scheduling [4] and energy/carbon efficient NAS [2,3]. While [2,3] are not exactly efficient NAS they do grapple with some of the concerns expressed in this work. These suggestions for missing reviews are only samples and not comprehensive; I encourage the authors to explore further starting from these papers. #### References [1] Xu, Shuying, and Hongyan Quan. "ECT-NAS: Searching efficient CNN-transformers architecture for medical image segmentation." In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1601-1604. IEEE, 2021. [2] Dou, S., Jiang, X., Zhao, C. R., & Li, D. (2023). EA-HAS-bench: Energy-aware hyperparameter and architecture search benchmark. In The Eleventh International Conference on Learning Representations. [3] Bakhtiarifard, P., Igel, C., & Selvan, R. (2024, April). EC-NAS: Energy consumption aware tabular benchmarks for neural architecture search. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 5660-5664). IEEE. [4] Radovanović, A., Koningstein, R., Schneider, I., Chen, B., Duarte, A., Roy, B., ... & Cirne, W. (2022). Carbon-aware computing for datacenters. IEEE Transactions on Power Systems, 38(2), 1270-1280. [5] Henderson, P., Hu, J., Romoff, J., Brunskill, E., Jurafsky, D., & Pineau, J. (2020). Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research, 21(248), 1-43.

Questions

See weaknesses.

Rating

7

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors do no discuss limitations of the work. I would encourage them to discuss the effects of the carbon intensity predictor. It could be done at two steps: Firstly, dependent on the accuracy. What if the predicted carbon intensity is inaccurate? What is the impact of scheduling the energy intensive step during carbon intensive times? Second, at a higher level. Let us assume everyone starts doing carbon efficient NAS (or even computations in general) then there is a possible risk of rebound effect which might then go on to increase the carbon footprint paradoxically.

Reviewer g77N8/10 · confidence 4/52024-07-11

Summary

This paper introduces a NAS framework, named CE-NAS, which focuses on reducing the carbon footprint of architecture search. To do this, CE-NAS decides when to use which NAS evaluation method depending the current carbon intensity, and how to allocate available GPU resources. Their carbon forecasting model also outperforms previous models for predicting carbon intensity. In experiments they show that CE-NAS outperforms other methods, like vanilla NAS and one-shot NAS, given a fixed carbon budget. Sampling and evaluation are decoupled in the framework. Sampling is done using a BO algorithm which generates new architectures. These are then added to a queue of architectures that will be put on the available GPU resources for full training. A proxy evaluation is also done before this using the supernet constructed from the most promising region of the search space, according to a multi-objective optimiser. This approximate performance is used by the BO sampling algorithm. During periods of high carbon emissions, the sampling is the priority, and in periods of low carbon emissions, the full training is prioritised. In the latter, it also updates the promising search space region used to construct the supernet. Every hour, the priority is reassigned and GPU resources are reallocated.

Strengths

**Originality** Constructing a framework which puts carbon efficiency at its core is an original idea that goes further than previous work. The paper also motivates this idea very well. **Quality** The proposed solution is effective at providing models on the Pareto frontier of the optimised objectives. It consistently returns architectures that have either better performance or lower cost/parameter count and these choices are produced in the same search, allowing the user to choose between them at test time. Additionally, the carbon cost of the framework lies between vanilla and one-shot methods while being able to outperform both in terms of accuracy. One of my concerns was that this paper presents a complex framework with many moving parts, including several trained networks. Fortunately there are good ablations of these components in the appendix, accounting for their contribution to the whole. **Clarity** The paper is clear and well presented overall. There are some minor issues on descriptive clarity which I have listed in the weaknesses section below. **Significance** I think the contribution is significant, but have some minor concerns whether the scope of this paper fits best within the NeurIPS conference. Would this paper that focuses on a complex and practical framework perhaps be better suited to a venue that focused on the practical aspects on running NAS, such as the AutoML conference? On the other hand I think the paper tackles important questions for the NAS community which should be highlighted at a venue that gives it more impact. This concern does not overall impact my rating but I would like to hear the authors thoughts on it.

Weaknesses

My main concern is that the paper does not make it clear enough how a practitioner would use the framework themselves. The critical point is that real-time carbon emission data needs to be available locally and there is no discussion of how realistic this is or any suggestion for where to find it. Without this aspect of carbon efficiency, there is still a decent contribution of a framework that seems to outperform other methods for multi-objective NAS but I would like more discussion on how I could realistically use the full framework myself. Minor comments: * Figure 1 needs a more descriptive caption that clarifies the diagram and how CE-NAS works on a high level. The diagram is complex and hard to parse at the moment. * The CIFAR10 performance differs between the abstract (97.35%) and the introduction (97.84%). In the end of the introduction, in the list of contributions, it is again 97.35%. This seems to be the two different models from the Pareto frontier. I suggest either explaining that you find both or be consistent with which you highlight in the text. * Is there a citation for the claim on lines 97-99? *A recent NAS work on transformer revealed that their comprehensive architecture search used 979 million training steps, totaling 274,120 GPU hours and resulting in 626,155 pounds of CO2 emissions.* * When reading, I was confused by both the relative carbon emission and hypervolume metrics. I think there should be a short explanation in the main paper along with the pointer to the full description in the appendix. This pointer is currently in section 4.2.1 but hypervolume is first mentioned in 3.5.1 which causes confusion for a while. * The caption for Figure 2 states that CE-NAS has the lowest relative carbon emission but this seems to be wrong. One-shot LaMOO has a lower CO_2 number in (a) and both One-shot LaMOO and Heuristic have lower numbers in (b). * On line 263: *“To date, there are three popular open-source NAS datasets, NasBench101 [77], HW-NAS-BENCH [25], and NasBench301 [80]”* There are other popular ones, e.g. DARTS, and I think a more appropriate term here is NAS benchmarks instead of datasets. * Table 2, row CE-Net-P1 says hybird instead of hybrid. * In Table 3, MnasNet achieves a TensorRT Latency of 0.53, which is lower than CE-Net-G1 and should be the bolded number (unless there was a typo). * Figure 9 of the appendix needs a caption that better explains it.

Questions

How easy is it in practice to access live information about carbon efficiency? This is key to the method and reducing carbon emissions from the NAS process, but as a practitioner I would not know how to get this information. If there is a straightforward way to get this data from, e.g. ElectricityMap, I would like to see that described in the paper or pointed to in the appendix. Additionally, it would be good to have a discussion on the availability of carbon emissions data across different countries/regions as I imagine it will be unavailable in many places.

Rating

8

Confidence

4

Soundness

4

Presentation

3

Contribution

4

Limitations

There is no clear section that discusses limitations, and I think this is where my suggestion on discussing the availability of carbon emissions data could be put.

Reviewer CjFX5/10 · confidence 5/52024-07-14

Summary

The paper introduces CE-NAS, a novel framework for neural architecture search that prioritizes carbon efficiency in the model design process. It addresses the high carbon cost associated with NAS by dynamically adjusting GPU resources based on predicted carbon intensity and search results. The framework integrates a reinforcement-learning agent, a time-series transformer for carbon intensity prediction, and a multi-objective optimizer to balance energy-efficient sampling with energy-intensive evaluation tasks. Experiments demonstrate CE-NAS's ability to reduce carbon emissions significantly while achieving good results on various NAS datasets and tasks.

Strengths

* CE-NAS presents a unique and timely approach to NAS that considers carbon efficiency, aligning with broader environmental sustainability goals. * The integration of a reinforcement-learning agent, time-series transformer, and multi-objective optimizer provides a comprehensive solution for carbon-aware NAS. * The framework's effectiveness on various datasets suggests that it could be widely applicable to different NAS scenarios.

Weaknesses

* Experimental comparisons are focused on traditional and one-shot NAS methods. However, zero-shot NAS methods are not mentioned in the paper. Some recent zero-shot NAS methods have achieved better results and higher efficiency. * The integration of multiple components introduces complexity in implementation.

Questions

Please describe the advantages of CE-NAS when compared to zero-shot NAS methods.

Rating

5

Confidence

5

Soundness

3

Presentation

3

Contribution

2

Limitations

None

Authorsrebuttal2024-08-13

Thank you for your kind feedback. We apologize for the code being inaccessible earlier. We have now made it public, and we will organize and release the code on GitHub once the paper is accepted.

Reviewer QYii2024-08-12

Thank you authors for addressing the mentioned concerns.

Reviewer d3FP2024-08-12

Response to author rebuttal

The authors have addressed most of my concerns in their rebuttal with appropriate clarifications. I urge them include these clarifications in the final version of the paper. I will raise my score to Accept.

Reviewer g77N2024-08-12

Response to authors

Thanks to the authors for their detailed response. After reading this and the other reviews and responses, I think the paper will be improved. On the assumption that these changes will be made for the final version, have decided to upgrade my score.

Reviewer CjFX2024-08-13

Thanks for the efforts.

Thanks for the response. I acknowledge that zero-shot NAS has its limitations and agree that integrating it into the CE-NAS framework is a promising direction. The work's novel combination with carbon efficiency indeed offers insights, and as such, I will increase my score. I suggest the authors to include a discussion on zero-shot NAS in the related work section to ensure the completeness of the paper. Additionally, the provided code in the Supp. is currently inaccessible, requiring an application for access. I kindly urge the authors to make the code publicly available.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC