A Comprehensive Survey of Benchmarks for Improvement of Software's Non-Functional Properties

Created by W.Langdon from gp-bibliography.bib Revision:1.8444

@Article{blot:2025:ACMsurveys,

• author = "Aymeric Blot and Justyna Petke",
• title = "A Comprehensive Survey of Benchmarks for Improvement of Software's Non-Functional Properties",
• journal = "ACM Computing Surveys",
• year = "2025",
• volume = "57",
• number = "7",
• pages = "Article no. 168",
• keywords = "genetic algorithms, genetic programming, genetic improvement, software performance, non-functional properties, benchmark",
• URL = "https://discovery.ucl.ac.uk/id/eprint/10203326/1/main.pdf",
• URL = "https://discovery.ucl.ac.uk/id/eprint/10203326/",
• URL = "https://sciencespo.hal.science/IRISA_SET/hal-04936383v1",
• URL = "https://hal.science/hal-04936383v1/file/blot_csur_2024.pdf",
• DOI = "doi:10.1145/3711119",
• data_url = "https://bloa.github.io/nfunc_survey",
• size = "35 pages",
• abstract = "Despite recent increase in research on improvement of non-functional properties of software, such as energy usage or program size, there is a lack of standard benchmarks for such work. This absence hinders progress in the field, and raises questions about the representativeness of current benchmarks of real-world software. To address these issues and facilitate further research on improvement of non-functional properties of software, we conducted a comprehensive survey on the benchmarks used in the field thus far. We searched five major online repositories of research work, collecting 5499 publications (4066 unique), and systematically identified relevant papers to construct a rich and diverse corpus of 425 relevant studies. We find that execution time is the most frequently improved property in research work (63percent), while multi-objective improvement is rarely considered (7percent). Static approaches for improvement of non-functional software properties are prevalent (51percent), with exploratory approaches (18percent evolutionary and 15percent non-evolutionary) increasingly popular in the last 10 years. Only 39percent of the 425 papers describe work that uses benchmark suites, rather than single software, of those SPEC is most popular (63 papers). We also provide recommendations for future work, noting, for instance, lack of benchmarks for non-functional improvement that covers Python, JavaScript, or mobile devices. All the details regarding the 425 identified papers are available on our dedicated webpage: https://bloa.github.io/nfunc_survey",
• notes = "Also known as \cite{blot:hal-04936383} replaces \cite{blot2022comprehensive}",

}

Genetic Programming entries for Aymeric Blot Justyna Petke

Citations