- author = "Nadia Alshahwan and Mark Harman and Inna Harper and Alexandru Marginean and Shubho Sengupta and Eddy Wang",
- title = "Assured LLM-Based Software Engineering",
- howpublished = "arXiv",
- year = "2024",
- month = "6 " # feb,
- note = "InteNSE 2024 Keynote",
- keywords = "genetic algorithms, genetic programming, genetic improvement, ANN, LLMSE, SBSE, prompt search space, facebook, automatic test oracle, refactoring, APR, searchable prompting language",
- eprint = "2402.04380",
- archiveprefix = "arXiv",
- primaryclass = "cs.SE",
-
URL = "
https://arxiv.org/abs/2402.04380",
- size = "6 pages",
-
abstract = "How can we use Large Language Models (LLMs) to improve
code independently of a human, while ensuring that the
improved code
- does not regress the properties of the original code?
- improves the original in a verifiable and measurable way?
To address this question, we advocate Assured LLM-Based Software Engineering; a generate-and-test approach, inspired by Genetic Improvement. Assured LLMSE applies a series of semantic filters that discard code that fails to meet these twin guarantees. This overcomes the potential problem of LLM's propensity to hallucinate. It allows us to generate code using LLMs, independently of any human. The human plays the role only of final code reviewer, as they would do with code generated by other human engineers.
This paper is an outline of the content of the keynote by Mark Harman at the International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering, Monday 15th April 2024, Lisbon, Portugal",
