Original Paper: https://arxiv.org/abs/2408.02479
By: Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, Huaming Chen
Abstract:
With the rise of large language models (LLMs), researchers are increasingly exploring their applications in var ious vertical domains, such as software engineering. LLMs have achieved remarkable success in areas including code generation and vulnerability detection. However, they also exhibit numerous limitations and shortcomings. LLM-based agents, a novel tech nology with the potential for Artificial General Intelligence (AGI), combine LLMs as the core for decision-making and action-taking, addressing some of the inherent limitations of LLMs such as lack of autonomy and self-improvement. Despite numerous studies and surveys exploring the possibility of using LLMs in software engineering, it lacks a clear distinction between LLMs and LLM based agents. It is still in its early stage for a unified standard and benchmarking to qualify an LLM solution as an LLM-based agent in its domain. In this survey, we broadly investigate the current practice and solutions for LLMs and LLM-based agents for software engineering. In particular we summarise six key topics: requirement engineering, code generation, autonomous decision-making, software design, test generation, and software maintenance. We review and differentiate the work of LLMs and LLM-based agents from these six topics, examining their differences and similarities in tasks, benchmarks, and evaluation metrics. Finally, we discuss the models and benchmarks used, providing a comprehensive analysis of their applications and effectiveness in software engineering. We anticipate this work will shed some lights on pushing the boundaries of LLM-based agents in software engineering for future research.
In the ever-evolving landscape of software engineering, the integration of artificial intelligence has brought forth unprecedented advancements. Large Language Models (LLMs) and LLM-based agents have emerged as powerful tools capable of revolutionizing various facets of software engineering. From automating code generation to enhancing software security, these models are setting new benchmarks for efficiency and innovation. This blog post delves into the applications, methodologies, and implications of LLMs and LLM-based agents in software engineering.
LLMs have shown remarkable capabilities in automating code generation, debugging, and even project documentation. Tools like OpenAI's Codex and GitHub Copilot are prime examples, integrating LLMs to provide real-time code suggestions and automate repetitive tasks. Studies show that developers using GitHub Copilot complete coding tasks 55.8% faster, demonstrating the significant impact of LLMs on productivity.