Conversation: LLMs and the what/how loop

In a recent conversation, Unmesh Joshi, Rebecca Parsons, and Martin Fowler discussed the role of large language models (LLMs) in shaping the abstractions within software systems. The trio highlighted the challenge of building systems that can withstand change while managing cognitive load, a critical aspect of software development. They emphasized the importance of mapping the "what" of desired software functionality into the "how" of programming languages, a process that occurs in a feedback loop.

The discussion began by acknowledging that software systems must evolve over time to adapt to new requirements and environments. This necessitates a balance between abstraction and flexibility. Abstraction helps developers manage complexity by focusing on the essential features of a system, while flexibility ensures that the system can adapt to changes. However, striking this balance can be challenging, as abstracting too much may lead to inflexibility, while too little abstraction can result in overwhelming complexity.

To address this challenge, the conversation explored the concept of a "what/how" loop. The "what" represents the desired behavior or functionality of the software, while the "how" refers to the specific implementation details in the programming language. This loop is a continuous process where the "what" informs the "how," and the "how" in turn influences the refinement of the "what." By iterating through this loop, developers can create software that is both abstract enough to manage complexity and flexible enough to adapt to change.

Test-driven development (TDD) was mentioned as a method that operationalizes this loop. In TDD, developers start by defining a test that outlines the desired behavior of a component. This test serves as the "what," and the implementation of the component represents the "how." As the implementation is developed, it may reveal new aspects of the desired behavior, prompting further refinement of the tests. This process reinforces the feedback loop, ensuring that the software's functionality and implementation remain aligned.

However, TDD can be a structured and formal process, which may not always be the most efficient or flexible approach. This is where large language models (LLMs) come into play. LLMs, such as those based on the GPT architecture, can assist developers in exploring the "what/how" loop in a more informal and fluid manner. By leveraging LLMs, developers can generate ideas, test hypotheses, and refine their understanding of the system's requirements and implementation details more quickly and efficiently.

LLMs can aid in several ways. For instance, they can help developers articulate their vision for the software's "what" by generating natural language descriptions of desired functionality. These descriptions can then be used to inform the "how" of the implementation. Additionally, LLMs can suggest alternative approaches to solving problems, enabling developers to explore different paths within the "what/how" loop. This can lead to more innovative and effective solutions that might not have been apparent through traditional methods.

Moreover, LLMs can facilitate communication and collaboration among team members. By generating clear and concise descriptions of the software's requirements and design decisions, LLMs can help bridge gaps between developers, product managers, and other stakeholders. This improved communication can lead to more cohesive and well-aligned software systems.

Despite these benefits, the conversation acknowledged that the integration of LLMs into the software development process is still an evolving field. Developers and organizations must carefully consider the trade-offs between the speed and fluidity offered by LLMs and the need for rigor, reproducibility, and maintainability in software systems. It is essential to develop best practices and guidelines for effectively utilizing LLMs within the context of software development, ensuring that they complement—rather than replace—traditional methods and tools.

In conclusion, the conversation between Unmesh Joshi, Rebecca Parsons, and Martin Fowler highlighted the importance of the "what/how" loop in shaping software abstractions. By operationalizing this loop through methods like test-driven development and exploring it more fluidly with large language models, developers can create systems that are both adaptable and manageable. As LLMs continue to advance, their role in this process will likely become even more significant, offering new opportunities for innovation and efficiency in software development.