LLM-assisted software development: A series of experiments on possibilities and limitations#

From the democratisation of earlier IT tools to the democratisation of code generation

Digital music production, photography and video editing democratised studios and darkrooms. Content management systems made web development accessible. The possibilities of LLM-supported software development point in part to a comparable democratisation. The central question: Where are the practical limits of these new possibilities – and what expertise remains essential?

To explore these questions, a series of experiments were conducted. Tools of varying complexity were developed, ranging from simple interfaces to multi-agent systems and code analysis tools. The results document both interesting possibilities and clear limitations.

Results: Development times ranging from 90 minutes to several days#

Development times varied depending on the complexity of the project:

A chat interface with history function: 90 minutes for 800 lines of code
A code analysis tool that hierarchically analyses 1.2 million lines of Java: 3 hours for 5,000-6,000 lines
A presentation tool with two-agent architecture: 2 hours of planning plus 30-60 minutes of coding for 3,000 lines
A document translation system with context management: 6-7 hours for 12,000 lines (initially estimated at 4-6 weeks)

The time savings range from hours to days compared to traditional development. All of the tools developed proved useful for their respective tasks.

Specification quality as a success factor#

Across all projects, the quality of the specification proved to be crucial. The clearer the functional requirements, technical dependencies and architectural decisions were worked out and clarified in advance, the more error-free the implementation was. An investment of 20 to 90 minutes in a structured specification significantly reduced the implementation effort.

The depth of the specification is important: it should cover the exact design of the user interface down to individual elements, name the technologies to be used and define the desired architecture. This precision enables the LLM to implement the specification much more accurately than vague instructions.

Where the methodology reaches its limits#

The experiments indicate some limits:

Approximately 1,000-1,500 lines of code per file should not be exceeded in order to maintain maintainability for LLMs.
With a good specification, up to 12,000 lines of code can be controlled for such applications.
With additional architecture structuring, up to approximately 20,000 lines can be achieved.
How to work with larger programmes beyond this still needs to be explored.

Currently, a sound understanding of software architecture is still essential for successful LLM-supported development. IT professionals with experience in system architectures can currently carry out such developments more easily. However, it is to be expected that these opportunities will also become accessible to other groups of people in the future – similar to previous waves of democratisation.

The concept of ‘casual code’#

Reduced development times are changing the nature of software: code is no longer exclusively a carefully maintained artefact, but is increasingly taking on transient and ephemeral aspects. Code can be created for use over just a few hours, days or weeks – as a disposable prototype or experimental tool.

At the same time, the increased use of short-lived code components poses new challenges for established operating processes. While traditional quality assurance is designed for stable, long-term systems, new approaches for dynamic code inventories must now be developed. It is therefore advisable to choose technologies whose behaviour in operation is already known. Specialised deployment logic is also required that can efficiently handle many small, short-lived services – without neglecting the necessary quality assurance.

Areas of application and limitations#

This methodology is suitable for:

Rapid prototyping and feasibility studies
Internal tools and analysis tools
Exploratory development and didactic scenarios
Software investigations
Demonstrators using interfaces

and is less suitable for:

Safety-critical production systems
High-quality production software with strict quality requirements
Complex systems with extensive dependencies

Implications for everyday IT#

The possibilities observed raise many further questions:

How can rapid prototyping be integrated into existing development processes?
Which qualifications will become more important? (Specification, understanding of architecture, knowledge of technology) Which will become less important? (Detailed implementation, knowledge of syntax)
How should the increasing amount of ‘casual code’ be handled?
Which governance guidelines apply to LLM-developed code?
Which deployment strategies are appropriate for many small, short-lived tools?

Conclusion#

LLM-supported development does not replace professional software development – it complements it for exploratory phases and rapid prototyping. Democratisation follows historical patterns of enabling simple to more complex scenarios. This simultaneously expands the circle of those who can develop functional tools.

We want your perspective: What has been your experience with LLM-assisted development? Where do you see the greatest opportunities – and the most critical challenges?

The series focuses on transferable patterns and methodological limitations, not on the tools developed themselves.