From idea to tool: How we developed a code analysis tool in 3 hours using an AI model#

We wanted to know how far you can really get with AI-supported development. The result: a tool that analyses 1.2 million lines of Java code – developed in just 3 hours using a language model.

This article shows what worked and where the limitations lie.

What was the goal?#

We wanted to develop a code analysis tool for Java projects – with realistic complexity. The questions we asked ourselves were: How does working with a language model work as complexity increases? What structures are necessary? And where are the practical limitations?

What can the tool do?#

The tool analyses Java projects on several levels:

• File level: Understanding the business logic, technical details and interfaces of each individual file
• Package level: Recognise relationships between packages (e.g. assignment to architecture layers)
• Module level: Capture the structure of larger modules
• System level: Describe the overall architecture of the project

The tool performs four specialised analyses for each Java file. The results are output in a readable form – as structured descriptions, not as raw data.

How was it developed?#

The process ran in two phases:

Phase 1 – Clarify requirements (1 hour): We worked out the requirements in dialogue with various language models. The model asked questions, we provided answers. It was important to actively steer clear of overly complex solutions, as language models tend to over-engineer.

Phase 2 – Generate code (2 hours): Implementation took place in 4-5 iterations. The process: first functional requirements, then user interface, then technology selection, then architecture. No further revision was necessary.

Total effort: 3 hours spread over several days.

Result: 5,000-6,000 lines of Python code in 6 files.

Why did it work so well?#

Because we invested a lot of time in clear requirements.

The more precise the description of what the tool should be able to do, the better the code generated. This clarity comes from dialogue – the language model can uncover gaps by asking specific questions. This makes the description consistent and complete.

A second factor: the consistent application of the ‘keep it small and simple’ principle. Language models tend to suggest complex solutions. Simplicity must be actively demanded.

Important insights#

1. Good descriptions are the key to success#

Language model-supported development works with precise descriptions. The generated code is only as good as the requirements that are formulated.

This clarity can be achieved in dialogue with the model. It asks questions such as: ‘What should happen if…?’ or ‘What is the priority of…?’ This results in complete descriptions.

2. Limitations lie in the code length#

Language models reliably generate code up to about 1,000-1,500 lines per component. Beyond that, the quality decreases.

This means that the architecture must be planned in such a way that individual parts do not exceed this limit. An understanding of software structure remains necessary – not in order to program yourself, but to specify meaningful divisions.

3. Actively manage complexity#

Language models often overestimate their capabilities. They suggest complex solutions where simpler ones would suffice.

The consequence: you have to question the choice of technical solutions. ‘Is there a simpler solution?’ is a good standard question to ask in dialogue, for example.

4. The role is shifting#

Developers are becoming architects and requirements clarifiers. The actual programming is delegated to the model.

This enables subject matter experts without in-depth programming knowledge to build specialised tools for their domain. This is very valuable for exploratory projects or internal tools.

What can others learn from this?#

• Invest time in clear requirements – this saves correction effort later
• Use dialogue: let the language model ask questions
• Actively demand simple solutions, otherwise it will become too complex
• Plan the architecture so that components remain under 1,500 lines – preferably under 1,000
• Validate regularly: is the generated code still on the right track?

Conclusion#

✔ A tool developed in 3 hours analysed a project with 1.2 million lines of code

✔ Language model-based development is particularly suitable for prototypes and exploratory projects

✔ Clear requirements and simple structures are the key to success

Important clarification: This was an experiment, not a production-ready system. The question was: How far can you get? The answer: Further than expected, but with clear limits.

This article is part of a series on development with language models. The focus is on what works and what doesn’t – not on perfect solutions.