LLM coding experiment: Document translation system#

Specification quality as an important factor – and the limits of LLM-based development

As part of my series of experiments on LLM-based software development, I developed a web-based document translation system. The project aimed to answer two questions: How far can you get with clear specifications for more complex applications? And what are the practical limitations of LLM-supported coding?

The developed system#

The tool translates documents of any size using local LLMs via OpenAI-compatible interfaces. It processes various formats (PDF, Word, text, Markdown) and uses a multi-stage process: Documents are converted into a uniform Markdown format, divided into processable segments through intelligent chunking, translated in parallel and reassembled. Particular challenges lay in preserving the document structure, maintaining consistency across chunk boundaries and handling complex elements such as tables.

The technical implementation was carried out using Python and Gradio, comprises around 12,000 lines of code in 40 files, and uses asynchronous processing for parallel API calls. Context management is used to ensure translation consistency, passing glossaries, previously translated key terms, and summaries of previous chunks to the LLM.

Development process and time required#

The total effort was 6-7 hours, spread over three main iterations. The initial functional specification was developed in about 30-60 minutes with the LLM, followed by code generation in large steps and iterative corrections. Another 60 minutes were spent on documentation and deployment. This contrasts significantly with the 4-6 weeks originally estimated for manual development.

Various LLMs were used during development, and the system runs on Mistral Small 2506 in productive operation. The tool is now in productive use and has expanded from 6 to 10 supported languages.

Key methodological findings#

1. Specification quality proved to be crucial#

The requirements for specifications for LLM-supported coding proved to be high. The more detailed the functional requirements and technical dependencies were worked out in advance, the more error-free the implementation was. Interaction with LLMs led to details and complications being overlooked – but it was precisely in these areas that errors occurred.

2. Explicit technical dependencies#

The specification in this project was functionally clear, but not yet fully thought through in terms of technical implementation. Important technical aspects such as the processing pipeline or the chunking procedure had to be clarified iteratively. Experience showed that technical dependencies and difficult aspects of functionality should be described as clearly as the functional requirements themselves.

3. LLMs tend to overengineer#

In problem areas that appear simple, LLMs often proposed complicated solutions. Initially, a format-specific processing pipeline was implemented, which was later replaced by a uniform Markdown-based pipeline – a significantly simpler and more maintainable approach. The complicated but clear path did not always prove to be the best. Actively demanding simple solutions and carefully weighing alternatives proved helpful.

4. Limitations of the methodology become apparent#

With around 12,000 lines of code, this project already reached the limits of what is possible for LLM-supported coding with a ‘good but not exceptionally high-quality’ specification. For larger projects, clearer and more systematic specification processes were found to be necessary, covering functional descriptions as well as architecture and detailed technical concepts.

Transferable insights#

Investing in a high-quality specification paid off – both in terms of development time and code quality. The dramatic time savings (6-7 hours instead of several weeks) were only possible because the requirements had been clarified in advance. At the same time, it became clear that the role of the developers had shifted: from code creation to architectural decisions, to critical evaluation of LLM proposals and to quality assurance of the specification.

Subsequent projects showed that an even greater investment in truly clear specifications makes sense – both for functional descriptions and for architecture and detailed technical implementation concepts.

This article is part of a series on the methodical documentation of LLM-supported development projects. The focus is on transferable insights, not on the tools developed themselves.