From hundred-page papers to clear summaries: A first LLM coding experiment#
280 lines of code. Several hundred uses every day.
A simple tool that automatically summarises documents – developed in just 5-6 hours as the first systematic experiment with AI-supported programming.
What was the goal?#
To create quick summaries of papers.
Even if the document is 50 or 100 pages long. And with flexible questions – sometimes a short version, sometimes a critical reflection.
At the same time, we wanted to understand: How does AI-assisted programming really work? What are the limitations, what are the possibilities?
What can the tool do?#
It processes documents and creates six different types of summaries:
- Standard summaries (e.g. short version, detailed version, focus on methodology)
- Structured analyses (main arguments, key data, open questions)
- Critical reflections (strengths and weaknesses of the study)
The tool accepts common formats such as PDF, Word and OpenDocument. Processing takes between 30 seconds and a few minutes, depending on the size of the document.
How was it developed?#
We worked in several steps:
- Specification (30 minutes): Clearly defined requirements
- First version (2 hours): Implemented basic functions with AI support
- Optimisation (3-4 hours): 4-5 iterations to improve the processing strategy
- Total effort: 5-6 hours spread over several weeks
- Result: 280 lines of Python code, 10 functions, 7 core libraries
The development was carried out with various AI models in small, clearly defined steps.
Why did it work so well?#
Because we deliberately kept it simple.
Instead of a complex overall architecture, we added each function individually. First the basis, then expanded step by step. This gradual approach prevented us from getting lost in complexity.
Another success factor was choosing the right tools. Libraries such as pymupdf4llm convert PDF content directly into an AI-friendly format (with headings, tables and captions as tags). This significantly improved the quality of the summaries.
Key insights#
1. Architecture planning remains human
The AI models were excellent for implementing individual functions. But decisions about the overall architecture? Those were significantly better when we made them ourselves.
The AI suggestions for the system architecture were often impractical or too complex. We learned to invest more time in our own preliminary planning and then use AI for narrowly defined implementation steps.
2. Prompt engineering is system architecture
The instructions given to the AI model are not just nice texts – they are a critical part of the solution.
Two techniques worked equally well: Writing critical instructions in CAPITAL LETTERS significantly increases their impact. And explicit negative instructions with double emphasis (‘If there is not enough information, do not write anything and do not add anything’) prevent fantasy content.
3. Simple strategies work surprisingly well
The two-step processing was simple: divide the document into sections, summarise each one, then create an overall summary.
Despite its simplicity, the tool delivers high-quality results. The size of the sections proved to be the most important parameter – smaller sections improve quality but increase processing time.
What can others learn from this?#
- Plan your own architecture and use AI models for defined implementation tasks
- Choose libraries that are already designed for AI integration (saves a lot of time)
- Understand prompt formulation as part of the technical solution, not as a side issue
- Prioritise simplicity over premature optimisation
- Work in small steps and add each function individually
Conclusion#
✔ 280 lines of code are enough for a productively used tool – simplicity beats complexity
✔ Strategic planning remains human, AI models support implementation
✔ Prompt engineering is not a detail, but the core of the solution
This is part of a series on AI-assisted programming. The focus is on what can be learned from such experiments – not just on the results.
Note: This tool was created almost a year ago. The technologies have evolved since then, but the basic methodological principles remain relevant.