Data visualisation with AI: When computers write code instead of performing calculations themselves#

In this series, we report on practical experiments with artificial intelligence in software development.

This time, we focus on a tool for automatic data visualisation in the form of charts. What makes it special is that the AI does not perform the calculations itself, but writes Python code that then generates the charts.

What was the idea?#

AI models often have difficulty handling numbers and frequently produce calculation errors.

We wanted to find out if there was a better way. The idea: the AI should not have direct contact with the numbers. Instead, it writes Python code (e.g. with Pandas, Plotly, NumPy), which is then executed in a secure environment.

The goal was quite pragmatic: to develop a tool that quickly helps users select suitable charts and creates simple visualisations largely automatically. The AI should understand what users want and then generate the appropriate code.

What can the tool do?#

We have developed an ‘Interactive Chart Generator’. The tool can:

Process different file formats: It accepts CSV and Excel files, including those with multiple sheets (multi-sheet support).
Create interactive graphics: The Plotly library is used to create professional, interactive charts (e.g. bar charts, time series, scatter plots).
Understand natural language: Simply write ‘Create bar charts for all sheets’ or ‘Colour the bars green for positive values’ and the tool understands what is meant
Work with local AI models: The system uses local AI models via an OpenAI-compatible interface – so the data remains on your own server

The tool has a simple web interface (based on Gradio), so no programming knowledge is required.

How was it developed?#

Development took place in three main phases, with AI support:

Version 1 – First attempt (unstable)

We started with two AI agents: a chat agent for communication and a chart agent for graphics creation. The problem: the system couldn’t reliably distinguish between when users just wanted to chat and when they wanted to create or modify graphics. The error rate was too high.

Version 2 – Intent recognition added

We introduced a third agent: the IntentService. This analyses each user request and classifies it into categories (e.g. ‘Modify an existing graphic’, ‘Create a single graphic’, ‘Create multiple graphics’, ‘Analyse data only’, ‘Discuss suitable visualisations’).

This approach significantly improved accuracy.

Version 3 – Three-step workflow (final version)

We developed a clear three-step process: Intent → Plan → Execute

Intent recognition: What is the user’s intention? (e.g. ‘Visualise all sheets’)
Planning: The PlanService creates a detailed execution plan with specific specifications for each graphic
Execution: The ExecutionService executes the plan, with automatic error correction if something goes wrong

Development time: (approx. 60 minutes per version)

Writing specifications: Detailed description of the architecture, UI design and functions
Implementation: The AI programs based on the specification
Adjustments and testing: Fix errors and make improvements
Docker deployment (30 minutes): Make the tool available for others to use
Total time: Approximately 3-4 hours for all three versions combined

Result: A system with approx. 8,500 lines of code distributed across 27 Python files.

Why did it work so well?#

The key was the combination of AI intelligence and proven code patterns.

Pure code generation by AI was too unstable. We therefore developed an extensive library with 44 code patterns. The AI no longer chooses freely, but from proven patterns (e.g. PATTERN_SIMPLE_BAR for simple bar charts, PATTERN_TIME_SERIES_GROUPED for grouped time series).

This library includes:

23 working implementation patterns for different chart types
7 anti-patterns: things the AI should avoid because they are unstable
9 modification patterns for adjustments such as colours or labels
5 semantic patterns for intelligent selection based on data types

The AI interprets what users want, selects the appropriate pattern and adapts it to the specific data. This hybrid approach works significantly better than pure code generation.

Key findings#

1. Good planning beats fast programming

For each version, we first wrote a detailed specification (approx. 60 minutes) before the AI began programming.

This is much better than ‘micro-prompting’ – i.e. giving the AI small instructions over and over again. Micro-prompting results in fragmented and inconsistent code. With a clear specification, everything remains structured and maintainable.

The specifications included: technical architecture (what components are there?), UI design (what does the interface look like?), interactions between components (who communicates with whom?) and functional goals (what should the end result be?).

2. Phase separation makes complex systems manageable

The separation into Intent → Plan → Execute has proven itself:

Intent recognition clearly distinguishes between discussion and action
The planning phase creates a structured execution strategy
Execution can handle errors in isolation

This keeps each phase testable and understandable. If something goes wrong, we know exactly in which phase the error lies.

3. AI control is better than fixed rules

We tested both approaches: rule-based heuristics (e.g., ‘If the word “all” appears, create multiple graphics’) and AI-controlled processing.

AI is superior because it generalises better. Heuristics only work for very specific formulations. AI also understands ‘Show me diagrams for all table sheets’ or ‘I need an overview of all sheets’ – without us having to program every variant.

What can others learn from this?#

Several principles can be derived from this project:

Code indirection for numerical tasks: If the AI is to work with numbers, let it write code instead of calculating itself. This is more reliable (e.g. for data analysis, calculations, visualisations).
Pattern libraries stabilise code generation: Give the AI proven patterns instead of letting it generate them freely. This significantly reduces errors and makes the results more predictable.
Specification-driven instead of micro-iterative development: Invest time in a good specification (60-90 minutes) before programming the AI. This saves time in error correction and leads to better code
Multi-agent systems for complex tasks: Break down complex problems into specialised agents (e.g., intent recognition, planning, execution). Each agent has a clear responsibility
Error correction with context: When code fails, return the error message to the AI and let it correct the code. Our fix_code() method noticeably reduced errors
Local AI models are usable: You don’t always need the largest cloud models. Local models can deliver good results with the right tools (such as pattern libraries)

Conclusion#

✔ Code generation is a promising way to circumvent AI weaknesses in numerical tasks – the AI writes the code, the computer executes it

✔ Good planning and clear specifications are more important than rapid iteration – investing time in preparation pays off

✔ Pattern libraries make AI-generated solutions more stable and predictable – the hybrid approach of AI intelligence and proven patterns works better than pure generation.

Status: The tool is currently in the testing phase with a wider user base. Initial tests with various Excel files (including complex multi-sheet structures) were successful. Further feedback will show where improvements are still needed.

This is part of a series on experiences with AI-supported software development. The focus is on what can be learned from such projects – not just on the finished tools.