Large language models for UML diagram generation: evaluating and improving local models against a hosted reference model

Date

2026-8

Author

Ateş, Onur

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

72
views

0
downloads

Cite This

Large language models and agentic tools are taking on more of the implementation phase, and the developer’s role is shifting toward defining intent and verifying that the output matches it. Code is generated more rapidly and in larger quantities; the challenge becomes ensuring that the model understood the request correctly. UML diagrams written in lightweight, text-based notations such as Mermaid give a concise view of a system’s intended structure and behavior, and can be reviewed much more quickly than the generated code. From this perspective, UML is assuming a new role in AI-assisted development by supporting system understanding and validation. For UML to fulfill this role, a model has to produce diagrams that render and that match what was described. This thesis investigates how closely local large language models reproduce UML diagrams from descriptions written in natural language, and how that can be improved. Using a reproducible, parser-based framework that scores diagrams against a reference on structural and semantic dimensions, the study evaluates locally deployable open models against a substantially larger cloud-hosted model across four diagram types: class, sequence, state and activity. To improve smaller models without fine-tuning, a complexity-aware few-shot in-context learning method is proposed, which matches each request with a single example that aligns with the target diagram’s complexity. The aim is to measure agreement with the reference across diagram types and complexity levels, and to determine how far this strategy narrows the gap between a small local model and a much larger one.

Subject Keywords

Large language models, UML diagram generation, Mermaid, Diagram complexity, Context engineering

URI

https://hdl.handle.net/11511/120691

Collections

Graduate School of Informatics, Thesis