The Case for Dynamic Diagrams
There are almost 29M devs in the world, why are most of them still wasting time manually drawing diagrams?
👋 Hi, this is Thomas, with a new issue of “Beyond Code: System Design and More”, where I geek out on all things system design, software architecture, distributed systems and… well, more.
QUOTE OF THE WEEK:
“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” - Martin Fowler
Book: Refactoring: Improving the Design of Existing Code, 1999
Recently, my article “Devs Need System Design Tools, Not Diagramming Tools” was featured on The New Stack and made it to Hacker News front page.
It was gratifying to see that it resonated with so many developers, with feedback like “I read the whole article and it has great points. Almost nodded the whole time.”.
This article stemmed from my frustration with the common approach of asking “What diagramming tool should we use?”rather than addressing the root problem: the need for up-to-date, easily accessible system architecture information.
I firmly believe that traditional diagramming tools are ill-equipped to handle the complexities of modern software systems and their architectural evolution (which is also why I ended up building my own solution for this problem 😅).
Todays Systems are too Complex and Dynamic for Static Diagrams
Let's take a step back and examine the factors contributing to the increased complexity in today's software systems:
Increased Technological Complexity. More and more companies are migrating from traditional centralized systems to distributed environments because of the significant advantages to performance, scalability, and fault tolerance. Furthermore, the widespread adoption of cloud-native technologies, SaaS, APIs, and composable platforms has dramatically increased the complexity of software systems, amplifying both hardware and software interdependencies.
Accelerated System Evolution. The adoption of Agile methodologies and modern design practices like Continuous and Evolutionary Architecture has significantly increased the rate of change within software systems. Teams have to update their systems to reflect changes in infrastructure, new technologies, evolving business requirements, or a plethora of other aspects that might change during the lifetime of any software system.
While in parallel we have more legacy software than ever, that continues to age and needs to be integrated with new solutions.
Changing Engineering Team Dynamics. System design is a collaborative activity that requires the alignment of multiple stakeholders, which may not always be co-located. This is especially true now with the globalization of the workforce and the rise in remote and hybrid work models. This challenge is further intensified by the recent tech industry layoffs (although previously, the high turnover rate in engineering teams was also a factor).
A platform that harmonizes communication and collaboration across teams, offices, and time zones, brings undeniable cost, time, and effort savings.
Limitations of Traditional Diagramming Tools
When using the term "Diagramming tools," I refer to tools that visualize specific portions, structures, or behaviors of a software system.
These tools offer static abstractions tailored for specific audiences, primarily focusing on visual representation. They often simplify or omit critical component details like APIs, dependencies, integrations, repositories, and environments. Common examples include Lucidchart, Visio, Miro, and FigJam.
Historically, when software systems were simpler, a single static diagram could potentially encapsulate all necessary system information. However, with the increase in complexity and scale of modern systems, these traditional tools fall short in several key areas:
Lack of Real-Time System Representation: Software systems are constantly evolving, undergoing frequent updates—changes in codebase, architecture, and dependencies. Static diagrams fail to capture these dynamics, often leaving teams with outdated documentation.
Unnecessary Manual Tasks: Manually creating and updating architecture diagrams can be time-consuming and error-prone, especially in large and complex systems.
Clunky User Interface: Updating diagrams can be cumbersome, with significant time spent on formatting and arranging components.
Version Control Issues: Maintaining updated versions across teams is challenging, not to mention tracking changes over time and reverting to previous versions if necessary. And, yet, effective version control is crucial for understanding the evolution of system architecture and the impacts of specific design decisions.
Limited Collaboration Capabilities: By providing a visual representation of the architecture, diagrams are meant to facilitate discussions, brainstorming sessions, and decision-making processes among stakeholders with varying levels of technical expertise. However, the lack of features for real-time collaboration and feedback diminishes their effectiveness as communication tools.
Lack of Key System Information: Diagrams rarely include information about requirements, trade offs, design decisions, etc. Which are essential details for fully understanding the system and are often scattered across multiple platforms like Confluence, Notion. and GDocs without proper integration.
Incapable of Managing Cloud Resources: Diagrams do not control or generate infrastructure code (e.g., CloudFormation, Terraform).
Where do Traditional Diagramming Tools Fit into the Modern SDLC
Traditional diagramming tools have a place in the software development lifecycle, primarily as whiteboarding tools. They excel in visually conveying concepts, ideas, and rough plans, capitalizing on our innate visual learning abilities—most people, after all, recognize the wisdom that "a picture is worth a thousand words."
However, it's crucial to use “the right tool for the job”. While traditional tools are great for initial brainstorming and conceptualization, they fall short in handling the dynamic and complex nature of modern software systems.
We don’t need system diagrams that are static, inaccurate, incomplete, inconsistent, and not interactive.
We need diagrams that can dynamically adapt to reflect changes in a system’s structure, components, and interactions without manual updates.
Leveraging system architecture observability, thanks to the growing adoption of open standards,such as OpenTelemetry, could be the key to keeping system architecture diagrams up-to-date while significantly reducing the manual overhead for engineering teams.
📚 Interesting Articles & Resources
“‘Visual’ is the New Global Development Language” - Vladi Stevanovic
An explanation as to why communication is usually the hardest part of any software developer’s job and how visual mediums can help.
“What is OpenTelemetry”
OpenTelemetry (OTel) is an observability framework and toolkit designed to create and manage telemetry data such as traces, metrics, and logs. It’s a vendor- and tool-agnostic, open source Cloud Native Computing Foundation (CNCF) project, born out of the need to standardize how to instrument code and send telemetry data to an observability backend.
“What are Abstractions in Software Development” - Steve "ardalis" Smith
There are many interpretations of the term “abstraction” in software: some are objectively measurable, like whether a C# type is abstract, while others, such as the effectiveness of an abstraction, are subjective.