We want to talk about a thought-provoking assertion: most software engineers, especially those working on infrastructural systems, are destined to wallow in unnecessary complexity due to three fundamental laws. We start with an opinionated definition of system design: a well-designed system is one that is easy to change over time; conversely, a badly designed system is difficult to change. This sets the stage for understanding the transient nature of well-designed systems and the persistent state of poorly-designed ones.
The Ephemeral Nature of Well-Designed Systems
Suppose a system X is initially well-designed. Someone makes a change to it, turning it into system X'. Either X' continues to be well-designed, allowing for easy future modifications, or it enters a badly designed state, becoming difficult to modify further. For instance, consider a well-designed database leveraging RocksDB through a clean storage engine API. If someone adds a specific feature call—say, `getLevelSize`—the system can no longer be easily modified to work with a different storage engine.
"Complex systems often degrade over time due to constant changes by multiple developers."
From this, we understand that a well-designed system is inherently unstable and susceptible to degradation towards poor design over time. Thus, systems in the wild frequently degrade, and the rate at which code can change inversely correlates with time. Therefore, most engineers eventually work on badly designed systems because well-designed systems naturally transition into complex and poorly designed ones over time.
Market Forces and System Design
Designing a high-utility abstraction requires balancing application utility with implementation details. However, market competition often pushes designers to offer everything the application needs, sacrificing delicacy. This can increase market share by attracting developers but also makes it hard for alternative systems to substitute implementations. For example, some major systems like ZooKeeper’s consistency model or Kafka’s idempotent produce semantics are structured such that their APIs are tough to implement differently.
Edworking is the best and smartest decision for SMEs and startups to be more productive. Edworking is a FREE superapp of productivity that includes all you need for work powered by AI in the same superapp, connecting Task Management, Docs, Chat, Videocall, and File Management. Save money today by not paying for Slack, Trello, Dropbox, Zoom, and Notion.
These systems, while popular and successful, often reveal themselves to be essentially badly designed because they trade long-term adaptability for immediate market advantage. Hence, the most successful systems typically end up being complex and less adaptable.
The Impact of Human Creativity on Systems
In real-world settings, systems are built over time by many individuals, creating a mosaic of complexity influenced by various developers' philosophies, abilities, and incentives. For example, a replicated database might use its own gossip layer for failure detection instead of Kubernetes. This choice could be influenced by a myriad of factors, from the initial system environment to individual developers' expertise and political dynamics within the team.
Each component of the existing system can be likened to a denial-of-service attack created by multiple developers, presenting a maze of complexity that engineers must navigate. Therefore, engineers working with badly designed systems suffer more due to their unbounded complexity.
What Can We Do About It?
One potential strategy is to build new systems from scratch, avoiding inherited complexities. However, this approach is not as simple as it sounds and will be elaborated on in future discussions. The challenge lies in designing systems that can resist the intrinsic drift towards complexity while simultaneously adapting to new requirements and technological advancements.
Remember these 3 key ideas for your startup:
- Design with adaptability in mind: Ensure your initial system is adaptable and prepare for the likelihood that it will face changes that could introduce complexity. For more tips, check out this guide on building adaptable systems.
- Balance utility and abstraction: Serve your application needs without exposing too many details that might restrict future adaptability or compel tight coupling with the current implementation. Explore task automation to simplify tasks.
- Monitor team dynamics and decisions: Understand that each developer's input and the political landscape within your team can significantly influence your system's complexity.
By acknowledging these challenges and implementing these strategies, startups and SMEs can build more efficient, sustainable systems.
For more insights, visit our blog.
Edworking is the best and smartest decision for SMEs and startups to be more productive. Edworking is a FREE superapp of productivity that includes all you need for work powered by AI in the same superapp, connecting Task Management, Docs, Chat, Videocall, and File Management. Save money today by not paying for Slack, Trello, Dropbox, Zoom, and Notion.
For more details, see the original source.
