Agility and the Borgs

I have been a trekkie for a long time. Maybe, because software engineers are more open minded for this kind of SciFi movies and series. In the EXPO-C conference in Karlskrona, Sweden, Jimmy Nilson and me were participating in some Open Space events after the conference day. If you wonder what an open space is about: the goal of an open space is to meet people and discuss interesting topics. For this purpose, everyone can add her/his topic to a poster hanging on a wall. Then for each of these topics people are asked to participate in meetings that take place for a fixed amount of time. It is allowed to leave a group and move to another one (if you weren't the one who came up with the topic). Works very fine to get input and feedback from smart people. In Karlskrona we discussed topics such as "when is simple too simple", "agility", "what does it need to edutacte architects", just to give you an impression. After one of those "agile" topics I was wondering how these agile processes apply to Star Trek. Strange combination of topics, you might think now. One of the points discussed at the conference was whether hierarchical structures are suitable for agile processes. Thus, I asked myself what kind of process paradigm the Borgs are using. As you know, the Borgs are known to be very rude, assimilating other creatures to integrate their knowledge and making them robots without any own will. The Borgs are controlled by a central instance - the queen - but are tightly integrated to a community of heavily communicating and interacting creatures. It seems as if the queen is in charge of providing central goals but Borgs are able to achieve these goals by any means they consider appropriate. For now, we recognize that Borgs use communication and interaction, share their knowledge with each other, can interoperate independently as groups but not as individuals, continuously share their knowledge and abilities, strive for common code ownership. It is not obvious from the series if the Borgs follow a BDUP approach or are able to refactor and refine their work in the typical case of evolving and changing requirement. However, their behavior suggests that they are capable of fastly adapting to changing contexts. Take their ability to modify the frequency of their protection shield as a prominent example. So, does the society of Borgs constitute an agile organization or do they follow a waterfall model or a kind of hybrid approach? From my guts feeling, I always thought that agility requires smart and autonomous developers who organize themselves in a P2P fashion within a project. Do hierarchies, central organization and agility denote concepts that can be combined or do they contradict themselves. Is there a dependency between organization and process paradigm? Are there any contexts in which agile processes are inappropriate. If agility means the ability to constantly adapt to changing environments and requirements, then the Borgs obviously denote an agile organization. Even in an agile process someone needs to set the goals, coordinate all activities, and control the process. Why are the Borgs so successful, anyway?

The Problem with Metrics

From time to time I am asked what I think about evaluating software systems using metrics. The issue with most metrics is their close association to implementation artifacts. One of the wide-spread benchmarks is LOC. Measuring a software systems by lines of code does not make much sense in many cases. What does it mean that a subsystem contains a specific number lines of code? What about lines generated by model-driven software development tools, middleware generators, GUI builders, or IDEs? That does not mean, however, that LOCs are completely worthless. At least, they can give you hints where quality problems might lurk. For example, if a method contains several hundreds lines of code, then you definitely got a problem with inappropriate modularization. Another example is Cyclomatic Complexity (CC) introduced by McCabe. The CC of your system's runtime graph can be calculated by CC = E - N + P (see wikipedia link to CC). Here, E denotes the number of edges in the graph, N is the number of nodes, and P the number of connected components. According to McCabe a value of CC greater than 50 means your (part of the) system has too much complexity and reveals high risk. When applied to the Observer pattern with 50 observers, the CC will be larger than 50! Unfortunately, we all know that the Observer pattern is everything but complex and risky even when used with high numbers of observers. The problem here is that CC simply recounts all connections even if they are all of the same type. What does all that mean? My point here is that metrics are of limited value for a software architect. For each metrics used, a software architect should be aware of the strengths and limitations. Architecture quality and their lack can be evaluated better using other means. I will write about these qualities in a further posting. Of course, in the meantime I am very interested in what you think about metrics?

What is Software Architecture

Anyone involved in software development projects is used to the term "Software Architecture". If asked what software architecture really means, most people will throw some of the well-known definitions at you. Just give yourself a try and search for it in Google, Wikipedia, or whatever information source you prefer. You definitely will be overwhelmed by the number of hits. In http://www.sei.cmu.edu/architecture/definitions.html some definitions are available. It's definitely entertaining to read all these definitions which are all valid but only show a part of the whole picture. To be honest, all of these definitions are mostly useless. It is much more important to ask what software architecture is going to provide for a project. In other words: if software architecture is the answer, what exactly is the question? Basically, every software system has a software architecture. However, a software architecture might be worthless when it is built by accident. Let me give you an example. If I just put together some spaghetti code using a trial-and-error approach without ever thinking about software architecture, I'll get an implementation that propably will work. Unfortunately, I'll have no clue why the system is working and I will definitely have a very bad time when being asked to maintain, evolve, change, integrate or otherwise modify or understand the system. As a consequence, software architecture in a more narrow sense should be the result of systematic analysis and design. But even systematic analysis and design might lead to problems when just considering implementation aspects. I've seen a lot of systems that were realized with a specific middleware, operating system, or database system in mind. As soon as some of these system details changed, development teams were doomed to fail. Thus, software architecture should focus on more implementation independent aspects, but also stay in harmony with the underlying system infrastructure (a.k.a. system architecture). In a further step, we should ask ourselves what drives the software architecture design? Basically, all kinds of forces respectively requirements such as desired properties, constraints, commonalities, variabilities have an impact on the architecture. But you may ask, what if there are conflicting requirements, and how to deal with requirements when there are so many of them? All requirements should be prioritized. If there are conflicting requirements with the same priority, then you got a problem! In this case the problem must be resolved with all stakeholders. That brings me to a further perspective on software architecture: First of all, software architecture is about communication of explicit and documented design decisions. This is the reason why I consider domain modeling so important. In addition, software architecture is not only about structures, relationships, dynamics but also about the process used to define all these structures and relationships as well as their interactions. In this context, it proves to be particularly important to differentiate between the strategic baseline architecture and tactical design. The core of the architecture should implement all important strategic requirements. Tactical refinements must not modify strategic decisions. Thus, designing software architecture is mainly a top-down approach, but may also use bottom-up integration of legacy systems or components. As already mentioned, requirements are the drivers for software architecture design. In the process, requirements depending on their priority need to be realized in the software architecture in such a way that requirement traceability is easily possible. Without traceability design erosion will destabilize the whole system, requiring additional efforts for refactoring or architecture recovery. Thus, software architecture mainly deals how to map requirements to appropriate structures. Here, granularity aspects are important. Have you ever tried to build a house by assembling atoms? That does not make sense as complexity cannot be mastered using a fine-grained approach. Therefore, software architecture is about mastering complexity by introducing appropriate abstractions. Patterns are a good example for such "abstract" architecture entities. As some requirements denote cross-cutting concerns, the right structuring of a software system under development is not easy. Consequently, software architects must introduce different perspectives. These different perspectives must be combined and converged to a unified and consistent software architecture. Therefore, software architecture is also about perspectives and the combination of these perspectives, which, however, turns out to be one of the most complex tasks in any software development project, because of emergent behavior. Of couse, this posting could only show the tip of the iceberg. I think, it is obvious why no single definition of "software architecture" can ever satisfy anyone. Nor is it really important to have such a definition. It is much more important to have a common understanding of the benefits of systematic software architecture design.