Tag Archives: Clean Architecture

Before DevOps: Delivering more by delivering less

Undesired complexity give undesired deliveries

There are many aspects to consider related to DevOps and delivery, many strategies and (in the right context and the right approach) there are also some very useful tools. It is a complex issue and last thing that we want is to make that even harder because of undesired complexity. A poor and an un-appropriate design is the main cause of the problems.

Suppose that the main aspects of a product functionalities and design are the followings:

  • Functionalities: f1, f2, f3
  • Business rules: r1, r2
  • External frameworks, technologies, and drivers (APIs, hardware interfaces, others) e1, e2
  • External interfaces: i1, i2,
  • Technical mechanisms: t1, t2, t3..

Should stay together only what it changes and should be deliver together.

As generic rule, if one of this aspects it is changed, we do not want to affect, change and deliver others aspects. Below are some examples of anti-patterns.

Anti-pattern: Lack of functional cohesion

Just a start example: If was requested to deliver a change for the functionality f1, it is highly undesired to change also functionalities f3 and f4 only because f1, f3 and f4 are unnecessary coupled in the implementation. This coupling could mean: same function, same class or same package as opposite of using distinct design elements.

Anti-pattern: Non-DRY business rules

If the business rule r1 it is used in more functionalities (that have dedicated components) and that rule implementation is duplicated (multiplied) in every functionality, then a change in r1 will require change and delivery for all those functionalities and components.

Anti-pattern: Mixing functionalities with external communication

Suppose that f1 and f2 are using TCP sockets for communication with some external systems and we need to replace this type of communication, if the f1, f2 implementation is mixed with socket management aspects, then we need to change and deliver also f1 and f2.

Anti-pattern: Mixing functionalities with external interfaces

External interfaces could suppose handling of some specific external data structures and/or of some specific communication protocol. If we are mixing these parts with internal functionalities, with any change in the interface, we should change and deliver also some not related internal parts, .

Anti-pattern: Mixing functionalities with technical aspects

You need to protect the representation of the business inside the product from the changes related to technology. Where is the technology involved? Any I/O aspect that wrap the hardware: GUI, network, databases, file systems is strongly related to technologies and platforms. If, for example, you have a lot of functionality and business representation in the design of the GUI elements, any change in GUI technologies will affect the business representation inside the product, that should also massively changed and then delivered.

Anti-pattern: “Utils”

A symptom of a poor design that could cause undesired supplementary work on development and delivery are the “utils” packages, especially without clearly dedicated sub-packages. Examples:

  • “utils” package that mix together classes belong to different functional aspects
  • “utils” package that mix various technical aspects without having dedicated sub-packages

This kind of design suppose that we need to change, test and deliver the “utils” almost with any change of the product.

Anti-pattern: Dirty code

Breaking simplest Clean Code rules (similar with the ones that are used in refactoring) could cause undesired coupling and undesired increase of delivery scope. Some examples that could induce such problems:

  • Any duplicate/multiplied code
  • Breaking SRP principle
  • Global context data, global variables (breaking Law of Demeter)

Minimizing the need of change and delivery

We need to reduce the need of change and delivery only to the required ones. There are several practices and approaches that could avoid that problem (the examples presented above or similar ones):

  • Keep the code clean by writhing Clean Code first and refactor and pay the technical debt whenever it is necessary
  • Use functional cohesion as the main criteria of creating components and packages
  • Use Clean Architecture that propose a default, strategic separation of concerns
  • Extend XP engineering practices (Simple Design, Refactoring, TDD) with the ones from Agile Modeling and DAD
  • Respect Law of Demeter on any level of the design – do not use any global context
  • Make the products adaptive by keeping up-to-date with target business by often and early injection of feedback from that business.

You can argue that I already write about all these in a previous post “Roadmap to an Agile Design”. Yes, indeed, in order to deliver just what it is needed and avoid waste you must have an Adaptive Design, the main characteristic of an Agile Design.

In order to that you should be open  to all outstanding agile practices for design and do not be closed in the smaller universe of some very lightweight Agile methods. And remember that those lightweight methods are not created as full process methodologies, but as indications to build a customized process.

Imagine that all these problems are far worse if we have multiple variations and variants of the same product. The massive effort for any change and delivery will block the overall agility and responsiveness of the development team and massively reduce the overall economics for both development and customer side.

Use a strategic separation of concerns

Do not use any reference to any global context

JIT – Just in time and Software Development

(See also Part 2 – Two dimensions: Just in time and Envisioning)

JIT – solution for incertitude and complexity

Driven forces that introduces JIT Life-cycle in software development

  • Business side: often changes – it is too complex to perform (too much) ahead requirements gathering
  • Development side: software solutions are mostly design (instead of production) it is too complex to manage big chunks

As a consequence of the degree of incertitude and complexity for both requirements and solution, the life-cycle (planning) that suit better will have a JIT model. Agile development has adopted from the start such approach in its principles and practices: often and small releases, iterative development.

JIT approach it is a solution for dealing with incertitude and complexity.

JIT approach it is a solution for dealing with incertitude and complexity. It is similar with the mathematical approach to solve non-linear problems: feedback based approaches (control theory).  The main issue is that you cannot compute (in mathematics) something that is too complex. In software development that mean you cannot envision too much requirements, solution and plan because of incertitude and complexity.

You cannot “compute” something that is too complex

Agile is one of the development approaches that already use JIT for more aspects. We can observe that XP that use “small releases” approach, use also “simple design” principle/practice – they do not want to make guesses about possible solutions aspects, required by possible future change request.

Let reformulate some JIT aspects:

  • do not make guesses about incertitude (what is difficult of impossible to clarify)
  • do not try to “compute” too complex problems

Do not make guesses about incertitude

If these principles are not follow, we will have the same problems as in mathematics: huge deviations of the solution for small changes in the inputs. Translating to the software development that mean huge waste.

JIT and Agile

Some Agile principles and practices that already use JIT approach:

  • Responding to change over following a plan” (Agile Manifesto – value)
  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” (Agile Manifesto – principle)
  • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale” (Agile Manifesto – principle)
  • “Business people and developers must work together daily throughout the project.” (Agile Manifesto – principle)
  • Make frequent small releases.” (XP rule)
  • No functionality is added early.” (XP rule)
  • Simple design (XP Practice)
  • Model Storming (Agile Modeling / DAD – Disciplined Agile Delivery practice)
  • Document late, document continuously (Agile Modeling / DAD practice)
  • Active stakeholder participation (Agile Modeling / DAD practice)
  • Just Barely Good Enough (Agile Modeling / DAD practice)
  • Explicit support for JIT based life-cycles: Agile, Lean, Exploratory  (DAD – Disciplined Agile Delivery)
  • Inspect and Adapt (Scrum principle)

JIT – main difference versus manufacturing

We need to deal with the main difference versus manufacturing: JIT design versus JIT production. In manufacturing we repeat the solution (design) from the last life-cycle and in software development we need to find a new solution for the newer requirements (metaphor: the market request every time car with a new design). The major problem here is to integrate the previous simple design with the current simple design (there are not just additive). We need that:

  • The existent design must be easy to extend
  • Integration of “next” design must be quick, easy and clean

I have described a solution for this problem in a previous post, a solution that re-arrange some already know thinks from XP and Refactoring (as it was defined in the Martin Fowler book) –  an Adaptive Design based on this rules:

  • Use simple and clean design and then adapt for new changes (example of adapt “tool”: refactoring)
  • Use design practices that increase the adaptability (Refactoring, TDD, Clean Code, Clean Architecture)

JIT production from manufacturing it is based on responsiveness of a highly automated production. JIT design from software development it is more difficult to be automated, but we need to find that solution for responsiveness – it is mandatory to have an Adaptive Design.

Summary

  • JIT approach it is a solution for incertitude and complexity, that it is validated also in mathematics
  • Software development main problems are related to incertitude and complexity, that mean JIT approach could be useful in various ways
  • JIT rules: do not make guesses about incertitude, do not try to “compute” what it is too complex
  • There are many Agile values, principles and practice that are based on JIT approach
  • JIT Design require an Adaptive Design

Agile design events and decisions options

Which are the concerns that should be considered for taking design decisions in an Agile Context?

Let think first about which are the “design events”, the moments in the life-cycle when design decisions are taken. The best description of these events could be found in Agile Modeling method (and inherited in DAD-Disciplined Agile Delivery):

  • Architecture Envisioning
  • Iteration Modeling
  • Look Ahead Modeling – opportunistically “model a bit ahead”
  • Model Storming – just in time modeling
  • … and Coding

TDD and Refactoring are rather ongoing activities, but main design decisions related to them could be also associated  with the any from the above mentioned events.

All the considered design concerns are generic, but some of them are more important in an Agile context (read Adaptive).  This is the proposed list:

  • New versus change: What existent parts will be affected and what is new? Sub-systems, components, external interactions.
  • Clean Architecture: Which are the changes in the business representations and which are the changes in the boundaries with frameworks and drivers.
  • Separation of Concerns: what we need to separate/decouple for Clean Architecture and for TDD?
  • Technical Debt: What Technical Debt we have to manage in the affected existent parts?
  • Cleanup: How much cleanup we need? How much Technical Debt should be payed now?
  • Adapt: What is the need of adapting (design) for affected parts, in the current known requirements context?
  • Adapt design techniques: which techniques should be used and in which amount? Refactoring or Redesign?
  • TDD: the main needed tests and decoupling decisions

Putting all together, we can model an Agile Design Decision Matrix with events and options.

Depending on the moment of the life-cycle, the magnitude of these aspects should be bigger or smaller, but I do not think that any of them should be skipped.

Adapt or not adapt? Cleanup is needed or is not needed? We need or we do not need auto-test?
All are possible options, and any answer it is a design decision.

Important: The answers to these questions will shape the needed work and must be used to estimate time and cost. Each of these events it is a moment of planning and re-planning.

Agile Design:  Simple Clean (& Tested) Design + Adapt

Upgrading Refactoring: Clean vs Adapt, Clean Code/Refactor/Re-Design

If I had to pick one thing that sets XP apart from other approaches, it would be refactoring, the ongoing redesign of software to improve its responsiveness to change.  – Jim Highsmith

The easiest technical debt to address is that which you didn’t incur to begin with. – Scott W. Ambler

 Refactoring definition – What, How and Why

There is a great work related to Refactoring, starting with the book “Refactoring: Improving the Design of Existing Code” of Martin Fowler and its collaborators (Kent Beck, John Brant, William Opdyke, Don Roberts) and continuing with the XP – Extreme Programming ecosystem of practices, that include Refactoring as the fundamental Agile practice.

The definition from Martin Fowler, at www.refactoring.com:

<<In the Refactoring Book, I made the following definition of “Refactoring”

 noun: a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior

 verb: to restructure software by applying a series of refactorings without changing its observable behavior

 Refactoring isn’t another word for cleaning up code – it specifically defines one technique for improving the health of a code-base. I use “restructuring” as a more general term for reorganizing code that may incorporate other techniques.>>

Some comments:

  • The objective could be generalized: Improving the design of existing code, without changing the observable functionality, must be performed for improving the health and economics of the software, where “easier to understand and cheaper to modify” are only some of the objectives (others: less defects, easier to detect the defects)
  • Clean is just one “technique“… Which are the others?
  • The extended definition contains also this part:  “Its heart is a series of small behavior preserving transformations. Each transformation (called a “refactoring”) does little, but a sequence of transformations can produce a significant restructuring. Since each refactoring is small, it’s less likely to go wrong. The system is kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring.”- Martin Fowler, at refactoring.com.

Improving the design of existing code, without changing the observable functionality, must be performed for improving the health and economics of the software.

.

Refactoring versus waste – Clean versus Adapt

Refactoring offers a process solution for design flexibility. A possible approach for getting a flexible solution could require some up-front design based on imagined future requirements. In many cases, these requirements never become real and that it is a waste. Martin Fowler says: “Refactoring can lead to simpler designs without sacrificing flexibility. This makes the design process easier and less stressful. […] Once you have a broad sense of things that refactor easily, you don’t even think of the flexible solutions. You have the confidence to refactor if the time comes.

In this case, the better design will come later, when the new context will be revealed.

Let’s think about some other kind of refactoring types, the ones that fix the majority of the design smells such: function and classes’ sizes, multiple responsibilities, magic numbers, duplicated code, meaningful names, comments and others. In this case, when these rules are applied or corrected later, the waste is bigger and even in exponential manner.

That seems contradictory: sometime it is better to refactor later and sometime is better to refactor sooner. Well, it is not! “Flexibility” and “Design smells” are two different problems.

There are two kinds of design decisions involved here:

  • Context independent – the ones that makes the code clean
  • Context dependent – the ones that are adapted to the current context

 Clean and Adapt are two different things and each one must be managed accordingly.

The sequence of decisions related to the design could be like this:

  • First development: Simple and clean first code – apply Simple Design and Clean Code. Use some “in development” refactoring, but try to write clean from the first time.
  • Next development: Adapt the first simple and clean design to the new context and perform also necessary cleaning.

The statement “You build the simplest thing that can possibly work.” should be reformulated as “You build the simplest clean thing that can possibly work”.

Sources of waste: any wild guess about future contexts, but also any mess remaining from previous development.

.

Reformulating the process

We have two kind of activities: Clean and Adapt and more practices that could realize their objectives:

  • Clean Code – write the code clean from the first time
  • Refactor – improve the design, while preserving observable behavior; executed in small transformations, while keeping the system stable
  • Redesign – similar with refactoring, but not necessary in small steps (with increased risks)

.

Clean Code & Refactoring

Both practices are introducing a set of design rules, specified in two books, where “Clean Code” it is creation of Robert C. Martin and “Refactoring: Improving the Design of Existing Code” of Martin Fowler and its collaborators (Kent Beck, John Brant, William Opdyke, Don Roberts). Logically it is a single set of rules that:

  • Keep the code clean
  • Are rather context independent

.

Keeping the code clean – practices and approach

In most of the cases, the design decisions related to a clean code are context (requirements) independent. In order to eliminate waste, the code could be kept clean as follows, using these practices:

  • Clean Code: write the code as clean it is possible from the first time
    • Refactor “in development”: earliest refactoring, during the development is the cheapest
  • Refactor “legacy”: paying the debt of dirty code as early is possible is the cheapest variant, when the debt already exist
  • Re-design “legacy”: only in cases where Refactoring (small steps) it is not possible, because the risks are increased in this case and the testing it is more difficult.

Simple Design it is considered as default complementary practice for all these cases.

Sources of waste:

  • Writing first code dirty is the most massive source of Technical Debt and refactoring it is not efficient/effective
  • Technical Debt reduce productivity, quality, agility and increase system rigidity and fragility
  • The cost of paying the Technical Debt could increase exponentially when it is accumulated in high amounts of dirty code

.

Adapt Design rules – practices and approach

These kind of decisions – adapt the design – are context dependent. In order to eliminate waste, the best time to take such decisions is to defer them until the context it is revealed. A possible approach could be:

  • Respect the above clean code rules
  • Simple (Design) and Clean (Code) – “build the simplest clean thing that can possibly work” for the current context/requirements
  • Adapt “legacy” design in new context – adapt the previous simple design to the new context /requirements, when this new context it is revealed (Simple Design it is applied again):
    • By Refactoring
    • By Re-design ( see comments for re-design from Clean Code rules)

Important: Refactor/Re-design to adapt the previous design must be done when this previous design does not match exactly to the new context. There are some practices that for Adaptive Design that increase the chance for an emergent design with less work related to adaptation – See “Roadmap to an Agile Design.

Simple Design Formula: Simple Design (context 1) + Adapt & Simple Design (context 2) + …

Sources of waste:

  • Premature design decisions based on wild guesses about future requirements, as opposite to Simple Design
  • Skipping the adapt design decisions needed in the new context
  • Trying to evolve the design only with these adapting tactical steps without any strategic approach for emergent design as Functional Cohesion and Separation of Concerns from Clean Architecture

Effective/Efficient “Clean”: clean and simple design first and then refactor.

Effective/Efficient “Adapt”: clean and simple architecture (& design) first and then adapt.

.

Process level upgrades – DAD style

The above logic could be re-formulated, refactored (sic!) again in terms of:

  • Process goals: keep the code clean, adapt the design (previous simple design in new context)
  • Goal options: An alternative for Refactoring is Re-design, where Refactoring is the default option and Redesign (with increased risks) should be considered only when the Refactoring is not possible
  • Note: Strategic options are complementary to tactical options and not alternatives

This process level logic it is used by DAD – Disciplined Agile Delivery, and finally help to fill the process gaps from other methods or a custom process and offer a much better support for process guidance.

For the Adapt goal ,  Refactoring is the core tactical technique, but in fact there are many practices that contribute to this goal:

  • Examples of tactical practices for Adapt goal: Refactoring, Test first, Re-design, Model Storming, Look Ahead Modeling.
  • Examples of strategical practices for Adapt goal: Clean Architecture, Continuous Integration.

More: a generic goal it is Design Envisioning & Change,  where the two main aspects are what is new and what is adapted. For an Agile context, based on Simple Design, the Adapt aspect is fundamental and has a “big share” in the process.

 There is not only Simple Design, there are “twins”: Simple Design + Adapt

%d bloggers like this: