Tag Archives: Refactoring

Refactoring: strategy & collaborative work

Introduction –  The extended definition of Refactoring contains also this part:  “Its heart is a series of small behavior preserving transformations. Each transformation (called a “refactoring”) does little, but a sequence of transformations can produce a significant restructuring. Since each refactoring is small, it’s less likely to go wrong. The system is kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring.”- Martin Fowler, at refactoring.com.

Note: this imaginary dialog it’s inspired from an intensive and extensive practice.

What if we will have to perform a Big Refactoring? 

Refactoring cannot be big, it is a special kind of redesign, performed in small steps. See the above definition again.

Reformulate: I need to do a lot of refactorings to clean a legacy code.  There is any best practice?    

You need a strategy?

Yes!

Well, agile and software engineering offer a lot of tactics, including the Martin Fowler refactoring sets but almost no strategy, except … you should work clean from the first time. Use Martin Fowler indications from the start (or early enough), also Uncle Bob Clean Code and Clean Architecture.

Hey! I already have a lot of legacy code debt to solve!

Ok! Let’s build a strategy: how do we start?

This was my question!

Refactoring supposed improving the design, while preserving the functionality. Tests included. Do you have good requirements specifications or a good set of automated tests?

Not in this case.

Then you should recover functionality knowledge from the code and put/perform incrementally some tests. Better: functionality should be explicitly represented in the code and should be testable. And remember: there are two kinds of functionality…

Two?

Yes, first, the one that is application-independent and represent the target domain (domain business rules) and the one that is application-depend aka flow of control.

I remember: that sound like Uncle Bob Clean Architecture.

Yes. You will need to be able to apply distinct tests to them, without mix them with other concerns such as UI, persistence, network and others. Anyway, where do I usually start? I will try to make the running scenarios very clear into the code and that mean the flow of control.

In English, please?

I want to clearly see these: were the event triggered by the system actors start and the end-to-end full path until return. More, I want to refactor to make this path clear enough.

How could be not clear? ­­­

Global context. If the functionality path chaotically accesses the global context, then we could have undesired intersections with other paths/scenarios, that will compromise both data and function. In the same time, we can decouple flow/orchestration from specialized concerns.

What we get?

We will have explicit representation of the functionality (with no undesired contacts with other flows), needed for tests (we can apply auto-tests on it). Also we will have the first entry points to the specialized parts that also could be <decorated> with some tests. Then we can apply tactical refactoring as we need.

And …the domain business rules?

Must be decoupled from other concerns and you have to dedicate them specialized design/test elements.

That’s all?

Almost. You need to test any redesign. Tests need knowledge about functionality. If some parts are missing, now it is the time to recover them in auto-tests (preferable) or in other form of specification.

How do I know that recovered requirements are correct?

You don’t. More, you should always suspect that spaghetti-like legacy code include many unobserved bugs. You should validate these functional requirements by intensive collaboration with your colleagues, with domain experts, customer and other stakeholders.

Do you have any idea about how to do that?    

Start with Pair Programming (refactor in pairs). Pairing is not enough, and you will probably need more people involved – use Model Storming: discuss the resulted functionality with more colleagues.

Model Storming?

Yes, it is an agile practice, part of Agile Modeling (and Disciplined Agile) and it was created to complement core practices from XP. Also, you should actively involve your stakeholders in validating the recovered functionality…. Active Stakeholder Participation, that it is another Agile Modeling recommended practices. And at the end you will have more free bonuses.

What bonuses?

Functionality it is easy to accurately read from code (seconds!) and your colleagues and your stakeholders will already have acquired the recovered functional knowledge.

Summary –  Refactoring for significant spaghetti legacy code need tests/testing. Usually, knowledge about functionality necessary for testing it is insufficient, so must be recovered from the code. An effective & proven way to do that is to apply Clean Architecture principles: decuple both domain rules and application specific flow of control (aka use cases). Anyway, legacy code with too much technical debt will contain a lot of bugs, so recovered functionality it is inaccurate and need to be validated.  Knowledge & expertise needed for validation it is distributed among team members, domain experts, customers and other stakeholders, so you need to work in a collaborative manner with all mentioned parts. There are some outstanding software engineering and agile practices that could help on this aspect:

Note: “need” and “necessary” are often use in above text, just because we have followed the logical path of necessary things for testing a redesigned legacy code.

Remember: A lot of technical debt ~ inaccurate functionality. To refactor & test, you must re-start the process & collaborative work from functional requirements acquisition.     

JIT – Just in time and Software Development

(See also Part 2 – Two dimensions: Just in time and Envisioning)

JIT – solution for incertitude and complexity

Driven forces that introduces JIT Life-cycle in software development

  • Business side: often changes – it is too complex to perform (too much) ahead requirements gathering
  • Development side: software solutions are mostly design (instead of production) it is too complex to manage big chunks

As a consequence of the degree of incertitude and complexity for both requirements and solution, the life-cycle (planning) that suit better will have a JIT model. Agile development has adopted from the start such approach in its principles and practices: often and small releases, iterative development.

JIT approach it is a solution for dealing with incertitude and complexity.

JIT approach it is a solution for dealing with incertitude and complexity. It is similar with the mathematical approach to solve non-linear problems: feedback based approaches (control theory).  The main issue is that you cannot compute (in mathematics) something that is too complex. In software development that mean you cannot envision too much requirements, solution and plan because of incertitude and complexity.

You cannot “compute” something that is too complex

Agile is one of the development approaches that already use JIT for more aspects. We can observe that XP that use “small releases” approach, use also “simple design” principle/practice – they do not want to make guesses about possible solutions aspects, required by possible future change request.

Let reformulate some JIT aspects:

  • do not make guesses about incertitude (what is difficult of impossible to clarify)
  • do not try to “compute” too complex problems

Do not make guesses about incertitude

If these principles are not follow, we will have the same problems as in mathematics: huge deviations of the solution for small changes in the inputs. Translating to the software development that mean huge waste.

JIT and Agile

Some Agile principles and practices that already use JIT approach:

  • Responding to change over following a plan” (Agile Manifesto – value)
  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” (Agile Manifesto – principle)
  • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale” (Agile Manifesto – principle)
  • “Business people and developers must work together daily throughout the project.” (Agile Manifesto – principle)
  • Make frequent small releases.” (XP rule)
  • No functionality is added early.” (XP rule)
  • Simple design (XP Practice)
  • Model Storming (Agile Modeling / DAD – Disciplined Agile Delivery practice)
  • Document late, document continuously (Agile Modeling / DAD practice)
  • Active stakeholder participation (Agile Modeling / DAD practice)
  • Just Barely Good Enough (Agile Modeling / DAD practice)
  • Explicit support for JIT based life-cycles: Agile, Lean, Exploratory  (DAD – Disciplined Agile Delivery)
  • Inspect and Adapt (Scrum principle)

JIT – main difference versus manufacturing

We need to deal with the main difference versus manufacturing: JIT design versus JIT production. In manufacturing we repeat the solution (design) from the last life-cycle and in software development we need to find a new solution for the newer requirements (metaphor: the market request every time car with a new design). The major problem here is to integrate the previous simple design with the current simple design (there are not just additive). We need that:

  • The existent design must be easy to extend
  • Integration of “next” design must be quick, easy and clean

I have described a solution for this problem in a previous post, a solution that re-arrange some already know thinks from XP and Refactoring (as it was defined in the Martin Fowler book) –  an Adaptive Design based on this rules:

  • Use simple and clean design and then adapt for new changes (example of adapt “tool”: refactoring)
  • Use design practices that increase the adaptability (Refactoring, TDD, Clean Code, Clean Architecture)

JIT production from manufacturing it is based on responsiveness of a highly automated production. JIT design from software development it is more difficult to be automated, but we need to find that solution for responsiveness – it is mandatory to have an Adaptive Design.

Summary

  • JIT approach it is a solution for incertitude and complexity, that it is validated also in mathematics
  • Software development main problems are related to incertitude and complexity, that mean JIT approach could be useful in various ways
  • JIT rules: do not make guesses about incertitude, do not try to “compute” what it is too complex
  • There are many Agile values, principles and practice that are based on JIT approach
  • JIT Design require an Adaptive Design

Upgrading Refactoring: Clean vs Adapt, Clean Code/Refactor/Re-Design

If I had to pick one thing that sets XP apart from other approaches, it would be refactoring, the ongoing redesign of software to improve its responsiveness to change.  – Jim Highsmith

The easiest technical debt to address is that which you didn’t incur to begin with. – Scott W. Ambler

 Refactoring definition – What, How and Why

There is a great work related to Refactoring, starting with the book “Refactoring: Improving the Design of Existing Code” of Martin Fowler and its collaborators (Kent Beck, John Brant, William Opdyke, Don Roberts) and continuing with the XP – Extreme Programming ecosystem of practices, that include Refactoring as the fundamental Agile practice.

The definition from Martin Fowler, at www.refactoring.com:

<<In the Refactoring Book, I made the following definition of “Refactoring”

 noun: a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior

 verb: to restructure software by applying a series of refactorings without changing its observable behavior

 Refactoring isn’t another word for cleaning up code – it specifically defines one technique for improving the health of a code-base. I use “restructuring” as a more general term for reorganizing code that may incorporate other techniques.>>

Some comments:

  • The objective could be generalized: Improving the design of existing code, without changing the observable functionality, must be performed for improving the health and economics of the software, where “easier to understand and cheaper to modify” are only some of the objectives (others: less defects, easier to detect the defects)
  • Clean is just one “technique“… Which are the others?
  • The extended definition contains also this part:  “Its heart is a series of small behavior preserving transformations. Each transformation (called a “refactoring”) does little, but a sequence of transformations can produce a significant restructuring. Since each refactoring is small, it’s less likely to go wrong. The system is kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring.”- Martin Fowler, at refactoring.com.

Improving the design of existing code, without changing the observable functionality, must be performed for improving the health and economics of the software.

.

Refactoring versus waste – Clean versus Adapt

Refactoring offers a process solution for design flexibility. A possible approach for getting a flexible solution could require some up-front design based on imagined future requirements. In many cases, these requirements never become real and that it is a waste. Martin Fowler says: “Refactoring can lead to simpler designs without sacrificing flexibility. This makes the design process easier and less stressful. […] Once you have a broad sense of things that refactor easily, you don’t even think of the flexible solutions. You have the confidence to refactor if the time comes.

In this case, the better design will come later, when the new context will be revealed.

Let’s think about some other kind of refactoring types, the ones that fix the majority of the design smells such: function and classes’ sizes, multiple responsibilities, magic numbers, duplicated code, meaningful names, comments and others. In this case, when these rules are applied or corrected later, the waste is bigger and even in exponential manner.

That seems contradictory: sometime it is better to refactor later and sometime is better to refactor sooner. Well, it is not! “Flexibility” and “Design smells” are two different problems.

There are two kinds of design decisions involved here:

  • Context independent – the ones that makes the code clean
  • Context dependent – the ones that are adapted to the current context

 Clean and Adapt are two different things and each one must be managed accordingly.

The sequence of decisions related to the design could be like this:

  • First development: Simple and clean first code – apply Simple Design and Clean Code. Use some “in development” refactoring, but try to write clean from the first time.
  • Next development: Adapt the first simple and clean design to the new context and perform also necessary cleaning.

The statement “You build the simplest thing that can possibly work.” should be reformulated as “You build the simplest clean thing that can possibly work”.

Sources of waste: any wild guess about future contexts, but also any mess remaining from previous development.

.

Reformulating the process

We have two kind of activities: Clean and Adapt and more practices that could realize their objectives:

  • Clean Code – write the code clean from the first time
  • Refactor – improve the design, while preserving observable behavior; executed in small transformations, while keeping the system stable
  • Redesign – similar with refactoring, but not necessary in small steps (with increased risks)

.

Clean Code & Refactoring

Both practices are introducing a set of design rules, specified in two books, where “Clean Code” it is creation of Robert C. Martin and “Refactoring: Improving the Design of Existing Code” of Martin Fowler and its collaborators (Kent Beck, John Brant, William Opdyke, Don Roberts). Logically it is a single set of rules that:

  • Keep the code clean
  • Are rather context independent

.

Keeping the code clean – practices and approach

In most of the cases, the design decisions related to a clean code are context (requirements) independent. In order to eliminate waste, the code could be kept clean as follows, using these practices:

  • Clean Code: write the code as clean it is possible from the first time
    • Refactor “in development”: earliest refactoring, during the development is the cheapest
  • Refactor “legacy”: paying the debt of dirty code as early is possible is the cheapest variant, when the debt already exist
  • Re-design “legacy”: only in cases where Refactoring (small steps) it is not possible, because the risks are increased in this case and the testing it is more difficult.

Simple Design it is considered as default complementary practice for all these cases.

Sources of waste:

  • Writing first code dirty is the most massive source of Technical Debt and refactoring it is not efficient/effective
  • Technical Debt reduce productivity, quality, agility and increase system rigidity and fragility
  • The cost of paying the Technical Debt could increase exponentially when it is accumulated in high amounts of dirty code

.

Adapt Design rules – practices and approach

These kind of decisions – adapt the design – are context dependent. In order to eliminate waste, the best time to take such decisions is to defer them until the context it is revealed. A possible approach could be:

  • Respect the above clean code rules
  • Simple (Design) and Clean (Code) – “build the simplest clean thing that can possibly work” for the current context/requirements
  • Adapt “legacy” design in new context – adapt the previous simple design to the new context /requirements, when this new context it is revealed (Simple Design it is applied again):
    • By Refactoring
    • By Re-design ( see comments for re-design from Clean Code rules)

Important: Refactor/Re-design to adapt the previous design must be done when this previous design does not match exactly to the new context. There are some practices that for Adaptive Design that increase the chance for an emergent design with less work related to adaptation – See “Roadmap to an Agile Design.

Simple Design Formula: Simple Design (context 1) + Adapt & Simple Design (context 2) + …

Sources of waste:

  • Premature design decisions based on wild guesses about future requirements, as opposite to Simple Design
  • Skipping the adapt design decisions needed in the new context
  • Trying to evolve the design only with these adapting tactical steps without any strategic approach for emergent design as Functional Cohesion and Separation of Concerns from Clean Architecture

Effective/Efficient “Clean”: clean and simple design first and then refactor.

Effective/Efficient “Adapt”: clean and simple architecture (& design) first and then adapt.

.

Process level upgrades – DAD style

The above logic could be re-formulated, refactored (sic!) again in terms of:

  • Process goals: keep the code clean, adapt the design (previous simple design in new context)
  • Goal options: An alternative for Refactoring is Re-design, where Refactoring is the default option and Redesign (with increased risks) should be considered only when the Refactoring is not possible
  • Note: Strategic options are complementary to tactical options and not alternatives

This process level logic it is used by DAD – Disciplined Agile Delivery, and finally help to fill the process gaps from other methods or a custom process and offer a much better support for process guidance.

For the Adapt goal ,  Refactoring is the core tactical technique, but in fact there are many practices that contribute to this goal:

  • Examples of tactical practices for Adapt goal: Refactoring, Test first, Re-design, Model Storming, Look Ahead Modeling.
  • Examples of strategical practices for Adapt goal: Clean Architecture, Continuous Integration.

More: a generic goal it is Design Envisioning & Change,  where the two main aspects are what is new and what is adapted. For an Agile context, based on Simple Design, the Adapt aspect is fundamental and has a “big share” in the process.

 There is not only Simple Design, there are “twins”: Simple Design + Adapt

%d bloggers like this: