Software Development Today

Tuesday, January 07, 2014

How injecting randomness into your project can help it succeed


Success and failure differ, very often, by very little. Take nature as an example. A small change in our DNA (a few mutated genes) can have catastrophic consequences. On the other hand, without these mutations humans would never have come into existence.

Humans and other species in the planet evolved because of chance (although not totally chaotic) changes in their DNA. As small change after small change happened, the surviving individuals were able to spread the viable, and ultimately the most adequate changes to the next generation - and the process repeats still in our own bodies today. If Nature has taught us something about improvement and evolution is that you must have a little bit of luck involved!

The DNA of a project

In software projects - my domain - I see some similarities to this natural model that are worth exploring. Projects also have a DNA which includes the structure and the communication connections between individuals. Communication being one of the most quoted causes for failure in software projects, and therefore one of its critical success factors.

The More I Practice the Luckier I Get

Once a project gets started, a particular structure is put in place (governance, management, teams, etc.). Very often that structure remains in place, and static until the end of the project. But is this a smart choice?

Another aspect of the project that rarely changes is the network of connections between the individuals. This network is what carries information from individual to individual and eventually "selects" the information that is acted upon by the project team. If a piece of information reaches an individual in the project, and that individual does not consider it relevant, that information will not spread further. In contrast, when a piece of information is perceived as important by an individual, that information will be actively spread by that individual.

The Illusion of Control

The structure that is set for a project strongly influences the communication links that can emerge between individuals. Who talks to whom? What meetings exist where people interact regularly? etc. But the reverse is also true. The existing communication network within an organization is a strong influence on what governance structure is put in place, how teams are formed, etc. These co-dependent processes create a positive (or negative) spiral of events for the project, but because neither (structure, or communication network) is often changed, that spiral is unstoppable. If it is positive, it will help the project succeed. If this co-dependent processes create, instead a negative spiral, that will relentlessly remove the chances of success for that project.

This co-dependent relation between two key processes (structure and communication network) is why we can increase the chances of success in our projects by simply causing random perturbations in the project. Randomness helps us explore different patterns for structure and communication networks.

As we carefully design and inject small - safe-to-fail - changes into the project, we can observe how it reacts and adjusts. Later, we can retrospectively amplify or remove/dampen the changes based on the outcome we see. If something works, keep it and ask how can you make it grow. If something creates a net negative outcome, dampen or remove it completely.

The cycle goes like this:

  • Define and implement small, safe-to-fail changes in the structure or communication network for the project
  • These changes lead to emergent behavior as the individuals and teams adapt to the changes
  • A new project configuration emerges which drives new project results
  • Finally, we evaluate the results of these changes and decide which components we will try to amplify or dampen

What Do You Mean Random?

There are many ways in which we can, randomly, explore better configurations for our projects. Below I list only a few to illustrate the concept:

  • At the start of a project, let the teams chose their own composition. This establishes new connections between individuals in the project as some will choose to work with new team members. However, these new connections will not eliminate the previous connections between individuals. The net effect should be that your project now has a more connected communication network (more individuals with strong connections to each other). In practice this works just like when we form new neural connections in our brain: more connections leads to different thinking and acting patterns
  • Organize common project events where people interact with each other outside the day-to-day routine. Planning events can be organized following a structured approach, but including also some unstructured time (à lá unconference, using e.g. Open Space) where people can interact based on their interests. This will also help form new connections between individuals in the project and spread information that would otherwise be locked down and not accessible
  • Have regular project "coffee breaks" that happen at the same time and same place. In these events people can connect with other individuals and ask questions from the project management team or the most connected individuals in the project. This unstructured communication increases the chance that some piece of information will "jump the hierarchical borders" and reach people in decision-making positions, or people with influence that can later act on that information.

Conclusion

These practices are designed to inject randomness (unplanned situations or connections) into the project. The goal is not to create Chaos (a scary word for many project teams), but to generate new pathways for the information to flow within the project team, as well as novel project structures that are more adapted to the current challenges the teams face.

Using these (or other) practices will increase your project's chances for success. They don't eliminate the need for the project to be managed, or to have structure. They do increase the chance of success for the project by exploring new organizations and structures through a process of small (safe-to-fail) changes that can lead to unplanned, but ultimately superior performance.

These were just a few examples. How would you inject randomness into your project? Have you done that in the past? Please share your experiences in the comments for the benefit of others.

Photo credit: John Hammink, follow him on twitter

Labels: , , , , , , , , , , ,

at 08:00
RSS link

Bookmark and Share

8 Comments:

  • Nice article, but it has some shortcomings, or let's say areas of possible improvement.
    First, comment I would like to make is about the communication network. You seem to model the communication inside the team as a network, where information is transmuted in a controlled manner. This overly-simplifies the context of agile and collocated teams, taking ignoring the osmotic communication model, that models communication inside a team closer to reality and to how us humans, these gene-complex carriers, communicate and act.
    If the importance of a piece of information is determined at a node level, and decision on spreading it or not, than I would say we are in big trouble, because the same piece of information has different value to different entities from inside the team, given by their role and the tasks they need to perform. This area has lot to elaborate upon ..
    Later on, as I finished reading the article, I realized that this assumption and model of the communication network is of capital importance to your model. Why does your model and method does not separate the assumptions and context beforehand ?

    Second, comment relates to "we can increase the chances of success in our projects by simply causing random perturbations in the project". This is again a simplification that only takes into consideration the upside and the positive effects of inducing randomness. You mention the existence of a feed-back loop for these small induced doses of randomness, but the feed-back loop I consider it to be too short, because the effects of some changes can propagate and manifest past the moment where effects are measured. At the same time, complexity-wise, inducing changes and then observing the effects, I believe, considers that all things remain the same and that all is known. At the same time let's keep in mind that correlation does not mean causation, and the other way around. Does your approach consider that the whole system can be controlled and understood by a single person or entity that evaluates the impact of these small changes?

    Third, I can also observe a bias that these changes will have positive effects. How does this model handle and justify the negative effects, and the supplementary work needed to fix them? In an evolutionary context, the entity that makes a decision that is detrimental will die and end it's race, but in a project context I do not believe this is an option or valid alternative, and it will cause delays or even failure in the project. In your view, what is the safeguard mechanism for this and how far can this randomness-inducing agent can go with these small experiments ?

    Fourth, isn't the natural randomness of human nature, human interactions and project context enough? How does one discriminate between changes created by self-induced project randomness and naturally-induced randomness? You mention the safe-to-fail paths, but these are taken a little bit to the extreme, as we all use them in our daily life to test and determine what action to perform, in an uncertain field. Do you imply that we should take more than usual safe-to-fail decisions in projects? If so, how does the cost of these explorations is accounted for, because in an economic context with competing projects, the one that will have the lowest cost for the same results will be the one deemed more successful.

    As an ending note, I would be curious to know why is the project-team's capability to adapt and change is so diminished. What you say is interesting and valid from a communication management perspective, but the article has a tricky approach and at least for me I had to ready it carefully and entirely to get the whole picture and context of your proposal.

    By Anonymous Lucian Adrian, at January 07, 2014 10:22 AM  

  • @Lucian You should write your own blog posts! :) This is a HUGE comment :)

    On your first comment:
    - filtering of information *does* happen at the node level. But it happens in all nodes, not just in one arbitrary node. That's just how it is. You can find *predetermined* ways to make information radiate farther from the originator (wikis, emails, etc.) but that only gives you *predetermined* results, not random - unexpected, but beneficial - results.

    By Blogger Vasco Duarte, at January 07, 2014 3:17 PM  

  • @Lucian
    On your second comment:
    - you should really read about "safe-to-fail" experiments. Your "random" perturbations do not need to be "catastrophic" - that's not what I am arguing for.

    By Blogger Vasco Duarte, at January 07, 2014 3:18 PM  

  • @lucian: On your third comment:
    - I do not make an assumption that all changes will have a positive effect. On the contrary! I explicitly expect that some will have a negative effect and therefore argue for "dampening", and careful observation in the post.

    By Blogger Vasco Duarte, at January 07, 2014 3:19 PM  

  • @lucian: on your forth point:
    - I do not regard comments like "you are being extreme" as valid. Sorry. You will have to try harder to explain why certain ideas cause conflict in you.
    - Naturally induced randomness always happens - we cannot get rid of it. What I argue for is some induced randomness, and the necessary adaptation.
    - The cost of all explorations should be taken into account before they are taken up. It's a simple proposition. However, some things cost more than you expect - be ready to stop those at any time.
    - Cost is not a factor in my suggestions. The project manager has a discretionary budget that she can manage. I expect it to be used for this (and other stuff).

    on your final note:
    - repetition breeds comfort. Comfort detracts from learning. Read "thinking fast and slow" ;)

    By Blogger Vasco Duarte, at January 07, 2014 3:23 PM  

  • I've been thinking about a similar idea - essentially applying the same thinking to the structure and design of a software system. We can get locked into ways of thinking due the the same kind of inertia.

    I like the idea of random perturbation. The experiment I would like to run is to delete certain parts of a code base, either specially selected or randomly selected (a la Netflix's Chaos Monkey), then as an exercise, rewriting the deleted section but without the burden of the existing mindset.

    By Blogger Joshua Lewis, at January 08, 2014 9:18 AM  

  • @joshua
    Great insight. Yes, indeed the same idea of "injecting randomness" can be used in the technical side as well!
    And you gave a great example the Chaos Monkey (which was also used by the Obama campaign in their IT systems pre and during election day) is a great example of how randomness can increase anti-fragility of a distributed system!
    Thanks for sharing that and let us know how the experiment works out :)

    By Blogger Vasco Duarte, at January 08, 2014 10:16 AM  

  • Hopefully I'll get to run the experiment :)

    I'll try report back

    By Blogger Joshua Lewis, at January 08, 2014 12:35 PM  

Post a Comment

Links to this post:

Create a Link

<< Home


 
(c) All rights reserved