There is a dilemma in the IT industry, and it is all about getting off the ground. It would appear that projects are like aeroplanes; the larger they are the longer the runway they require, and once they get too big they don’t get off the ground at all. If you’ve been in IT for any time at all, I’m sure you will have come across at least one legendary project that seems to run and run without end, never quite delivering anything of real value.
Those technologists who work in large organisations bemoan the inevitability of failure at scale whilst staring longingly at the few young upstarts who deliver at blistering speed in trendy start-ups. They point out how easy it would be if they could work in a green-field environment with no constraints and none of the rigour required for large systems.
But that is simply a luxury you can’t hope to have in any “serious” company. Even start-ups get slow as they grow… don’t they?
Well yes, in many cases they do. Of the ones that actually survive (and of course this is one of the biggest risks of working in such a world), many start-ups turn into average companies and then into lumbering corporations. The same people work there, but they’ve lost the spark and become the very thing they used to mock. Many of them even buy a suit, and start using Powerpoint!
However, in some rare cases the agility remains, and new stuff gets done quickly and often. This leads us to ask the question, how can this be possible, and is it something that can be introduced into a company that has long since lost that loving feeling?
The attempts that have been made to achieve this are many and varied and most result in failure. Innovation teams sprout up, only to wither on the vine, starved of funding and lacking any true stakeholder buy-in. Companies strip away governance in the hope that things will progress faster, and then discover the mess they get into when large projects go off down blind or dangerous alleys.
They even try agile, introducing it to a sceptical workforce but even when they manage to create new evangelists for iterative development they still end up with something that an old colleague of mine referred to as “water-scrum-fall”. Things go so well at first as they try it out on small projects, and then as they scale up it all goes wrong. Once more the size overwhelms them and the speed disappears.
But agile seems so promising. It reinvents something the older amongst us instinctively knew when we started out in software development; you always start with the user need (which is very different to the user want). Requirements capture, huge documents, and wall sized Gantt charts felt like something from another world; one populated not by engineers but by accountants and lawyers. Agile felt right because it felt creative and dynamic, and that’s what we were doing; creating new things that had never been seen before, using technologies we didn’t really understand yet.
The size problem is still the killer though. With a small number of people, the agile creative approach works beautifully. As the number of people grows, the stand-ups become unwieldy and communications break down. So, you break the project down into manageable units, each delivered by a single agile team, but then find yourself needing to do more up-front design work to ensure that each team knows the boundaries of what it is delivering.
This is the thinking that leads to “water-scrum-fall”, an uncomfortable union in which agile teams deliver component parts of a much larger solution that in turn has emerged from waterfall style thinking. For some this seems ideal as it keeps all of the “best practice” of the “tried and tested” methods whilst still allowing the development teams to get busy (and wear jeans). Unfortunately it misses the point completely as those “best practices” are the very problem.
Discovery is fundamentally different to requirements capture. You are not asking the user to describe what they want in abstract terms and dry words; you are observing them and working with them to prompt thinking and try out new approaches. You fail fast in order to succeed faster. Alpha and Beta are fundamentally different concepts as well. You don’t design then build then test; you design, build and test as a single activity but in an iterative manner, learning as you go. If you embed this approach inside waterfall then you are not doing agile at all; you are just pretending. How often have you heard people who have adopted agile refer to the introduction of a “fire break” between discovery and alpha, and between alpha and beta to allow them to “get the paperwork together and apply the governance processes?”
In agile these things are not abandoned, but they are also not done as a separate activity. They are instead done in-flight as an integral part of each iteration. Flipping the problem doesn’t work either. Trying to embed waterfall inside or alongside an agile approach simply leads to a contention between the desire to do things (waterfall) and the intent to do them iteratively.
This is the point where I mention the wake-up call regarding waterfall. It doesn’t work at scale! Sorry to be the bringer of bad news, but just because something seems to work for small situations doesn’t mean it will scale, and waterfall doesn’t scale. The concepts are good in theory, and when planning your revision timetable, or managing a small team of people performing a well-defined and repeatable activity the tools of the trade work very well, but therein lies the problem. It is a very rare event in IT when we do the same thing for a second time.
Each time we build a system we are building it for the first time, and we have no information on which to base a meaningful plan. All the analysis and research in the world will not give you the answers you need; you will only find out anything real once you start building. Waterfall does not allow this, and this is why agile wins every time. There is also the reality problem. The disciplines of waterfall are fine in theory, but as you scale up the time and resource required scales up linearly; there are no economies of scale. Unless you have an infinite number of monkeys you are not going to get the complete works of Shakespeare and even if you did, no-one has time to read it.
So what is needed to break out of the big-company thinking patterns that lead to programme bloat and long running programmes that promise the world but never seem to come to fruition? At the start of this post I talked about the delivery mentality that allowed start-ups to work at a pace unmatched in larger organisations, and it is from this model that I draw my solution.
There is more to the start-up world than just the advantage of small size. There are also the external pressures that are brought to bear. Start-ups are dependent on funding for survival in a way that far exceeds any equivalent pressures in more established companies. They are acutely aware of their own vulnerability and that in turn increases the volume of the ticking of the clock to deafening levels. Every day counts and quite often, every hour counts.
This creates appetite and pace.
There is also the sense of purpose that exists in a start-up. There is of course a sense of purpose in more established organisations but it is never as personal and certainly never as all consuming. In an organisation a bad year for the company is a disappointing year for the employee. In a start-up, a bad month for the company can be disastrous for the individual. A large organisation becomes like a machine whilst a start-up is a living entity with moods and emotions.
This creates engagement and personality.
Finally, there is something bigger to aim for. Everyone in a start-up is aware of the potential exit or growth strategies. The commitment in time and energy is worth it because firstly at some point it will come to an end and something new will begin, and secondly everyone will benefit from the outcome.
This creates motivation, commitment and persistence.
Duplicating these environmental factors might just make large companies deliver benefit at speed, and in the second part of this post I will describe how to foster this start-up mentality and give your organisation pace, personality and persistence.
To be continued…
RegardsThe Enterprising Architect