Two frequent questions we get asked at Fluid UI is what the benefit of software prototyping is, and whether it is a more or less effective approach to gathering software requirements when compared with other more traditional approaches.
We’ve investigated much of the published research relating to companies that use prototyping focused requirements gathering practices. When combined, these show a definitive increase in project success when prototyping is used. In addition, by piecing together the reports and studies and related materials, we can make some useful estimates around the efficiency gains associated with prototyping - plenty to make the case that every software project needs to use prototyping as an approach, and that the benefits far outweigh the cost.
One Pro tip at this stage is to always remember to start every project with a one pager.
It’s not hard to find someone in the software development field who knows and understands the effectiveness of prototyping but doesn’t have access to the hard data that backs up their wisdom. Once a project is complete, no company is going to go back and redo a project just to provide a suitable data point. Similarly, setting up a controlled academic test of a suitably complex piece of software is also prohibitively expensive. To date, we’ve had little to back up our instincts except “it’s just better - trust me” - an answer which can result in less technically inclined project stakeholders expressing concerns about whether the time and cost needed to create a prototype can be justified.
The research we’ve summarised below will at last put this concern to rest. But let’s start with a simple question.
The cost of fixing issues at the various stages of the software development life cycle is well known and documented. It’s a classic hockey stick shaped graph that any investor or financial wizard would die for (but sadly in our case, could result in the death of our project, or in the worst of cases, the firing of your entire 200 person team).
The cost of fixing issues at the different stages of the software life cycle
From this, we can tell that any time we let an issue slip past the requirements phase, it could end up costing up to 100 times more to get it fixed later on. By showing that prototyping captures more detailed and accurate requirements, we can estimate the overall impact of software prototyping on project budgets and successful delivery rates.
Instinctively, we know that prototyping is the key to efficient requirements capture, and by consequence, critical to bringing software projects in on time and on budget. The additional clarity of communication, tightening of scope and shared understanding created by a prototype is simply a better way of defining requirements. Everywhere we have looked, we’ve found this to be the case.
In “Prototyping: A practitioner’s guide”, Todd Zaki Warfel talks about a UK based consultancy that switched from a traditional requirements gathering process to a prototyping based one and measured the changes in productivity that came about as a result. Unsurprisingly, the results were astounding:
Clearly we are heading the right direction. By eliminating 75% of the most expensive and time consuming work (post launch rework and fixes) we’ve already made massive saving in the project budget. The amazing thing is however - the requirements phase actually took less time and also made the project timeline more predictable than the previous approach too, completely debunking the “We don’t have time to prototype” myth. Rather, the opposite is true - you simply cant afford the time not to prototype.
A famously absent-minded Irish prime minister once said of an idea “That’s fine in practice, but will it work in theory?”
When it comes to prototyping, practice and theory most definitely align. A UCLA experiment points to impressive improvements in productivity when comparing requirements gathering methods in a like-for-like experiment:
“…some development teams used conventional development methodologies while others employed prototypes in the software development process (with no particular emphasis on the interface). Code of the final systems produced by prototyping groups was only about 40 percent as large as that of their counterparts, possibly at a cost in generality of design. Finally, the prototyping groups accomplished their task with 45 percent less effort than the other groups.”
So for smaller test projects and studies under more controlled circumstances, similar gains are still seen when using prototypes - this time in the form of a 45% increase in productivity and a 60% smaller code base (which in turn bodes well for future maintainability and stability - other key considerations in terms of managing a living, breathing software product).
So far, these two cases clearly show that there is a clear relationship between using prototypes to gather user requirements and the time, cost and predictability of the overall project.
Having to rework existing features and code once operational is the biggest single cost overrun risk of a software project. Away from academia and back in the world of business, the Borland corporation released research detailing the costs of having a poor requirements capture process:
“One analysis of the potential return on investment from better requirements suggests that requirements errors can consume between 70 and 85 percent of all project rework costs”
and
“…rework can consume 30 to 40 percent of the total effort expended on a software project”
Similarly, a report on the Impact of Business Requirements on the Success of Technology Projects noted that the two most important elements driving the degree to which a project will be successful are the management of scope and the excellence in elicitation of requirements.
“…there is strong support for the finding that there is a 60% time and cost premium to be paid on projects with poor quality requirements”.
There is a clear correlation between improving how you capture requirements (using tools like Fluid UI) and the likelihood of your project coming in on time and on budget. Many of the frequent reasons against prototyping (e.g. “we don’t have time” or “we don’t have the budget”) are simply a misunderstanding of how important it is to accurately and sensibly capture requirements and gain consensus before launching into the development of a software project. Failing to take this time will, on average, cause your project to be delayed and cost anywhere between 40% and 60% more than needed.
It is definitely possible to deliver projects on time and on budget without using a high quality requirements gathering process such as prototyping - the impact of business requirements study makes this point quite clearly. But while an individual project might come in on time and on budget with a less effective or non-existent process, the study points out that it is very unlikely that even an above average team will succeed in making this happen in a repeatable manner.
Using a prototype to gather requirements is actually cheaper for many projects while providing a host of other benefits. So, for the good of your budget, your development team and your sanity that you would insist on creating prototypes for your next project - and that you point anyone who disagrees in this direction of this post.
