Estimating the Software Development Projects. Part 1

My name is Oleksandr Katrusha, and I work as a Senior Engineering Manager. Throughout most of my career, I have been involved in the development and implementation of ERP systems, as well as the assessment and management of corresponding projects. Though the topic of project estimation is as inexhaustible as an atom, I hope my experience will be beneficial to many engineers and managers in the software development industry. Particularly valuable are the insights on what to do and what to avoid, how to steer clear of common errors in estimation and its communication.

This article comprises two parts. The first part provides general considerations, discusses the objectives and complexities of project development estimation, the structure of estimation, and measurement units, as well as the requirements for estimators and performers. The second part describes a step-by-step method for composing and formatting estimations, along with an attempt at a mathematical justification of the proposed method.

I hope this material will be useful to both beginners and experienced project managers, sales department staff, as well as anyone involved in the estimation, sale, and execution of software development projects.

By project estimation, I'll specifically mean the estimation of labor effort (efforts). Estimating costs such as office rent, hardware, software, licenses — is a topic for a separate article. Also, I'll practically avoid discussing the calendar planning of project execution, which often follows immediately after and based on the estimation of efforts.

I also want to note that in agile practices lately, it's become more common to talk about forecasting rather than estimation, thereby emphasizing the inherent uncertainty associated with project execution, and focusing more on achieving the client's business goals rather than meeting deadlines and budgets.

By estimation accuracy, I'll mean its ability to facilitate quality project planning and execution. Since over the course of a project's life, its boundaries, assumptions, and other parameters typically undergo significant changes, calculating the "mathematical" accuracy of estimation (i.e., the ratio of the plan to the actual) is practically impossible.

Uncertainty

One of the fundamental challenges of estimation lies in the contradiction between the inherent uncertainty of future project parameters and the need to make decisions here and now. We live in a world of annual budget cycles, tenders, ROI, time-to-market, and other practices and concepts that compel early investment decisions in the project lifecycle, precisely when the risks of underestimation or overestimation are greatest.

Moreover, during negotiations or participation in tenders for a contractor, there is additional pressure to reduce the estimation and take on more risk. It is essential to always bear this contradiction in mind during the sale, estimation, and execution of a project.

When Estimation is Needed

In what scenarios might it be necessary to estimate the efforts required to implement a project? There are several potential situations:

• Negotiations with the client regarding a new project or project phase.
• Responding to an RFP or participating in a tender.
• Budget planning and allocation of other organizational resources, such as scheduling.
• Calculating the ROI of the project during the pivotal decision-making phase of its launch.

If you need an assessment immediately?

If you're compelled to estimate a project right from the outset, during the stage of gathering business requirements or even earlier, there are a few ways to mitigate risk:

• Leveraging your experience from previous projects, describe and estimate the maximum theoretically possible scope, even to a somewhat absurd degree. However, each figure should have at least some rationale behind it. Typically, after this, the client tends to become more cooperative regarding providing the necessary input data, time, and resources for the estimation process.
• Provide a range for the estimation, emphasizing the uncertainty and level of risk inherent in the project at this stage. This allows project stakeholders to make more informed decisions regarding risk acceptance.
• Proceed iteratively, refining the forecasted figures along the way.

Purposes

The answer to the question "why to estimate?" may not be as straightforward. In addition to the actual calculation of labor and budget, there is a lot of work to be done in the estimation process to justify the forecast numbers and mitigate risks. I would highlight the following main objectives of the the estimation process:

• Define and constrain the project scope and establish assumptions under which the estimation is valid.
• Determine the price and cost of the project based on labor efforts.
• Substantiate the project budget to external or internal sponsors.
• Obtain input data for other processes such as sales, planning, employee hiring, and budgeting.

At the top of the objectives list is defining the project scope. It's hard to overstate the importance of this point. No estimation makes sense without a clear description of what and under what conditions we are estimating. In my opinion, the lack of a clear description of project scope and assumptions is one of the main reasons for problems in implementation.

The second and fourth points mentioned above are fairly obvious, whereas the third might be challenging for developers and those unfamiliar with bureaucratic structures to grasp. To allocate a budget, especially in large corporations, it is often necessary to take vague, absurd or non-existent requirements and make a beautifully reasoned assessment. Then, within this budget, the team could proceed to clarify the "real" requirements and implement them. Meanwhile, all participants in the process may well understand the absurdity, unreality, and variability of the requirements. However, the more detailed and professional your estimation looks, the higher the chances of getting the budget and executing the project "as needed".

This is particularly relevant for non-IT clients selling services and products with much less uncertainty than software development. Their managers may find it difficult to understand why the price for the same service can differ by orders of magnitude depending on "insignificant" changes in requirements, project phases, or suppliers.

On a higher level, according to Steve McConnell in his excellent book "Software Estimation", the main goal of estimation is to determine if the project goals are realistic enough to achieve, in other words, whether the project is worth pursuing at all.

Experiment

During project estimation seminars, I've asked the audience several times to estimate the total effort required for developing a fairly simple business application — a program for a florist with features for order management, fulfillment, payment, and basic customer information storage.

As a result, even senior developers and architects, who have worked together for many years on the same platform, with the same input data, in the same room, and reading the same text, provided estimates that differed by 20 times or more! The requirements were not very detailed, but sufficiently realistic, as if they were really formulated by the business owner.

Clearly, the reason for such disparity lies not in the technical competence of the developers, in which I had no doubt. Rather, it lies in the fact that they were estimating different scopes. Some of them were only considering the development of basic functionalities, while others were also thinking about requirements gathering, testing, potential changes, integrations, and risks.

The essence of the important part of the art of project estimation lies in being able to see the entire scope of work and risks. Another vital aspect is communication with all project stakeholders.

The Ideal Estimator

The ideal estimator is a developer with qualifications no lower than senior level, preferably with experience in unsuccessful estimations and failed projects. He or she should also have a good understanding of sales processes, estimation, planning, and project execution as a whole. At the same time, they should feel psychologically safe, i.e. they should not be afraid of being penalized for underestimating or overestimating specific tasks. Otherwise, he / she may tend to make more pessimistic estimates or refuse to give them at all. In contrast, the team may be eager to utilize all allocated time, and your project budget could inflate several times larger than necessary.

This phenomenon is well articulated in Parkinson's Law: "Work expands to fill the time available for its completion." An experienced developer will always find a way to spend 40 hours on a 16-hour task. Therefore, an inflated estimate can also impact actual labor efforts.

Moreover, obviously overestimating individual tasks can undermine the client's confidence in the entire assessment and its authors.

Cognitive Biases Affecting Estimation

Even the most experienced professionals can make mistakes. One of the reasons for errors, in my opinion, is cognitive biases — properties of our brains that hinder us from thinking rationally. You can read more about them in the book by Nobel laureate in economics Daniel Kahneman or on Wikipedia. Here, I'll describe three biases directly relevant, in my view, to our topic.

Planning error and optimism. We tend to think we're better than others. Therefore, we usually give our tasks a more optimistic estimate than similar tasks by other people. A good assessor should adequately assess his/her own strengths and be ready to resist psychological pressure techniques such as "why is it taking so long?!" questions, calmly arguing his/her position.

Also, everyone, even experienced developers, is inherently optimistic in estimating tasks. According to empirical data, developers on average give estimates that are underestimated by at least 30%. And this optimism, unfortunately, is supported by managers and other project stakeholders.

Errors in probability estimation. The actual time to complete a task is a random variable. Even people who have taken a university course in probability theory are not always able to manipulate them correctly. See, for example, the Monty Hall problem. This is especially true for conditional, very large, and very small probabilities, which insurance companies and lottery organizers successfully utilize. It can also be challenging for an estimator to distinguish, for example, the expected efforts from the most likely labor effort for a task.

To reduce the effect, agree with the developer on what you mean by optimistic, realistic, pessimistic, and overall task completion estimates. The question "how long can this task take?" implies a completely different answer than questions like: "What is the most likely time to complete this task?" or "What is the average time to complete this task?"

Anchoring effect. People subconsciously gravitate toward numbers they've heard or been suggested. Marketers know this well, for example. In our context, this could mean that:

• If you want to get truly independent figures from different estimators, you shouldn't inform them of each other's results.
• Avoid giving any hints, orientations, expectations, or budget options to the estimator before you receive their numbers. Otherwise, the result will be heavily distorted by your input data.

Performers

Another complexity is that estimations are usually made by senior developers, while the majority of tasks are carried out by mid-level or even junior developers. As research conducted in the early days of the industry has shown, the productivity of programmers of different qualifications can differ several times, and sometimes even by tens of times.

Recommendation here would be to conduct estimation for a mid-level developer, especially if the performers are unknown at the time of estimation. That is, the estimator should put themselves in the shoes of a mid-level developer. Then, with a balanced team in terms of qualifications, the estimation will be sufficiently realistic. However, this may require additional effort from the estimator and separate communication with them.

If you know for sure that there will be a qualification imbalance in your team, adjust the estimates accordingly. Based on my calculations, considering the productivity of developers of different qualifications and the time spent on training and reviewing junior colleagues, the productivity of such two teams may differ by approximately one and a half times.

Units of Measurement

Effort vs Duration. I prefer to estimate all tasks and projects in terms of effort (in person-hours) rather than duration (in days or weeks) for the following reasons:

• At the time of estimation, the team may not be fully formed or its composition may change in the future.
• Teams often get distracted by other projects or tasks.
• It is relatively easy to get a rough estimate of duration from labor costs by simply dividing by available resources.

Duration, on the other hand, is more intuitive and convenient for a stable team over short periods of time.

Person-hours, -days, -weeks, -months. Person-hours, in my opinion, are more optimal, and here's why:

• Different people, countries, and companies operate on different calendars. Working days and weeks may contain different numbers of working hours, and some team members may work part-time. This is especially relevant for distributed teams.
• Development rates in the global outsourcing market are typically quoted in person-hours. Therefore, by simply multiplying the estimate by the rate, you can obtain an approximate budget in terms of money.
• Time tracking, if available, is done in hours. It will be much more convenient to compare the plan with the actual time spent.
• Estimates in days are inherently less precise than in hours and may allow developers to be less responsible towards the estimation.

Degrees of two. Given the choice of man-hours as the unit of measurement, it seems appropriate to use degrees of twos to estimate specific tasks. They are quite intuitive, correspond well to the working time of an eight-hour working day and are easy to calculate:

0.5 – 1 hour — time for a minimally possible task; 2 hours — time that can be comfortably spent in front of the screen without distractions; 4 hours — half of the workday; 8 hours — a full workday; 16 hours — two workdays; 32 hours — "almost a week".

If necessary, intermediate numbers can be used: 6, 12, 40. Estimates above 40 hours may appear less reliable, indicating poorly controllable tasks. In such cases, it's desirable to break them down into subtasks.

Actual Hours vs Effective Hours. It's no secret that developers, and not only they, actually work about five to six hours out of an eight-hour workday. The rest of the time is spent on breaks, coffee, smoke breaks, chats, Facebook, and other interesting activities. It's nearly impossible to sustain creative work for eight effective hours over many consecutive days.

My recommendation is still to estimate projects in actual, or astronomical, hours rather than effective hours. And the main reason for this is simple: it's difficult to explain to the client why they're paying you for eight hours when you're only working six... Even if your client is an IT company and "does the same thing", it's not worth giving them an extra card in potential disputes, especially legal ones, that may arise during the project's lifespan. Besides, planning with actual hours is simpler: no need to introduce additional coefficients.

The downside of this approach becomes apparent with small tasks where there's no difference between effective and actual hours. Three tasks of two hours each or one task of six hours of effective time will be completed in a single eight-hour workday; this should be remembered when estimating and planning.

As we've discussed in the first part of the article, the purpose of estimation is not only to find workload but also to describe the scope, risks, and present them correctly to all project stakeholders. Therefore, the "Project Estimate" document should contain at least:

• Scope. What we're estimating in the form of a list of requirements, tasks, and what's not included in the scope.
• Assumptions and risks. Under what conditions the estimate is relevant and what, in our opinion, could go wrong.
• Numbers. How much labor would be required to do 1 under condition 2.
• Packaging. Last but not least. Properly presenting the estimate is just as important as estimating it accurately, especially in the sales process.

Note that the actual estimation in terms of numbers is only in the third place because a qualitative definition of the scope and assumptions is much more critical for project success than the accuracy of the estimate. Numbers are inseparable from what and under what conditions they are estimated. An error in estimating an individual task may be 5%, 10%, or even 50%, and it may not be critical for the project. Whereas an incorrect scope and risk assessment could increase work effort several times.

It's much cheaper to resolve all ambiguities, potential conflicts with the client, and other stakeholders at the beginning of the project than in its middle or end. This is why a clear description of the scope, assumptions, and risks is necessary. Therefore, don't hesitate to gently but persistently point out the lack of information or its inconsistency, client errors, and visible risks.

So, in the first part, we've covered the main challenges in project estimation, the goals of the estimation process, requirements for the estimator (developer), the structure, and units of measurement of estimation. In the next part, we will dive deeper into the labour-intensive process of its preparation.