Predictable futures: a management myth

I would like to look at the assumption underlying many project plans, that the future is actually predictable. All management models have this assumption at their core and all organizations spend a lot of time planning all kinds of things: budgets, revenues, project outcomes, etc. This article will spend some time identifying the flaws of these models and give some tips on how to overcome them. I hope you will find them both useful and interesting.

The standard management model

For decades, management tools and techniques have been studied and developed. Intuition and gut feel have been surpassed. A standard model of management has arisen, based on the belief that management is a science resulting in predictable outcomes. Project management is one element of this science, and project planning and estimating a core pillar of ensuring predictability.

Good managers make a variety of predictions against the future. These predictions are based on models of how the world operates. The manager predicts how this will be achieved, what budget is required, and the plans for achieving it. The predictions are documented in business cases, project plans, forecasts, financial models and budgets. These describe what is expected to happen in the future.

The culture of business values detailed predictions above broad brush estimates. Businesses value managers whose results or actuals match their predictions. This is reinforced by the press, commentators and stock markets. If you cannot predict accurately surely that means you do not know what you are doing?

Flaws in this model

There are several fundamental flaws in this model. They are not hard to identify, and deep down everyone in business knows them. The flaws are clearly expressed by those who are new to business – asking questions like why on earth do we do it like that?

The most obvious problems are that there is always too limited information to make an accurate prediction. On top of this, the world changes and so the conditions at the time of the prediction are not the same as at the time of the result. We have limited foresight. This is why forecasts make risks and assumptions explicit, and if they are sophisticated, provide some form of sensitivity analysis.

The problem of limited information is real, but also generally understood. There are more insidious problems deriving from the status of management laws and the behaviour of managers.

Management “laws”

Management is not a fully understood science whose pronouncements have the validity of laws of nature. It is at best a series of reasonably consistent empirical viewpoints based on limited samples of data. The rules and algorithms are mostly rough estimates and rules of thumb. Management forecasts usually concern the behaviour of people, and assume people are rational. There is plenty of evidence human beings are not. There is nothing wrong with this, as long as the limitations of management models are always borne in mind. All too often, they are not.

Underlying this is a confusion between broad generalisations and universal laws. When an action results in an outcome, it is assumed that this is repeatable, this inductive logic is the basis of science. But it is only repeatable if the conditions are the same. Conditions are never exactly the same and it is not clear what makes conditions sufficiently similar for an action to result predictably in the same outcome. Rather than doubt the generalisation, it is assumed that conditions are similar enough – and best practice is developed, which is treated as if it is a law. This can be seen in the tendency in business to go from “I have observed” to “it is true that” to “you should” to “everyone must” without the requisite additional evidence.

Management behaviour

Another issue is management behaviour. I am not suggesting that all managers behave as I am about to describe – but some, and often many, do.

Managers are not stupid. Managers know that understandings are incomplete and therefore they build in safety factors into predictions. Often huge safety factors which over-estimate needs, at least if the manager can get away with it. This results in the normal budgeting game where managers ask for as much as they can, and their line managers challenge them and try to reduce it as much as possible. Neither side really knows what is required – but the assumption is that if the manager is challenged enough, the resulting compromise will be right. If a manager gains a larger budget than required, the aim is to achieve expected results using up the entire budget. Hence the actuals will match predictions.

Managers use the process of prediction not to find answers but to justify an approach. Managers know the answer needed from a prediction to justify their approach. Inputs, algorithms and assumptions are fudged to produce the answer desired.

Some managers actually don’t care about predicted results. If the manager is making a prediction several years into the future, she knows her tenure in role will be shorter than this. She has no reason to worry if the prediction is right or wrong. It is just a mechanism to gain approval. The first thing a new manager taking over the role will is to reset the baseline of all the major plans and forecasts and blame any problems on her predecessor.

Flaws in the objectives

Even if the management models are flawed, surely, what they are trying to achieve is right? If we continue to collect evidence and build better models won’t this solve the problem? It may help, but it will not solve the underlying problem. The problem is not just the way the standard model achieves its objectives. It problem is that the objectives are flawed. The objectives are:

  • To have perfect predictability
  • To replace subjectivity and intuition with universal best practice and science

Perfect predictability is a myth, an unobtainable goal. Anyway, shouldn’t managers be aiming for the best result, rather than the most predictable?

Many successful entrepreneurs rely on intuition. They tend to smirk at management models and calculations. Intuition is fast and uses very limited resources in coming to decisions and insights. Modelling, forecasting, and predictions use up huge amount of resources, without actually delivering anything of real value. A business makes no product, sells nothing and satisfies no customer by forecasting or planning.

You don’t want better algorithms and even more data. You want better intuition. The reason managers avoid intuition is because it does not fit in the way organisations work. No one gains a budget because they convince their manager that they have an intuition.

The problem with universal best practice is that it goes against all needs of competitive business. If you apply the same best practice as your competitors, you will be the same as them. When someone does something better, you will be uncompetitive or may go out of business. And although best practice implies there should not be something better, sooner or later there will be.

But ... it’s still worth planning!

The conclusion from this may seem to be, why should I bother to make predictions and plans? There are good reasons why you should bother.

The first is that forecasting, planning and budgeting are part of the ritual of business. Every society has its own rituals and customs, and if you want to be part of it, you need to conform. You can argue all you like about the flaws in budgets and forecasts, but if you don’t play along, you won’t be in the business for long. As much as anything planning, budgeting, and forecasting processes are political processes. You will be judged not simply for your results, but how well you act the expected role of a manager in performing these activities. If you do not perform, you will get nowhere.

Secondly, flawed is not the same as being of no use. I have been very critical, but good forecasts and plans, used appropriately, are powerful. Although it may be rare, there are some situations in which you understand the conditions well, and your level of accuracy in a forecast will be high. But even more generally, forecasts, plans, and predictions are useful, as long as:

  • You use them as guides, not answers
  • You understand the degree of accuracy and sensitivities in them
  • You are aware of the risks and assumptions made in plans, and if these turn out to materialise, then you do something about it
  • You invest a proportionate amount of time in developing forecasts. Ideally, this is a relatively small proportion of your overall time

Finally, as general guidance, be cynical about anything labelled best practice. It may or may not be. If you are presented with best practice, ask who defined it as best practice, based on what sample of data.

What do you think?

I would be very interested to hear your views about this topic. Whether you agree or completely disagree let me know your thoughts. I think having a conversation on this topic would lead to very interesting outcomes.

The article is based on a chapter of my book “The Management Book”, published by Financial Times Prentice Hall in September 2011.

Related content:

Best Practice, Continuous Improvement and Progress in Project Management
Project Management Methodologies: Guidance or Rules?
Six Symptoms of Poor Prioritisation

- nice2 - sales1