Although, in theory, probabilistic modelling techniques such as Monte Carlo method provides better insights over deterministic models, there are many situations where probabilistic models may not even be appropriate to use.
Monte Carlo simulation is a probabilistic modelling approach that helps the users understand the possible outcome and the likelihood of certain events. On the other hand, deterministic models are “what-if” type models that helps us understand what is likely to happen if certain event occurs. If you would like to know more about Monte Carlo method, you can read this article to know more.
Monte Carlo simulation, as a concept, is more than half a century old. However, once upon a time, running these simulations would have required using “super computers” as the memory and processing power of personal computers were inadequate to run it. So, it was not a choice for many.
Today, we can run a simulation with most of the modern laptops. This has also probably resulted in an increase in its adoption and has also certainly increased the level of awareness and enthusiasm. In paper, it is considered superior to deterministic modelling approaches. This is because, while deterministic models tells us what is likely to happen, it doesn’t help us understand how likely it is to happen and what else could happen. On the other hand, Monte Carlo method can provide us all those answers.
But it is important to understand that Monte Carlo simulation or probabilistic modeling, in general, need not always be a superior alternative to deterministic models. In fact, there are many scenarios where the Monte Carlo method is inappropriate and could give misleading results.
Let us understand when not to use the Monte Carlo method.
The Monte Carlo method cannot be used if the underlying distribution is unknown
Let me use a very simple example, here. We all know that the outcome from fair dice is uniformly distributed. The probability of getting a number that is 3 or below is 50% and the probability of getting a number above 3 would also be 50%. The average would be 3.5 (average of 1 to 6) and the standard deviation would be 1.71.
For a moment, let us say you did not know that it is uniformly distributed.
What would be our conclusion if we applied a normal distribution to know the probability that the value is 3 or below? The probability would be approximately 38.0% [Try Norm.dist(3,3.5,1.71,TRUE) in MS Excel ]
Is this result reliable or is it misleading? Thus, if we use a wrong distribution, the results we obtain would be completely useless.
This is a very big limitation in corporate finance as we often do not get sufficient to ascertain the underlying probability distribution.
And as mentioned before, a Monte Carlo simulation with the wrong distribution would only lead us to an incorrect outcome.
Monte Carlo method would be unreliable if the key parameters are unknown
This limitation is in continuation of the above. In order to build a reliable Monte Carlo model, it is not just enough to know what distribution the data follows but it is also important to know the underlying parameters. For instance, if we were to assume that the underlying data follow a distribution, we still need to ascertain its mean and standard deviation. For instance, let us say we need to project profits for a new business. Since these are a new business there are hardly any past data to characterize the distribution.
This author has seen quite a few financial modellers in that scenario choose the triangular or PERT distribution to model the underlying random variables. These distributions require analysts to give three data points: the most likely outcome, the worst case, and the best case.
Even if we assume that the triangular distribution is the appropriate distribution, the result of this simulation model would be reliable only if the three parametric values were correct. If we are not sure of the three values, then the probability data we obtain would be misleading and unreliable.
It is highly expensive for optimization problems
Optimization problems involve using trial and error method to find out what should be an input value so that a particular objective is met.
For example, let us say we are building a model where demand is a function of price i.e. if price increases volume would fall and it would increase if the price is reduced. The company may want to set the price with the objective that we maximize the profit.
In deterministic models, it is relatively easy to do it through trial and error. We first build the model with price being an input. Once we estimate the profit, we can run solver (if you are using MS Excel) to maximize profit by changing the price. Even if the system requires 1000 trials to optimize the price, you may be able to get the answer in less than 30 seconds in a typical consumer segment laptop.
In Monte Carlo simulation models, the outcome is random. The model objective would not be the profit but either average of profits across trials or a probability number. Let us say our Monte Carlo model runs 10,000 trials (very common) to compute the averages and probability.
Our optimization would now have to be done like this. First take one price as a fixed value, run trials with 10,000 set of random numbers, obtain the average. Change the price, run trials with 10,000 set of random numbers and keep repeating. So, if you want to try 1,000 different prices, we would have 10,000 times 1,000 (i.e. 10 million) trials have to be run. This can take a very very long time running into days, not minutes or hours.
This author has used tool like ‘Evolver’ in Pallisade’s @Risk software which seems to have managed to do it reasonably faster. But it is still a very time-consuming task.
Cannot answer “What-if” questions
Sometimes as a decision-maker we just want to know what is likely to happen if certain events occur. A project manager may want to know what the impact on profit would be if we crash the project timelines by making people work overtime. If this analysis has to be done, the project timeline cannot be stochastic (i.e. random variable). It must be a deterministic variable.
Similarly, let us say an airline company wants to know how much the cost could increase if fuel cost were to go up by 10 percent. This is possible only in a deterministic model. A simulation model would take the price as a random variable and hence it would not be possible to capture that result.
As we can see from the above, there are many reasons why we may want to choose deterministic models over stochastic models including Monte Carlo method. In fact, this author would argue that situations where Monte Carlo method can be reliably applied are cases where we may be able to use many other machine learning algorithms.