Sensitivity analysis (SA) is an essential tool in modelling, used to evaluate how to input parameter changes affect a model’s outcomes. It is particularly important when uncertainty exists regarding the model’s parameters or when model predictions influence significant decisions.
This process not only helps to identify which parameters most influence the model’s behaviour but also provides insights into the robustness of the model and its predictions. By systematically varying input parameters, sensitivity analysis helps modellers understand the range of possible outcomes, thus improving the reliability and interpretability of the model.
Table of contents
What is Sensitivity Analysis?
At its core, sensitivity analysis examines how changes in the input values of a model influence its output. It helps to assess the degree of responsiveness of the model to different assumptions and conditions. There are two primary types of sensitivity analysis: parameter sensitivity and structural sensitivity. While both involve examining the model’s behaviour under varying conditions, parameter sensitivity focuses specifically on variations in model parameters.
Parameter sensitivity is typically tested by systematically altering the model’s parameters and observing how these changes affect the model’s behaviour. This process helps to identify which parameters are crucial for the model’s outcomes and provides insights into the level of uncertainty surrounding the model’s predictions.
Role of Sensitivity Analysis in Model Building
Sensitivity analysis serves multiple purposes in model building. One of its key roles is to assess the uncertainty associated with a model’s parameters. Often, model parameters are difficult to measure precisely, and the values chosen may only be estimates or approximations.
The modeller performs sensitivity analysis to determine how much uncertainty in the parameters can be tolerated. This helps identify the point at which uncertainty does not significantly affect the model’s predictions. The analysis helps build confidence in the model. It does so by identifying which parameters require high accuracy and which ones can be approximated with less concern.
In addition to quantifying uncertainty, sensitivity analysis can highlight leverage points in a model. A leverage point is a parameter that, when adjusted, can lead to a significant change in the system’s behaviour. Identifying these parameters is crucial because they can represent areas where small changes could lead to large-scale impacts, offering valuable insights for decision-making processes.
For example, in complex systems like climate modelling or economic forecasting, identifying such leverage points can lead to more effective policy interventions.
Parameter Sensitivity and Model Behavior
The process of parameter sensitivity involves varying model parameters within a defined range and observing how these changes affect the behaviour of the system. The results of such an analysis can reveal several important insights:
- Identification of Critical Parameters: Sensitivity analysis helps identify which parameters have the greatest impact on the system’s behaviour. For instance, in a climate model, parameters like carbon dioxide emissions or solar radiation may play a crucial role in predicting temperature changes. Understanding which parameters significantly affect the model’s predictions can guide the model in prioritizing data collection or refinement.
- Understanding System Dynamics: The modeller alters parameters and observes the changes in model behavior. Sensitivity analysis provides insights into the dynamics of the system being modelled. This can include how certain changes lead to exponential growth, asymptotic growth, or oscillatory behaviour. Such understanding is vital for improving the model’s accuracy and its ability to predict real-world outcomes.
- Model Validation and Calibration: Sensitivity analysis helps validate a model by comparing its predictions with real-world data. The model behaves as expected based on historical observations when parameter values are altered. This suggests that the model accurately reflects real-world dynamics. Conversely, if the model produces unrealistic results, it indicates the need for further calibration or refinement.
- Exploring Extreme Scenarios: One of the benefits of sensitivity analysis is the ability to explore how the model behaves under extreme conditions. By testing a wide range of parameter values, including those that are unlikely but still possible, modellers can identify potential risks or vulnerabilities in the system. This is particularly useful in fields such as risk assessment, where understanding worst-case scenarios is crucial for planning and mitigation.
Types of Sensitivity Analysis
Sensitivity analysis can be performed using various techniques, each suited for different purposes and model types. Some of the most common methods include:
- One-Way Sensitivity Analysis: This is the simplest form of sensitivity analysis. In this form, one parameter changes at a time while others remain constant. The analysis then observes the impact of this change on the output. For example, varying the probability of success for medical treatment might show how sensitive the model’s cost-effectiveness outcome is to this parameter. This type of analysis is useful for identifying which parameters have the most significant impact on the model.
- Multi-Way Sensitivity Analysis: Unlike one-way analysis, multi-way sensitivity analysis involves changing two or more parameters simultaneously. Researchers use this approach when they suspect interactions between parameters play a crucial role. This approach helps understand the model’s behavior. For example, in a healthcare model, both the cost of a treatment and its effectiveness may need to be varied together to assess their combined impact on the cost-effectiveness ratio.
- Probabilistic Sensitivity Analysis: In more advanced forms of sensitivity analysis, researchers treat the uncertainty in model parameters as a distribution. They do not treat it as a single value. This allows the modeller to account for variations in the parameters based on real-world data, such as confidence intervals. Probabilistic sensitivity analysis uses Monte Carlo simulations to run the model multiple times with different randomly selected parameter values, generating a range of possible outcomes. This method is particularly useful when dealing with models that include significant uncertainty.
- Extreme Sensitivity Analysis: This method involves testing the model under the best-case and worst-case scenarios, where all parameters are set to their extreme values. Extreme sensitivity analysis is helpful in identifying the potential bounds of model predictions and understanding the robustness of the model under extreme conditions.
Applications of Sensitivity Analysis
Researchers widely use sensitivity analysis across various fields. These fields include engineering, economics, environmental science, and healthcare. Some of the notable applications include:
- Risk Management: Sensitivity analysis plays a crucial role in risk assessment by identifying the most influential risk factors in a model. By understanding how changes in these factors impact the model’s outcomes, organizations can prioritize their efforts to mitigate the most significant risks. For example, sensitivity analysis is commonly used in financial models to assess the impact of interest rates, inflation, and market volatility on investment returns.
- Climate Change Modeling: In climate science, sensitivity analysis helps to evaluate the impact of different variables, such as greenhouse gas emissions, solar radiation, and deforestation rates, on global temperatures and other environmental outcomes. By testing the sensitivity of climate models to these parameters, scientists can better understand the range of possible climate futures and the uncertainty associated with their predictions.
- Healthcare Decision Making: Sensitivity analysis is often used in health economics to assess the cost-effectiveness of medical interventions. For instance, when evaluating the benefits of a new drug, sensitivity analysis helps determine how changes in key parameters, such as the drug’s effectiveness or cost, influence the overall cost-effectiveness ratio. This information is critical for healthcare policymakers who must make decisions based on uncertain or incomplete data.
- Policy Analysis: Sensitivity analysis is also an important tool in policy analysis, especially when it comes to making decisions that have significant economic or social implications. For example, government agencies may use sensitivity analysis to evaluate the effectiveness of different policy interventions under varying economic conditions.
Challenges and Limitations of Sensitivity Analysis
While sensitivity analysis is a powerful tool, it is not without its challenges. Some of the key limitations include:
- Complexity: For large and complex models with numerous parameters, sensitivity analysis can become computationally expensive and time-consuming. In such cases, modellers must prioritize which parameters to test, often relying on expert judgment or previous research to guide their decisions.
- Lack of Data: Sensitivity analysis often relies on accurate data for its inputs. In many cases, however, reliable data may be scarce or unavailable. This can limit the effectiveness of the analysis and make it difficult to draw definitive conclusions.
- Model Assumptions: Sensitivity analysis is based on the assumptions made when building the model. If these assumptions are flawed or unrealistic, the results of the sensitivity analysis may be misleading. As such, it is important to continuously update and refine the model based on new data and insights.
- Interdependencies: In some cases, parameters in a model may be highly interdependent, meaning that changing one parameter could also affect others. In such cases, a more sophisticated approach, such as multi-way sensitivity analysis, may be necessary to fully capture the relationships between parameters.
Practical Use of Sensitivity Analysis
To understand sensitivity analysis better, let’s consider an example of a custom moulder tasked with producing three types of glassware. The original problem, as described, includes the production of six-ounce juice glasses, ten-ounce cocktail glasses, and a new type of champagne glass.
The Setup of the Example
The challenge is to find the optimal product mix that maximizes profit while adhering to these resource constraints.
Solving the Linear Program Using the Simplex Method
We solve this linear programming problem using the simplex method. After applying this method, we obtain the optimal solution where:
This solution suggests that, based on the current constraints and contributions, producing champagne glasses is not worthwhile.
Shadow Prices and Reduced Costs
One of the key components of sensitivity analysis is the concept of shadow prices and reduced costs.
Shadow Prices
A shadow price represents the marginal value of increasing a resource by one unit. It is the change in the optimal objective function value per unit increase in the right-hand side of a constraint while keeping all other factors constant. In the context of the example, the shadow prices for different constraints are as follows:
- The shadow price for production capacity is 11/14 per hour, meaning that an additional hour of production capacity would increase the optimal objective function by this amount.
- The shadow price for warehouse capacity is 1/35 per hundred cubic feet of storage.
- The shadow price for the demand for six-ounce juice glasses is zero, as this constraint does not limit the solution.
These shadow prices provide valuable insights into the marginal value of each constraint. For example, if the production capacity were increased, the company could increase its total profit by 11/14 per additional hour of production.
Reduced Costs
The reduced cost is the marginal cost of increasing a decision variable from zero. It is important to identify whether introducing a new activity (like producing champagne glasses) would be beneficial. If the reduced cost is negative, it indicates that the new activity will decrease the objective function value.
In our example, the reduced cost of producing champagne glasses is negative, meaning that diverting resources to produce champagne glasses would reduce the profit. This aligns with the shadow price analysis, which shows that the opportunity cost of the required production and storage resources is higher than the potential contribution from champagne glasses.
Analyzing Changes Using Sensitivity Analysis
Now, consider what would happen if the right-hand side of a constraint changed. For instance, suppose the production capacity increases by one unit (from 60 to 61 hours). This change would increase the optimal objective value by the shadow price of production capacity, which is 11/14.
Similarly, if the warehouse capacity were increased, the shadow price of 1/35 would indicate an increase in profit. However, since the demand for six-ounce juice glasses is not binding (i.e., the slack variable for this constraint is positive), its shadow price is zero, implying no impact on the optimal solution from increasing the demand for this product.
Final Words
Sensitivity analysis is a vital tool in model evaluation and development, allowing modellers to assess the impact of parameter uncertainty on their predictions. By performing sensitivity analysis, modellers can identify key parameters, assess model robustness, and gain insights into system behaviour under different conditions.
Despite its challenges, sensitivity analysis remains an essential practice in fields ranging from engineering and economics to healthcare and environmental science. As modelling techniques continue to evolve, the importance of sensitivity analysis will only grow, ensuring that models remain reliable and useful in decision-making processes.
