Sensitivity analysis 620083 219036488 2008-06-13T08:07:03Z 130.89.143.221 "'''Sensitivity analysis''' (SA) is the study of how the variation (uncertainty) in the output of a [[mathematical model]] can be apportioned, qualitatively or quantitatively, to different sources of variation in the input of a model.<ref>Saltelli et al., 2007</ref>". In more general terms uncertainty and sensitivity analyses investigate the robustness of a study when the study includes some form of [[mathematical modelling]]. While uncertainty analysis studies the overall [[uncertainty]] in the conclusions of the study, sensitivity analysis tries to identify what source of uncertainty weights more on the study's conclusions. For example, several guidelines for modelling [http://www.epa.gov/crem/library/CREM%20Guidance%20Draft%2012_03.pdf (see e.g. one from the US EPA)] or for [[impact assessment]] [http://ec.europa.eu/governance/impact/docs/SEC2005_791_IA%20guidelines_annexes.pdf (see one from the European Commission)] prescribe sensitivity analysis as a tool to ensure the quality of the modelling/assessment. The problem setting in sensitivity analysis has strong similarities with [[Design of experiments]]. In design of experiments one studies the effect of some process or intervention (the 'treatment') on some objects (the 'experimental units'). In sensitivity analysis one looks at the effect of varying the inputs of a mathematical model on the output of the model itself. In both disciplines one strives to obtain information from the system with a minimum of physical or numerical experiments. ==Overview== Most mathematical problems met in social, economic or natural sciences entail the use of mathematical models, which are generally too complex for an easy appreciation of the relationship between input factors (what goes into the model) and output (the model’s dependent variables). Such an appreciation, i.e. the understanding of how the model behaves in response to changes in its inputs, is of fundamental importance to ensure a correct use of the models. A [[mathematical model]] is defined by a series of [[equations]], input factors, parameters, and variables aimed to characterize the [[process]] being investigated. Input is subject to many sources of uncertainty including errors of [[measurement]], absence of information and poor or partial understanding of the driving forces and mechanisms. This uncertainty imposes a limit on our [[confidence]] in the response or output of the model. Further, models may have to cope with the natural intrinsic variability of the system, such as the occurrence of [[stochastic]] events. Good modeling practice requires that the modeler provides an evaluation of the confidence in the model, possibly assessing the uncertainties associated with the modeling process and with the outcome of the model itself. [[Uncertainty]] and Sensitivity Analysis offer valid tools for characterizing the uncertainty associated with a model. [[Uncertainty analysis]] (UA) quantifies the uncertainty in the outcome of a model. Sensitivity Analysis has the complementary role of ordering by importance the strength and relevance of the inputs in determining the variation in the output. In models involving many input variables sensitivity analysis is an essential ingredient of model building and quality assurance. National and international agencies involved in impact assessment studies have included section devoted to sensitivity analysis in their guidelines. Examples are the European Commission, the White House Office for Budget and Management, the Intergovernmental Panel on Climate Change and the US Environmental Protection Agency. ==Methodology== There are several possible procedures to perform uncertainty (UA) and sensitivity analysis (SA). The most common sensitivity analysis is [[Sampling (statistics)|sampling]]-based. A sampling-based sensitivity is one in which the model is executed repeatedly for combinations of values sampled from the [[probability distribution|distribution]] (assumed known) of the input factors. Sampling based methods can also be used to decompose the variance of the model output (see references). In general, UA and SA are performed jointly by executing the model repeatedly for combination of factor values sampled with some [[probability distribution]]. The following steps can be listed: #Specify the target [[function (mathematics)|function]] and select the input of interest #Assign a [[probability density function]] to the selected factors #Generate a [[matrix (mathematics)|matrix]] of inputs with that distribution(s) through an appropriate design #Evaluate the model and compute the distribution of the target function #Select a method for assessing the influence or relative importance of each input factor on the target function. ==Applications== Sensitivity Analysis can be used to determine: #The model resemblance with the process under study #The [[quality]] of model [[definition]] #Factors that mostly contribute to the [[output]] variability #The region in the [[space]] of [[input]] factors for which the model [[variation]] is maximum #[[Optimization (mathematics)|Optimal]] - or instability - regions within the space of factors for use in a subsequent [[calibration]] study #Interactions between factors Sensitivity Analysis is popular in [[financial]] applications, risk analysis, [[signal processing]], [[neural networks]] and any area where models are developed. Sensitivity analysis can also be used in model-based policy assessment studies see e.g. [http://www.modeling.uga.edu/tauc/background_material/Washington-Main.pdf]. ===Environmental === Computer environmental models are increasingly used in a wide variety of studies and applications. For example [[global climate model]] are used for both short term [[weather forecasts]] and long term [[climate change]]. Moreover, computer models are increasingly used for environmental decision making at a local scale, for example for assessing the impact of a waste water treatment plant on a river flow, or for assessing the behavior and life length of bio-filters for contaminated waste water. In both cases sensitivity analysis may help understanding the contribution of the various sources of uncertainty to the model output uncertainty and system performance in general. In these cases, depending on model complexity, different sampling strategies may be advisable and traditional sensitivity indexes have to be generalized to cover [http://www.iemss.org/iemss2006/papers/s7/268_Fasso_0.pdf multivariate sensitivity analysis], [[heteroskedastic]] effects and correlated inputs. ===Business === In a decision problem, the analyst may want to identify cost drivers as well as other quantities for which we need to acquire better knowledge in order to make an informed decision. On the other hand, some quantities have no influence on the predictions, so that we can save resources at no loss in accuracy by relaxing some of the conditions. See [[Corporate_finance#Quantifying_uncertainty|Corporate finance: Quantifying uncertainty]]. Sensitivity analysis can help in a variety of other circumstances which can be handled by the settings illustrated below: * to identify critical assumptions or compare alternative model structures * guide future data collections * detect important criteria * optimize the tolerance of manufactured parts in terms of the uncertainty in the parameters * optimize resources allocation * model simplification or model lumping, etc. However there are also some problems associated with sensitivity analysis in the business context: * Variables are often interdependent, which makes examining them each individually unrealistic, e.g.: changing one factor such as sales volume, will most likely affect other factors such as the selling price. * Often the assumptions upon which the analysis is based are made by using past experience/data which may not hold in the future. * Assigning a maximum and minimum (or optimistic and pessimistic) value is open to subjective interpretation. For instance one persons 'optimistic' forecast may be more conservative than that of another person performing a different part of the analysis. This sort of subjectivity can adversely affect the accuracy and overall objectivity of the analysis. ==See also== * [[Perturbation analysis]] * [[Robustification]] ==Footnotes== {{Reflist}} ==Bibliography== *Cacuci, Dan G., Mihaela Ionescu-Bujor, Michael Navon, 2005, <I>Sensitivity And Uncertainty Analysis: Applications to Large-Scale Systems (Volume II)</I>, Chapman & Hall. *Fassò A. (2007) Statistical sensitivity analysis and water quality. In Wymer L. Ed, <i>Statistical Framework for Water Quality Criteria and Monitoring</i>. Wiley, New York. *Fassò A., Esposito E., Porcu E., Reverberi A.P., Vegliò F. (2003) Statistical Sensitivity Analysis of Packed Column Reactors for Contaminated Wastewater. <i>Environmetrics</i>. Vol. 14, n.8, 743 - 759. *Fassò A., Perri P.F. (2002) Sensitivity Analysis. In Abdel H. El-Shaarawi and Walter W. Piegorsch (eds) <i>Encyclopedia of Environmetrics</i>, Volume 4, pp 1968–1982, Wiley. *J.C. Helton, J.D. Johnson, C.J. Salaberry, and C.B. Storlie, 2006, Survey of sampling based methods for uncertainty and sensitivity analysis. <I>Reliability Engineering and System Safety</I>, <b>91</b>:1175-1209. *Homma, T. and A. Saltelli (1996). Importance measures in global sensitivity analysis of nonlinear models. <i>Reliability Engineering and System Safety</i>, <b>52</b>, 1–17. *Kennedy, P. (2007). <I>A guide to econometrics</I>, Fifth edition. Blackwell Publishing. *Morris, M. D. (1991). Factorial sampling plans for preliminary computational experiments. <i>Technometrics</i>, <b>33</b>, 161–174. *Rabitz, H. (1989). System analysis at molecular scale. <I>Science</I>, <b>246</b>, 221–226. *Saltelli, A., S. Tarantola, and K. Chan (1999). Quantitative model-independent method for global sensitivity analysis of model output. <i>Technometrics</i> <b>41</b>(1), 39–56. *Cacuci, Dan G. <I>Sensitivity & Uncertainty Analysis, Volume 1: Theory</I>, Chapman & Hall, 2003. *Saltelli, A., K. Chan, and M. Scott (Eds.) (2000). <I>Sensitivity Analysis</I>. Wiley Series in Probability and Statistics. New York: John Wiley and Sons. *Saltelli, A. and S. Tarantola (2002). On the relative importance of input factors in mathematical models: safety assessment for nuclear waste disposal. <i>Journal of American Statistical Association</i>, <b>97</b>, 702–709. *Santner, T. J.; Williams, B. J.; Notz, W.I. <I>Design and Analysis of Computer Experiments</I>; Springer-Verlag, 2003. *Saltelli, A., S. Tarantola, F. Campolongo, and M. Ratto (2004). <I>Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models</I>. John Wiley and Sons. *Saltelli, A., M. Ratto, S. Tarantola and F. Campolongo (2005) Sensitivity Analysis for Chemical Models, <I>Chemical Reviews</I>, 105(7) pp 2811 – 2828. *Saisana M., Saltelli A., Tarantola S., 2005, Uncertainty and Sensitivity analysis techniques as tools for the quality assessment of composite indicators, <I>Journal Royal Statistical Society A</I>, <b>168</b> (2), 307-323. *Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D. Saisana, M., and Tarantola, S., 2007, <I>Global Sensitivity Analysis. The Primer</I>, John Wiley & Sons. See [http://sensitivity-analysis.jrc.ec.europa.eu/ A forum on sensitivity analysis] for more information. *Sobol’, I. (1990). Sensitivity estimates for nonlinear mathematical models. <i>Matematicheskoe Modelirovanie</i> <b>2</b>, 112–118. in Russian, translated in English in Sobol’ , I. (1993). Sensitivity analysis for non-linear mathematical models. <I>Mathematical Modeling & Computational Experiment (Engl. Transl.)<I>, 1993, <b>1</b>, 407–414. *Sobol’, I. M. <I>Mathematical Modeling & Computational Experiment (Engl. Transl.)<I>, 1993, <b>1</b>, 407. Two new special issue devoted to sensitivity analysis will appear in 2008: one on Reliability Engineering and System Safety (RESS) and one on the International Journal of Chemical Kinetics. Both are selection of papers presented at the 2007 Conference of Sensitivity Analysis of Model Output (SAMO) held in Budapest in June. See [http://samo2007.chem.elte.hu/ SAMO 2007] for the slides of the presentations. A Special Issue on Sensitivity Analysis has been published in the Journal Reliability Engineering and System Safety (Volume 91, 2006). See [http://www.sciencedirect.com/science?_ob=PublicationURL&_tockey=%23TOC%235767%232006%23999089989%23627574%23FLA%23&_cdi=5767&_pubType=J&view=c&_auth=y&_acct=C000036252&_version=1&_urlVersion=0&_userid=4900406&md5=2e0db332618dbf12cbb137b5be9feb64]. ==External links== * [http://sensitivity-analysis.jrc.ec.europa.eu/ A forum on sensitivity analysis] (main source - also includes a tutorial and a bibliography.) * [http://simlab.jrc.ec.europa.eu/ The SIMLAB software for sensitivity analysis: download it for free] * [http://home.ubalt.edu/ntsbarsh/senanaly/SenAnaly.htm Sensitivity Analysis Index] * [http://cran.r-project.org/src/contrib/Descriptions/sensitivity.html The sensitivity package: sensitivity analysis in R] * [http://www.odellion.com/pages/online%20community/NPV/sensitivity.htm Sensitivity definition, and finance applications] [[Category:Business terms]] [[Category:Scientific modeling]] [[Category:Process management]] [[de:Sensitivitätsanalyse]] [[fi:Herkkyysanalyysi]] [[zh:敏感度分析]]