47:, a method used to communicate the quality of quantitative information with the generation of `Pedigrees' of numbers. Likewise, sensitivity auditing has been developed to provide pedigrees of models and model-based inferences. Sensitivity auditing is especially suitable in an adversarial context, where not only the nature of the evidence, but also the degree of certainty and uncertainty associated to the evidence, is the subject of partisan interests. These are the settings considered in
36:
things, derive from the relevance of the policy study to different constituencies that are characterized by different norms and values, and hence by a different story about `what the problem is' and foremost about `who is telling the story'. Most often the framing includes implicit assumptions, which could be political (e.g. which group needs to be protected) all the way to technical (e.g. which variable can be treated as a constant).
35:
In settings where scientific work feeds into policy, the framing of the analysis, its institutional context, and the motivations of its author may become highly relevant, and a pure SA - with its focus on quantified uncertainty - may be insufficient. The emphasis on the framing may, among other
223:
Saltelli, Andrea, Ăngela; Guimaraes
Pereira, Jeroen P. van der Sluijs, and Silvio Funtowicz. 2013. âWhat Do I Make of Your Latinorum. Sensitivity Auditing of Mathematical Modellingâ. International Journal of Foresight and Innovation Policy 9 (2/3/4): 213â34.
190:"In general sensitivity auditing stresses the idea of honestly communicating the extent to which model results can be trusted, taking into account as much as possible all forms of potential uncertainty, and to anticipate criticism by third parties."
307:
Funtowicz, S.O. and Jerome R. Ravetz (1991). "A New
Scientific Methodology for Global Environmental Issues." In Ecological Economics: The Science and Management of Sustainability. Ed. Robert Costanza. New York: Columbia University Press:
129:
These rules are meant to help an analyst to anticipate criticism, in particular relating to model-based inference feeding into an impact assessment. What questions and objections may be received by the modeler? Here is a possible list:
175:"where there is a major disagreement among stakeholders about the nature of the problem, ⌠then sensitivity auditing is more suitable but sensitivity analysis is still advisable as one of the steps of sensitivity auditing."
71:
Science, coined in 1994 by
Gibbons et al., refers to a mode of production of scientific knowledge that is context-driven, problem-focused and interdisciplinary. Sensitivity auditing consists of a seven-point checklist:
267:
Van der Sluijs JP, Craye M, Funtowicz S, Kloprogge P, Ravetz J, Risbey J (2005) Combining quantitative and qualitative measures of uncertainty in model based environmental assessment: the NUSAP system. Risk
Analysis
289:
Gibbons, Michael; Camille
Limoges; Helga Nowotny; Simon Schwartzman; Peter Scott; Martin Trow (1994). The new production of knowledge: the dynamics of science and research in contemporary societies. London: Sage.
180:"Sensitivity auditing, is a wider consideration of the effect of all types of uncertainty, including structural assumptions embedded in the model, and subjective decisions taken in the framing of the problem."
320:
Funtowicz, S. O., & Ravetz, J. R. 1992. Three types of risk assessment and the emergence of postnormal science. In S. Krimsky & D. Golding (Eds.), Social theories of risk (pp. 251â273). Westport, CT:
27:(SA) of a model-based study is meant to demonstrate the robustness of the evidence provided by the model in the context whereby the inference feeds into a policy or decision-making process.
240:
67:, which proposes a methodology of inquiry that is appropriate when âfacts are uncertain, values in dispute, stakes high and decisions urgentâ (Funtowicz and Ravetz, 1992: 251â273).
23:
for use in policy-relevant modelling studies. Its use is recommended - i.a. in the
European Commission Impact assessment guidelines and by the European Science Academies- when a
93:
Check if the data used in the model were manipulated to make the results look more certain than they really are, or if they were made overly uncertain to avoid regulation.
185:"The ultimate aim is to communicate openly and honestly the extent to which particular models can be used to support policy decisions and what their limitations are."
342:
79:
Ask if complex math is being used when simpler math could do the job. Check if the model is being stretched beyond its intended use.
295:
171:
Sensitivity auditing is described in the
European Commission Guidelines for impact assessment. Relevants excerpts are (pp. 392):
100:
It's better to find problems in your study before others do. Do robust checks for uncertainty and sensitivity before publishing.
337:
114:
Ensure your model is addressing the correct issue and not just solving a problem that isn't really there.
43:
to provide an assessment of the entire knowledge- and model-generating process. It takes inspiration from
39:
In order to take these concerns into due consideration, sensitivity auditing extends the instruments of
40:
24:
20:
280:
Funtowicz, S. O. & Ravetz, J. R. 1993. Science for the post-normal age. Futures, 25(7), 739â755.
56:
48:
86:
Find out what assumptions were made in the study, and see if they were clearly stated or hidden.
121:
Conduct in-depth tests to measure uncertainty and sensitivity using the best methods available.
291:
162:
Your scenarios only capture a limited set of the possible development/evolution of the system
60:
255:
204:
331:
64:
256:
Making sense of science for policy under conditions of complexity and uncertainty
205:
Making sense of science for policy under conditions of complexity and uncertainty
137:
It would be sufficient for a 5% error in X to make your statement about Z fragile
107:
Don't keep your model a secret. Make it clear and understandable to the public.
140:
Your model is but one of the plausible models - you neglected model uncertainty
225:
147:
143:
You have instrumentally maximized your level of confidence in the results
134:
You treated X as a constant when we know it is uncertain by at least 30%
203:
describes in detail sensitivity auditing in its 2019 report entitled â
68:
52:
241:
European
Commission. 2021. âBetter Regulation Toolbox.â November 25.
200:
44:
199:
The
European Academiesâ association of science for policy
167:
Sensitivity auditing in the
European Commission Guidelines
254:Science Advice for Policy by European Academies,
153:You have artificially inflated the uncertainty
316:
314:
276:
274:
8:
125:Questions addressed by sensitivity auditing
226:https://doi.org/10.1504/IJFIP.2013.058610
250:
248:
216:
7:
236:
234:
159:You are answering the wrong question
156:Your framing is not socially robust
150:- why should I trust your results?
14:
59:(PNS) is a concept developed by
91:Avoid Garbage In, Garbage Out:
1:
343:Environmental social science
112:Focus on the Right Problem:
359:
98:Prepare for Criticism:
119:Do Thorough Analyses:
84:Look for Assumptions:
338:Sensitivity analysis
41:sensitivity analysis
25:sensitivity analysis
21:sensitivity analysis
17:Sensitivity auditing
57:Post-normal science
49:Post-normal science
19:is an extension of
350:
322:
318:
309:
305:
299:
287:
281:
278:
269:
265:
259:
252:
243:
238:
229:
221:
146:Your model is a
77:Use Math Wisely:
61:Silvio Funtowicz
358:
357:
353:
352:
351:
349:
348:
347:
328:
327:
326:
325:
319:
312:
306:
302:
288:
284:
279:
272:
266:
262:
258:, Berlin, 2019.
253:
246:
239:
232:
222:
218:
213:
197:
169:
127:
105:Be Transparent:
33:
12:
11:
5:
356:
354:
346:
345:
340:
330:
329:
324:
323:
310:
300:
282:
270:
260:
244:
230:
215:
214:
212:
209:
196:
193:
192:
191:
187:
186:
182:
181:
177:
176:
168:
165:
164:
163:
160:
157:
154:
151:
144:
141:
138:
135:
126:
123:
32:
29:
13:
10:
9:
6:
4:
3:
2:
355:
344:
341:
339:
336:
335:
333:
317:
315:
311:
304:
301:
297:
296:0-8039-7794-8
293:
286:
283:
277:
275:
271:
268:25(2):481-492
264:
261:
257:
251:
249:
245:
242:
237:
235:
231:
227:
220:
217:
210:
208:
206:
202:
194:
189:
188:
184:
183:
179:
178:
174:
173:
172:
166:
161:
158:
155:
152:
149:
145:
142:
139:
136:
133:
132:
131:
124:
122:
120:
115:
113:
108:
106:
101:
99:
94:
92:
87:
85:
80:
78:
73:
70:
66:
65:Jerome Ravetz
62:
58:
54:
50:
46:
42:
37:
30:
28:
26:
22:
18:
303:
285:
263:
219:
198:
195:SAPEA report
170:
128:
118:
116:
111:
109:
104:
102:
97:
95:
90:
88:
83:
81:
76:
74:
38:
34:
16:
15:
332:Categories
321:Greenwood.
211:References
55:science.
148:black box
308:137â152.
31:Approach
294:
69:Mode 2
53:Mode 2
51:or in
201:SAPEA
45:NUSAP
292:ISBN
63:and
207:â.
117:7.
110:6.
103:5.
96:4.
89:3.
82:2.
75:1.
334::
313:^
273:^
247:^
233:^
298:.
228:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.