As today's engineering systems have become increasingly sophisticated, assessing the efficacy of their safety-critical systems has become much more challenging. The more classical methods of “failure” analysis by decomposition into components related by logic trees, such as fault and event trees, root cause analysis, and failure mode and effects analysis lead to models that do not necessarily behave like the real systems they are meant to represent. These models need to display similar emergent and unpredictable behaviors to sociotechnical systems in the real world. The question then arises as to whether a return to a simpler whole system model is necessary to understand better the behavior of real systems and to build confidence in the results. This question is more prescient when one considers that the causal chain in many serious accidents is not as deep-rooted as is sometimes claimed. If these more obvious causes are not taken away, why would the more intricate scenarios that emanate from more sophisticated models be acted upon. The paper highlights the advantages of modeling and analyzing these “normal” deviations from ideality, so called weak signals, versus just system failures and near misses as well as catastrophes. In this paper we explore this question.

@en

Even in a pandemic there seem to be inherent conflicts of interest between the individual and societal consequences of remedial actions and strategies. Actions taken in the sole interests of patients, as required by the Hippocratic oath, can have broadly inconvenient economic implications for the State. (“Average” benefits for a population can impose individual inconveniences for the vulnerable.). Understandably these decisions are not normally made explicitly and transparently by governments. This leads to seemingly illogical and inhumane strategies which are not understood and hence mistrusted and often ignored by the public. Vaccination sentiments on social media are often an unwanted symptom of this dilemma. This article outlines and discusses a number of examples of such situations with a focus on ethical aspects. It concludes that each case must be considered individually as to the issues that need to be weighed in these difficult decisions; and that there are no clear and universally acceptable ethical solutions. What can be learned from the COVID-19 crisis is that short term utilitarianism has consequences that in the eyes of the population are unacceptable. This lesson seems equally valid for cost benefit evaluations regarding other risks, such as from hazardous industries, flood defenses, and air transport. Decisionmakers and politicians can learn that persuasion only goes so far. In the end the people appear to prioritize in terms of deontology.@en

Legacy risks from infrastructures and industrial installations often reveal themselves when a potential for failure has been discovered much later than at the stage of the design and construction of a structure. In which case, there might already be a problem with the legacy installation, or even a crisis, without having had an accident. When the hazard cannot be taken away, the question arises as to how much effort, if any, should be spent on improving the situation. The usefulness of the three archetypical approaches to this problem: setting a standard, the as low as reasonably practicable approach and a case-by-case discourse approach are discussed for their applicability for these legacy risks. Although it would be desirable to retrofit legacy risks to previously set legal requirements as is the case when acceptability limits are set in law or demonstration of ALARP (As Low As Reasonably Achievable) is demanded, it may be impossible to reduce the residual risk to an otherwise acceptable level without taking away or replacing the infrastructure, which is not acceptable either. Therefore in conclusion the only available solution to persistent legacy risk problems seems to be to have a thorough discussion with all relevant stakeholders until an agreement is in some way found.

@en

In analysing the performance of complex sociotechnical systems, of particular interest is the inevitable and inherent variability that these systems exhibit, but can normally tolerate, in successfully operating in the real world. Knowing how that variability propagates and impacts the total function mix then allows an understanding of emergent behaviours. This interdependence, however, is not readily apparent from normal linear business process flow diagrams. An alternative approach to exploring the operability of complex systems, that addresses these limitations, is the functional resonance analysis method (FRAM). This is a way of visualising a system’s behaviour, by defining it as an array of functions, with all the interactions and interdependencies that are needed for it to work successfully. Until now this methodology has mainly been employed as a qualitative mind map. This paper describes a new development of the FRAM visualisation software that allows the quantification of the extent and effects of this functional variability. It then sets out to demonstrate its application in a practical, familiar test case. The example chosen is the complex sociotechnical system involved in a Formula 1 pit stop. This has shown the potential of the application and provided some interesting insights into the observed performances.@en

The assumption that risk, represented as an expected value of the loss could be implied to be a measure of safety, in a cost benefit analysis, is firmly entrenched in economic risk analysis. However, this does not mean that without a marker, the value of a loss, can be established with any necessary level of certainty to make such a cost balancing act ethically possible. The appropriateness of using the Value of a Statistical Life (VOSL) at all in a safety analysis, is a matter of perspective, which renders attempts to establish a uniform value of a statistical life questionable. This makes it questionable whether decisions from which values for a VOSL were evaluated, really were based on consideration of saving lives, or whether other arguments, such as available budget, were much more dominant. Ethical considerations do not seem to be in the frame of corporate risk management, where loss-of-life catastrophes appear to be simply the cost of doing business. Because there is no real basis for any estimate of the value of a statistical life, the values employed in cost-benefit analyses therefore only seem to serve the purpose of dissembling, concealing that the decision is taken on grounds other than saving human lives, or even that potential harm to humans was not even considered. The strict meaning given to resilience as at most to make a plan for recovery and see if we can live with the consequences, seems just another step towards putting the economy before people.

@en

Since Nassim Taleb coined black swan as an event that occurred as a complete surprise for everybody, the metaphor of the black swan has been applied to a much wider variety of events. Black swan events now comprise events that are a surprise for some but not for others, events that have a low likelihood, events that were not believed to be possible but still proved to be possible, events that were dismissed as being too improbable to worry about but happened anyway. For a decision maker the black swan problem is choosing where to put effort to prevent, or mitigate events for which there are warnings, or for which the possibility has been put forward. Does the fact that there are thousands of books written about fire breathing dragons warrant the development of an Anti-Dragon Defense Shield? The black swan may have been a surprise for Willem de Vlamingh in 1697, it was not a surprise for the inhabitants of Australia, for which the appearance of tall white humans was their “black swan event”. In this paper we explore the options available to decision makers when confronted with the various sorts of swan (or dragon) events.

@en

In the philosophy of SAFETY-I variability is seen as a threat, because it brings with it the possibility of an unwanted outcome. Variability of hardware is curtailed by, amongst other things, precise specifications. Variability of human behavior is curtailed by inter alia regulations and protocols. In the philosophy of SAFETY-II variability is seen as an asset. In SAFETY-II, humans are seen as able to cope with the variability of technology circumstances to keep systems working. This capacity of coping has been designated resilience. Recently the meaning of resilience has been stretched to include the ability of restoring the operational state after an excursion into the realm of inoperability. The belief that humans will cope if an unexpected situation may arise, reduces the emphasis on preventive measures that limit the probability that the system may behave in an unsafe manner. The stretched meaning of resilience exacerbates this problem, because there is no real limit of what systems or society using these systems may bounce back from. The philosophies behind resilience engineering promote safety by exploiting the ingenuity of humans to keep systems within the desired operating envelope. Unfortunately the errors that may be introduced by over-relying on humans correctly assessing situations may also be fatal or catastrophic: maybe not for society as a whole, but surely for an individual, a group of individuals, or a company.@en

In the philosophy of SAFETY-I variability is seen as a threat, because it brings with it the possibility of an unwanted outcome. Variability of hardware is curtailed by precise specifications, controlled manufacturing and installing. Variability of human behaviour is curtailed by training and selection of personnel and by regulations, prescriptions and protocols. In the philosophy of SAFETY-II variability is seen as an asset. In SAFETY-II, humans are seen as able to cope with the variability and imperfections of technology and the variability of circumstances to keep systems working. In SAFETY-II this capacity of coping has been often designated as resilience. Recently the meaning of resilience has been further stretched to include the ability of restoring the operational state after an excursion into the realm of inoperability, or failure. Artificial intelligence allows systems to evolve by processing information acquired by sensing the result of their actions and variable environment in which they operate. This makes such systems intrinsically more variable than deterministic systems and therefore less predictable. For operators of these systems it is essential that they understand and are able to deal with this variability in order to keep systems operational and adaptive on the one hand and prevent excursions into unwanted territory on the other. The SAFETY-II philosophy seems to be more suitable to such an environment. At the same time it increases uncertainty about potential future states. The belief that humans will cope if an unexpected situation may arise, will reduce the emphasis on defensive, prevention measures that can limit the probability that the system may behave in an unwanted, unsafe manner. The stretched meaning of resilience exacerbates this problem, because there is no real limit of what systems or society using these systems may bounce back from. A highway bridge that collapses can be re-built. Thus society is resilient against bridge collapses. The question is however, should society accept a situation in which there is a significant probability that such a bridge collapses as safe or safe enough. The philosophies behind SAFETY-II and resilience engineering promote safety by exploiting self-correcting mechanisms in technology and the ingenuity of humans to keep systems within the desired operating envelope. In this approach, a form of trial, error and correct, the prior occurrence of the error, or deviation is essential. Unfortunately the error may also be fatal or catastrophic: maybe not for society as a whole, but surely for an individual, a group of individuals or a company. With an increasing tendency to evaluate every decision in terms of – preferably monetarized – costs and benefits, striking a balance between a SAFETY-I, a SAFETY-II and a resilience approach is not made easier by the inherent vagueness of the definition of success and the essentially qualitative nature of the latter two concepts. In this paper we explore how Safety I, Safety II and resilience can be cast in a way that one levers off the strengths of each one to compensate for the weaknesses of the other.@en

In making decisions, rationality is often equated to economic rationality. This means that in every decision, the benefits should outweigh the costs, when both are expressed in monetary terms. Balancing of cost and benefits through monetary Cost Benefit Analysis (CBA), which is used more and more widely in health and safety decision-making, evokes the criticism that it leads to decisions in which only money counts; and all that cannot be expressed in money, or is perceived of no monetary value, is neglected. An important parameter in the CBA rationality, is the value of a statistical life (VOSL). Scientists serving decision makers in the attempts to monetize the VOSL have spent decades of research into what a reasonable value should be. These evaluations of the VOSL lead to widely varying results. This wide variation seems to move decisions on risks to life and health, from the political arena to the scientific laboratory. Scientists are required produce the right number after which politicians can then decide on the basis of CBA. In this paper it is argued that rather than attempting to harmonize on an average with large margins of uncertainty, the conclusion can be drawn that a consistent valuation of a human life cannot be expected. One should accept that standardization of the VOSL is limited by the – lack of – similarity in nature of the activity and the nature of the risk. In many cases one also has to accept the only available alternative not involving violence, which is a political debate, terminated by the more general rule of law or constitution on how to settle such a debate and then accept the decision.

@en