By Dr Simon Place, Lecturer in Air Transport Engineering, Department of Air Transport
SubscribeThere is a current move to widen the adoption of Safety Management Systems (SMS) within the air transport industry, driven by ICAO among others, in particular for airports and air transport operators.
This carries with it a need to undertake risk assessments where necessary, either qualitative or quantitative.
As an example in the UK, the CAA have published “Safety Management Systems for Commercial Air Transport Operations”CAP712 [CAA, 2002], which describes the means of implementation of SMS by an aircraft operator. Risk assessment is an essential part of such a system, and CAP712 therefore includes a risk tolerability matrix for use when quantifying risk. Risk is normally calculated on a “per-hazard” basis, and is equal to the consequence of the hazard (in terms of equivalent fatalities or cost) multiplied by the frequency.
The discipline of risk assessment has most often been applied in the field of safety assessment of aircraft systems, as required for aircraft certification under FAR23 and FAR25 and (in Europe) the EASA Certification Specification (CS)-23 and 25. Techniques for accomplishing the assessment of safety are quoted by the SAE in their Aerospace Recommended Practice (ARP)-4761. However, work is now ongoing in the assessment of risk in the Flight Operations; this article will describe some of the tools, techniques and information sources that are available.
One of the most promising techniques under development is the Flight Operations Risk Assessment System, known as FORAS [Hadjimichael & McCarthy, 2004]. This is a risk management tool to ‘encode’ human knowledge about a type of risk and does not depend on statistical probabilities, but on knowledge of variables that constitute risk. The FORAS methodology employs a ‘fuzzy’ expert system to identify the factors which have the greatest impact on overall risk. It is based on five principles:
1.Risk models are based on human expertise
2.Focus on preventing accidents, by identifying accident/incident precursors.
3.Risk analyses must be rapid, consistent and independent of bias from individual users.
4.Risk assessments must be quantitative to facilitate comparison and communication.
5.FORAS analysis is intended to be used for communicating risk to all levels in an organisation.
A research project is currently being undertaken with EVA Air to build knowledge based model for Approach to land. However in its current form, the model is only suited for bespoke risk assessment of an enclosed system.
A different approach has been adopted by[Luxhøj, 2003] who has developed the Aviation Safety Risk Model (ASRM). This makes use of the Human Factors Analysis & Classification System (HFACS) proposed by [Wiegmann and Shappell, 2003]. HFACS is a classification scheme which has been developed to capture and analyse the different types of human error that occur. The framework draws on the work of Reason, who developed the so-called “Swiss-cheese” model of accident causation[Reason, 1990]. ASRM was originally developed for use by US Naval Aviation, but has since been used more widely within the aviation industry. The ASRM uses Bayesian Belief Networks to model the uncertainty within the model, using either data or the opinion of “experts”. The network is created to represent the dependencies between the different factors identified by applying HFACS. Data has been obtained by examining case studies of accidents, e.g. Air Ontario Flight 1363[Luxhøj, 2003].
An additional technique has been adopted by [Bazargan & Ross, 2004], who have carried out a risk assessment of General Aviation. This has used the proportionate occurrence of causal factors obtained from accident reports, where fatalities or serious injuries were reported. This information is then combined with expert judgments on the relative importance of the flight attributes using Analytical Hierarchy Process (AHP). The latter allows the development of importance weights for different criteria in a decision process, which in this case are the flight characteristics (e.g. hazards, Flight phase and pilot experience).
With regard to helicopter safety, work carried out in the off-shore helicopter sector has also been undertaken by a number of bodies. One of the most relevant to this project was carried out by the Norwegian Industrial management organisation, SINTEF. The work was published in two reports entitled “Helicopter Safety Study”; (HSS-1) was based on work carried out in 1989/1990, and has since been updated by HSS-2[Hokstad et al, 1999].The work focused on North Sea helicopter accidents and incidents in the period 1990-1998 in order to calculate the risk in terms of fatalities per million person flight hours.
This was achievable due to the recording of flight hours and personnel carried by the North Sea operators. The work also studies the factors that influence risk in terms of Frequency (e.g. Operations Procedures, Air Traffic & Navigational services) and Consequence (e.g. Impact absorption upon hard landings, Stability on sea). These factors are termed Risk Influencing Factors (RIFs). Estimated values for the respective importance of each RIF were elicited by means of a series of expert panels, which then allowed the RIFs with the largest impact on overall risk to be identified.
There are other (non-Flight operations specific) risk assessments studies which are also worthy of note. The FAA carried out a risk assessment for Land & Hold Short Operations at Airports [FAA, 1999], using quantitative information represented by Fault and Event Tree Analysis (ETA and FTA) in order to calculate risk. This was possible due to the limited nature of the risk assessment and readily defined scope of the assessment.
Other techniques were adopted to calculate public risk in vicinity of Schiphol airport [RAND, 1993] and for the risk analysis of aerodrome design rules for the Norwegian CAA[AEA Technology, 2001]. The work by RAND in modelling risk in the region surrounding Schiphol airport was based on parameters which represented population distribution, flight operations data, aircraft fleet data and aircraft accident rates. The effects of different risk reduction strategies were also included, e.g. adding an additional runway or changing the aircraft fleet mix. The work to analyse aerodrome design rules was based on accident data for runway over-runs and under-shoots, and applied this with the dimensions of the runway available. Dimensions were then calculated to achieve the required Target Level of Safety (TLS) and recommendation made as to remedial action to be taken to reduce risk, if needed.
Further information has been published by the Global Aviation Information Network (GAIN), which has set up Working Group B to promote safety management systems for the aviation industry. This includes fostering the use of existing analytical methods and supporting the introduction of new methods and tools. The published guidance is contained in the “Guide to Methods & Tools for Airline Flight Safety Analysis” [GAIN, 2003], which provides useful advice on a wide range of risk assessment tools. It also contains advice on the use of Fault Tree Analysis (FTA), Event Tree Analysis (ETA), software packages such as @Risk[1] as well as more recent techniques such as FORAS (see above).
In terms of data to be used in a risk assessment, one the most regular sources of accident data is the Statistical Summary of Commercial Jet Accidents [Boeing, 2004]. This data represents information drawn from worldwide aircraft operations from 1959 to 2003. It presents data on the types of accidents and the phase of flight in which they occurred. Similar reports are available from the UK CAA Safety Regulation Group, which has produced reports containing data drawn from accident and incident sources world-wide, in the Global Fatal Accident Review CAP681[CAA, 1998]. This report is significant in that it contains judgements of the causal factors of each fatal accident to aircraft >5.7 tonnes. It also contains circumstantial factors which could have had a bearing on each accident.
The NTSB have produced downloadable databases for all US aircraft accidents from 1982 – 2002, which are available on the Internet. These are in the form of MS AccessTM databases that may be sorted by any of the > 50 data fields, including “aircraft manufacturer”, “weight” or “type of operation”, e.g. FAR Part 91, 121 and 135. Other information includes accident narrative, sequence of events, number of fatalities etc. The NTSB also publishes annual reports for both Commercial – Part 121 and Part 135 [NTSB, 2004a] and General Aviation – Part 91, the most recent of which covers the year 2000. However, web-based data are available up to 2004.
Further work remains to be carried out in this most interesting field, where the modelling of causal factors will be used in order to quantify risk. Such modelling will be used to help decide where improvements need to be made in aircraft operations, through for example, adoption of different procedures, more training or more equipment (e.g. for situational awareness) on board aircraft.
References
1.AEA Technology, Final Report on the Risk Analysis in support of Aerodrome Design Rules, Report No AEAT/RAIR/RD02325/R/002 Issue 01 (2001).
2.Bazargan M & Ross D L, A Comparative Risk Measure for General Aviation, presented at MCDM, Whistler, B C Canada Aug 6-11, 2004
3.Boeing, Statistical Summary of Commercial Jet Airplane Accidents Worldwide Operations 1959-2003, May 2004.
4.CAA Safety Regulation Group, CAP 681 Global Fatal Accident Review 1980-1996. Civil Aviation Authority, Gatwick (1998).
5.CAA Safety Regulation Group, CAP 712 Safety Management Systems for Commercial Air Transport Operations. Civil Aviation Authority, Gatwick (2002).
6.FAA, Pilot Safety brochure, “Prevention of Controlled Flight into Terrain In General Aviation Operations”, ref DOT/FAA/AM-400-02/2
7.FAA, Land and Hold Short Operations Risk Assessment, Sep 1999.
8.GAIN Guide to Methods & Tools for Airline Flight Safety Analysis, 2nd edition, 2003.
9.Hadjimichael M & McCarthy J, Implementing the Flight Operations Risk Assessment System, in 57th International Air Safety Seminar Shanghai, China, Nov 2004.
10.Hokstad P, Jersin E, Hansen G K, Sneltvedt J and Sten T, Helicopter Safety Report 2, SINTEF Report No STF38 A99423, 1999
11.Luxhøj, J T, “Probabilistic Causal Analysis for System Safety Risk Assessments in Commercial Air Transport,” in Proceedings of the Workshop on Investigating and Reporting of Incidents and Accidents (IRIA), Williamsburg , VA, Sep 2003.
12.NTSB, US Air Carrier Operations Calendar Year 2000, Annual Review of Aircraft Accident Data, Report No NTSB/ARC-04/01 (2004a).
13.RAND, Airport Growth & Safety, A study of the External Risks of Schiphol Airport and possible Safety-Enhancement Measures, 1993.
14.Reason J, Human error, New York, Cambridge University Press, 1990.
15.Wiegmann D A & Shappell S A, A Human Error Approach to Aviation Accident Analysis, Ashgate, 2003.
