A multicriteria model to evaluate airport ramp safety: A study at São Paulo International Airport

This paper presenting as contribution a model for the evaluation, control, and improvement of airport safety by analyzing different operational events. This model is based on multicriteria methods, in particular, the entropy and TOPSIS methods. It allows to operationalize the evaluation of the airport ramp safety performance in contexts of scarce information about the weights of each criterion, and where the absence of personal judgments in the weighting process is considered important. Operational events were used to develop the criteria. In addition, an application was conducted and the data from airport ramp safety events were used to identify trends, diagnose problems, support the planning of actions, and provide a basis for allocating resources for prevention of catastrophic accidents and improvement of safety. This application was used to validate the model. It was concluded that the airport ramp safety did not maintain a trend of continuous improvement and the proposed model allowed the identification of trends, of criteria with low performance, and the realization of recommendations for improvinge airport safety.


RESUMEN
Este trabajo se presenta como contribución a un modelo para la evaluación, el control y la mejora de la seguridad del aeropuerto mediante el análisis de diferentes eventos operacionales.Este modelo se basa en métodos multicriterio, en particular, los métodos de entropía y TOPSIS.Permite hacer operativa la evaluación del rendimiento de seguridad de la rampa del aeropuerto en contextos de escasa información acerca de la ponderación relativa de cada criterio y en donde se considera importante la ausencia de juicios personales en el proceso de ponderación.Se utilizaron los eventos operacionales para desarrollar los criterios.Además, se llevó a cabo una aplicación y se utilizaron los datos de los eventos de seguridad de la rampa del aeropuerto para identificar tendencias, diagnosticar problemas, apoyar la planificación de acciones y proporcionar una base para la asignación de recursos para la prevención de accidentes catastróficos y mejora de la seguridad.Esta aplicación se utilizó para validar el modelo.Se concluyó que la seguridad de la rampa del aeropuerto no mantuvo una tendencia de mejora continua y que el modelo propuesto permitió la identificación de tendencias, de criterios con bajo rendimiento y la realización de recomendaciones para mejorar la seguridad de los aeropuertos.

INTRODUCTION
Although major air disasters are rare, less catastrophic accidents and a wide variety of operational safety events occur frequently.These events of smaller proportions and those that occur with a major frequency possibly indicate safety problems that could result in more serious accidents [1].
According to [2], since the characteristics of operations in aviation differ from those in other fields, comparing the performance indicators of safety in aviation with those in other fields is inappropriate.
No country, association, or company has declared its own indicators or evaluation models of airport operational safety.Therefore, there are opportunities for contributions concerning the model to be adopted.This model should provide methods to evaluate the performance of airport operational safety to enable continuous improvement, understand the main questions related to safety, and indicate actions to resolve them.
To analyze the problems of operational safety in airports, this study proposes a multicriteria model that tries to evaluate the performance of airport safety by (i) developing performance indicators that characterize airport operational safety, (ii) measuring the performance level of these indicators, (iii) weighing the indicators of operational safety in the absence of information to support this process, (iv) analyzing the overall airport safety performance to verify if it follows a continuous improvement pattern over the years, (v) identifying trends and diagnosing problems in the airport ramp safety, and (vi) making recommendations to assist the planning of actions to improve the performance of airport operational safety.
In this context, unicriteria and multicriteria performances were used as indicators for the continuous monitoring of the operational airport safety level, as recommended by the International Civil Aviation Organization (ICAO) in its audit programs [1].Moreover, the historical evolution of the global performance of the airport ramp was evaluated with the objective to identify trends and analyze them.
In the developed model, the entropy method [3] was used to calculate the weights of the criteria and the technique for order preference from similarity to ideal solution (TOPSIS) method [4] that was used to assess the performance of the airport ramp safety on the basis of the number of occurrences of operational safety events.In this sense, we applied the proposed method at the São Paulo/Guarulhos International Airport to analyze the consistency, efficiency, and effectiveness of its application in practice.
Briefly, the rest of the manuscript is structured as follows: the section "Airport Safety Evaluation" presents the theoretical foundation of airport safety, the basics concepts, and how to monitor the airports' safety performance; the section "Multicriteria Decision Aid" describes the entropy and TOPSIS methods, which were used in this study for weighting the criteria and assessing the overall safety level; the section "Proposed Model" presents the model to evaluate the performance of airport safety, the steps to use it, and the methods that integrate it; the section "Application" describe the use of the proposed model at the Guarulhos/São Paulo International Airport, including the criteria used in the assessment and their weighting, the description of the steps for performance evaluation and recommendations for improving airport safety; finally, the section "Conclusions" presents the final remarks.

AIRPORT SAFETY EVALUATION
The ICAO defines safety as the state in which the possibility of damage is reduced and maintained below an acceptable level through a continuous process of hazard identification and safety risk management [1].
In the context of an airport, the identification of hazards and safety risk management are conducted by means of procedures and methods that comprise an airport safety system.According to [5], an airport safety system consists of mechanisms aiming to control the risks that may affect the safety of the stakeholders while ensuring compliance with relevant legislation.
The airport management should provide goals for the prevention of accidents, which should guide the procedures for continuous monitoring of safety and the implementation of mechanisms for investigation and taking corrective actions in case of non-conformance [6].
According to [7], the performance of an airport safety system is monitored by means of safety performance indicators.These indicators measure the changes in the level of safety with time, which can support decision making for carrying out actions that reduce risks [8].
According to [9], performance indicators have the following main objectives: (1) monitor the safety level of a system, (2) support decisions about how and where to act to improve operational safety, and to (3) motivate the stakeholders with power of decision to take the necessary actions to maintain and improve operational safety.
Traditionally, accident rates were used as indicators to evaluate safety performance, but greater statistical databases became necessary when the safety increased and the accidents become rare, making the evaluation process difficult [7].
The occurrence of an accident is preceded by a sequence of operational safety events with minor consequences, which trigger effects consequences that disturb components relevant to safety [10].
In this context, the analysis of data on safety events of minor consequences can reveal gaps in operational safety and enable the implementation of corrective actions before triggering an event considered more catastrophic.
According to [11], the advantage of collecting and analyzing safety event data is that they could provide lessons such as: • If the correct conclusions are obtained and implemented, they act as "vaccines" to mobilize defenses of the safety system against more serious occurrences in the future.As good vaccines, they do so without harming anyone or anything in the system.• The safety events of minor consequences offer qualitative insights about how small gaps in safety can get together and align in a way to cause major disasters.• They occur more frequently than accidents, disasters, and other bad occurrences of major proportions, providing the numbers required for more advanced quantitative analyses.• The safety events of minor consequences provide a strong reminder of the dangers that confront the system.
The safety performance should be monitored continuously.In this sense, when the goals established in the safety policy are reached, most severe objectives in the policy should be defined to increase the safety requirements to a new level [6] The indicators of operational safety can be classified as leading or lagging indicators.Lagging indicators use the history of accidents, incidents, or near misses as information for posterior analysis.Leading indicators monitor non-conformances and events of lower severity that may occur in the system aiming to make corrective actions before the system fails [10].
There are many discussions about the best type of indicator to use, but there is no consensus on this subject yet [12][13][14][15].
In this context, this study is based on the guidelines of ICAO, which states that an organization should use both kinds of indicators in a coordinated manner to achieve its safety goals [1].This position was ratified by [14], who argued that the distinction between these two types is not clear, and therefore, not important.
The problem of obtaining a global indicator of ramp safety was first approached by [16], who proposed an empirical methodology based on carrying out a weighted average number of operational occurrences in a given period.The weightings were calculated based on the risk rating for each indicator, negotiating the weights based on the risk rating matrix defined by [1].The lack of theoretical basis of this study revealed the need for a new approach, using models able to provide more representative results of reality and whose foundations for supporting calculations could be defended before a decision maker.
The same problem has been addressed by [17] using multicriteria methods, calculating the weights of the indicators via bicriterial analysis performed on the Borda Method from the risk classification of each and calculating the overall indicator of safety with the PROMETHEE II method.With the use of multicriteria methods in the decision, theoretical support could be provided for the calculation of overall safety indicators.
This issue was also addressed by [18], who used the Swing Weights Method and Multi−attribute Value Theory to obtain an overall indicator of ramp safety, weighing indicators based on expert judgment.As a result, they managed to operationalize the calculation of the overall security indicator at airports where risk ratings do not exist for all security events.
Thus, this work aims to operationalize the calculation of overall ramp safety indicators in airports where there is no information to parameterize the weighting of individual safety indicators.

MULTICRITERIA DECISION AID
The evaluation of airport operational safety requires a structured process with multidimensional evaluations analyzed.In this sense, this evaluation problem is classified as pertinent to the area of the multicriteria decision aiding (MCDA).This area consists of a set of formal approaches that seek to take into account multiple criteria to assist a person or group to explore decisions [19].
Considering this problem, in this study, the entropy and TOPSIS methods are used, which are presented in the following sections.

Entropy method
The entropy method is a well-known method for obtaining weights of criteria in MCDA problems, especially useful when a weighting method based on the preferences and experience of the decision makers cannot be applied [20].
Specifically in the case of airports, there may be cases in which the decision-makers choose not to use their preferences and experience to create the weight criteria.These cases can occur when: The decision-maker is not secure enough in his/ her judgment and believes that the use of a more objective character weighting method will bring better results; • The decision-maker prefers to adopt a weighting method in which he/she can be politically protected in the event that the results of the weighting does not please all stakeholders.• There are several decision-makers who can not come to consensus on the weight of one or more indicators, but agree to use an objective weighting method, independent, free from subjectivities and/or personal bias.• The Airport management prefers to use a method that minimizes the possibility of malicious manipulations of the ramp safety assessment results.
The calculation of global safety indicator is delegated to a decision analyst who, for various reasons, has no access to the decision-maker.
According to [21], the entropy method allows the measurement of the effective information provided by the performance data of a criterion, allowing weighting it proportionally to the amount of information provided.When the alternatives studied showed large differences in the performance between them on the same criterion, the information provided by this criterion is considered most useful and it should have a greater weight.On other hand, the lower the performance differences between alternatives on a single criterion, the lower the amount of information provided by it, and its weight should be smaller.When the alternatives have the same performance on the same criterion, it means that this criterion does not provide useful information for the decision-making, and its exclusion from the decision-making process should be considered.
Assuming that there are m alternatives and n criteria evaluated, the first step to obtain the weights using the entropy method is to assemble the decision matrix P = (p ij ) m × n , where p ij is the performance in the i th alternative and in the j th criterion [19].
According to [18], once the decision matrix is assembled, its data is normalized, obtaining the decision matrix S = (s ij ) m × n , where s ij are the normalized values of the matrix P.This normalization can be performed in two ways, depending on whether the decision makers intend to maximize or minimize the values of a criterion.For criteria whose values we seek to maximize, the normalization is performed using equation (1).
For criteria whose values we seek to minimize, the normalization is performed using equation (2).
The data are normalized to eliminate anomalies related to different scales and units of measurement by turning the scales and units in a way to allow comparisons between different criteria in common measurable units [17].
According to [20], once the normalization of the decision matrix is made, we proceed to calculate the entropy indexes h i of each criterion, which are determined by equation (3).
where f ij is calculated according to equation (4).
According to [24], in the next step, the level of deviation d j is calculated, which shows that the amount of useful information for decision-makers is presented by the j th criterion.A low value of d j means that the alternatives have a high degree of indifference in the performances of the j th criterion and its weight can be reduced.The level of deviation d j is calculated according to equation (5).
In the last step of the entropy method, the weights w j of each criterion are calculated.According to [25], the weights are calculated using equation (6).
Finally, [25] concluded that the weights are obtained directly from the decision matrix, without the influence of the decision makers.This characteristic makes the entropy method to be considered as an objective and unbiased method of evaluation.

Topsis method
The TOPSIS method was developed by [4] for solving ordination in the area of MCDA.The TOPSIS method is based on the idea of choosing the alternative that simultaneously has the smallest euclidean distance from a positive ideal solution (V+) and the longest distance from a negative ideal solution (V−) [26].
According to [27], there are many studies using the TOPSIS method in several problems of performance evaluation and selection.A complete review of these studies can be found in [28].
Assuming a decision matrix P = (p ij ) m × n , the first step of the TOPSIS method is to normalize this matrix, which may be performed using equations ( 1) and ( 2) [29].
Once the decision matrix is normalized, [30] stated that weighing its columns using the weights of each criterion is necessary.Assuming a set of weights w j , for j = 1, 2, …, n, the normalized weighted matrix V = (v ij ) m × n is obtained through equation (7).
According to [31], after obtaining the normalized weighted matrix, we proceed to obtain V+ and V− using equations ( 8) and (9).
According to [32], the next step of the TOPSIS method is to calculate the euclidean distance E i + and E i − from every alternative in relation to the V+ and V− solutions, respectively, using equations (10) and (11).
In the next step, the relative closeness index C i is calculated in relation to the V+ and V− solutions, which according to [24], is performed using equation (12).
Finally, [33] concluded that in relation to any alternative i, the closer it is to V+ and farther from V− simultaneously, the higher the relative closeness index C i , enabling to evaluate the performance of each alternative.

PROPOSED MODEL
The model proposed in this study to evaluate the performance of the airport ramp safety is presented in Figure 1.
In this model, different types of safety events are used to define the criteria that evaluate the performance of the airport ramp safety.The performance of each criterion is obtained by analyzing the number of occurrences of each safety event for every 100,000 airplanes handled in the airport.
Once the criteria are defined, we proceed to the preparation of the decision matrix P = (p ij ) m × n , with p ij being the number of occurrences for the i th year and the j th criterion evaluated.After the elaboration of Figure 1.Proposed model.
the decision matrix, its values are normalized using equations ( 1) and ( 2), obtaining the normalized decision matrix S = (s ij ) m × n .The performance of each criterion is calculated linearly and inversely proportional to the number of occurrences of each safety event.
To facilitate the obtaining of weights in the absence of information, the entropy method is used in the next step of the model, and the weights obtained are used to calculate the weighted normalized decision matrix V = (v ij ) m × n , from which the unicriteria performance is analyzed and the relative closeness index C i is calculated.
The unicriteria performances and relative closeness index C i allow the evaluation of the performance of the airport ramp safety, from which recommendations for action improvement are elaborated.
The airport management uses these recommendations to plan and implement the improvement of actions, according to their capabilities and limitations.The cycle of monitoring the airport safety ends with the airport operation, and finally with data collection and storage to update the database used for monitoring the performance of airport operational safety.

Object of the study
The São Paulo/Guarulhos International Airport was chosen as the object of the study to exemplify the application of the proposed model because it is the biggest airport in Latin America.

Definition of the criteria
The definition of the criteria was based on the study of [16], who analyzed the safety events as follow: • Injuries to people (c 1 ): offenses against the bodily integrity or health of others.
• Incursion of non-flying animals (c 2 ): refer to the entry of non-flying animals, such as capybaras, dogs, alligators, and snakes, in the movement area.• Bird strike (c 3 ): consist of collisions between aircraft and birds, which may cause disastrous accidents and serious damage to aircraft and their engines.• Runway incursions (c 4 ): occur when there is the incorrect presence of an aircraft, vehicle, or person on a protected area of a surface designated for the landing and takeoff of the aircraft.After defining the events that would be used as criteria, a decision matrix was developed using the number of occurrences of each safety event after the movement of 100,000 aircrafts, as shown in Table 1.

Weighting of the criteria
The weights of the criteria were calculated using the entropy method.Considering that the airport Source: [16].
management aims to minimize the number of the occurrence of safety events, the values of the decision matrix were normalized using equation (2).
After the normalization of the decision matrix, the entropy indexes h i of each criterion were calculated using equations ( 3) and ( 4).Once these indexes were obtained, equation ( 5) was used to calculate the level of deviation d j for each criterion.
Finally, the weights w j of each criterion were calculated using equation (6).The values of the entropy indexes, the level of deviation, and the weights of each criterion are shown in Table 2.

Performance evaluation
After determining the weights for each criterion, the TOPSIS method was used to evaluate the performance of the airport ramp safety between the years 1 and 5, using the criterial performances shown in Table 1.
Table 3 shows the weighted normalized decision matrix relative to the performance of the airport ramp safety, along with the relative closeness indexes C i of each studied period.According to the analysis of the relative closeness indexes, the achieved safety was better in year 2, followed by years 5, 3, 1, and 4.This analysis shows that the operational performance has not sustained a trend of continuous improvement, evidenced by the fall of performance during certain periods.
Once the analysis of the overall performance of the airport ramp safety was performed, we proceeded for the unicriterion evaluation of the weighted normalized decision matrix.The performance values of year 5, which is considered as the most representative of the current state of the system because it is the most recent, were analyzed to emphasize the criteria with lower performances.In this sense, the prioritization of the first five criteria was c 3 , c 5 , c 6 , c 9 , and c 10 .This prioritization was performed to highlight the criteria with lower performance, which require more urgent actions for improvement.
Furthermore, to examine the existence of trends of decrease in performance values for each criterion, the data in Table 3 were used to perform a linear regression analysis.For this purpose, we calculated the angular coefficient of the regression line (α) and the coefficient of determination (R 2 ) for each criterion and for the relative closeness indexes C i .We analyzed the criteria that showed negative values of α and high value of R 2 , which determines the degree of correlation of the regression model.Trends of decrease were identified in criteria c 5 and c 6 , which also had critical performances in 2010.Therefore, monitoring these criteria and plan actions to reduce and mitigate the risks arising from them is necessary.
The application of the proposed approach yielded results considered consistent and accurate by the airport administration, which was considered as a validation of the proposed model.The next section

Recommendations
To plan and improve the performance of criteria c 3 , c 5 , c 6 , c 9 , and c 10 prioritized in the previous section, interviews with industry experts were conducted to develop recommendations for the airport administration to advance the preventive and corrective actions.
The criterion c 3 (bird strike) refers to aircraft collisions with birds of various sizes and species.As mitigation actions, the identification and elimination of areas acting as magnets for birds looking for food were recommended for eliminating problems caused by the disorderly occupation around the airport.Among these areas are deposits of organic material, markets, garbage dumps, and slaughterhouses.Moreover, the purchase and use of robotics hawk and rockets were recommended for dispersing birds, capture animals in the area of movement, control vegetation (including elimination of fruit trees), and continuously patrol the outbreaks of animals nearby.
Regarding criterion c 5 (collisions between aircraft), after analyzing the information on airport safety, it was concluded that most of these occurrences was caused by the inattention of the operators, with a percentage being occurred because of equipment failure, failure of procedures, and violation of rules.In this regard, the adoption of the following measures was recommended: realization of annual inspection/review (technical and documentary) of equipment and vehicles that enter the patios and lanes, development of policies of routine inspections by patio tax, enhancement of signaling equipment (e.g., reflective cones) to demarcate the extreme limits of the parking area of each aircraft, training and emphasizing the importance of taking attention in relation to the movement of vehicles and equipment near aircrafts, trainings on the operating rules of the patio, and realization of preventive maintenance of vehicles and equipment.
Concerning the criterion c 6 (collisions with properties), it was observed that the majority of collisions were caused by inattention of drivers, noncompliance with standards, and equipment and procedures failures.Therefore, in addition to the recommendations for criteria c 1 , the following actions were recommended: conducting updating courses of drivers involved in incidents, issuing specific warnings posted in places of great circulation, and controlling the movement of vehicles through tags similar to those used in electronic tolls.
The criterion c 9 (damage caused by foreign objects) refers primarily to incidents/accidents occurred when objects are ingested by engines or collide against aircrafts.To improve the performance of this indicator, the following actions were proposed: increasing the frequency of inspections of runways and taxiways for the removal of foreign objects, promoting awareness campaigns on the consequences of the presence of foreign objects for the entire airport community, and working with the outsourced firms to reassess their procedures to increase operational safety of the airport.
Finally, the criterion c 10 (leaks) refers to the leakage of various kinds.As mitigation actions, the use of blankets to absorb fluids, the implementation of policies to improve the equipment maintenance, deployment of the containment systems for leaks, removal of scraps of aircrafts and nearby equipment nearby were recommended.

CONCLUSIONS
This study proposed and tested the applicability of a multicriteria model to use multiple indicators, weigh them, and measure the performance of the airport ramp safety.This model had the purpose of identifying trends, diagnosing problems, and making recommendations to support the improvement planning of the performance of airport operational safety.
In this context, it was concluded that the airport ramp safety did not maintain a trend of continuous improvement, evidenced by the decline in performance occurring during certain periods.Furthermore, prioritizing the criteria with lower performance was possible, allowing the realization of recommendations for airport management about planning actions for improvement.
The use of the entropy method allowed the analysis of the airport ramp safety performance in contexts of scarce information about the weights of each criterion.This method presented the advantage of eliminating subjective aspects found in other similar methods in eliciting weights, excluding biases related to political decisions, and individual interests.
The proposed approach allowed the expansion of the understanding of the airport ramp safety.In this sense, it allowed the airport managers to identify, organize, and measure the indicators with low performances showing them as critical to airport ramp safety, as well as aided in viewing the historical evolution of the safety performance of the system.
The application of the proposed model yielded results considered consistent and accurate by the airport management, which corroborated the validation of the proposal.Moreover, its application was considered simple and efficient, contributing to the continuity of its practical application as part of the airport's safety management systems.This model provides the airport managers a method to measure the performance of the airport ramp safety and use of indicators that reflect the airport reality, to enable the evaluation of continuous improvement in the safety field.
For future studies, we suggest to perform multicriteria modeling using different methods to verify the applicability and compare the results obtained by each of them.It is also suggested to study software for the integration of different analyses developed by the proposed model.Moreover, given that this study investigated only ramp events, it is proposed to carry out analyses that include and integrate a larger number of components of the airport system to perform a more holistic analysis of the topic.
Finally, we emphasize that the application of the proposed model was considered efficient by airport managers, contributing to the continuity of its practical application as part of the control and improvement of operational performance of the airport ramp safety.

•
Collisions between aircrafts (c 5 ): refer to collisions occurred when the aircraft involved are in the apron and with no intention of flight.• Collisions with properties (c 6 ): occur when damages are caused because of collisions involving vehicles, equipment, and other properties.• Detachment of trailer/dollies (c 7 ): refer to equipment detachments from towable vehicles.• Parking out of safety area (c 8 ): occurs when equipment and vehicles are left outside the safety area and next to the parking position of aircraft, as well as aircraft parking outside their parking positions.• Damage caused by foreign objects (c 9 ): occurs when the action of a foreign body causes damage to an aircraft, usually the powertrain or the mechanisms of flight control.• Leaks (c 10 ): occur when various types of fluid spills, such as hydraulic oils, lubricants, and fuels, that may cause incidents, accidents, and damage to the environment.

Table 2 .
Calculation of the weights by the entropy method.