As PMA becomes a global concept, using the existing mechanisms for type certificate validation becomes more important to the process of opening up new markets for PMAs.
Although validation has historically been focused on validating type design for complete products, the same principles would apply when a PMA design needs to be validated in an importing nation. PMA design may need to be validated where the importing nation does not have a bilateral agreement with the regulatory authority that issued the PMA; PMA design may also need to be validated where the special nature of the PMA part requires validation (e.g. the design must be validated and/or approved in Europe as a condition of acceptance if the PMA is for a critical part).
Representatives from North America and Europe discussed the international validation processes at the 2009 Europe / US International Aviation Safety Conference. Moderator Walter Desrosier explained that validation permits the authorities to minimize the type certification process by accepting the findings of another authority through a type validation process. The objective of this validation panel was to discuss how validation is working, and where the world can improve its validation processes.
Following are summaries of the comments by the panel participants (note that the panel’s primary focus was on type design validation, without a specific focus on PMA parts validation, but the principles may be analogous in many cases).
David Turball, TCCA
While there should not be any restriction on the validating authority, the validating authority should nonetheless seek to maximize the reliance on the certifying authority. Transport Canada has historically only validated a design only when there was a Canadian buyer for the type, but is now seeking to provide a parallel validation.
Where there is a new technology or the applicant is new, or there are controversial issues, TCCA would like to be involved in a risk-based, focused review. This is a knowledge-gathering exercise. Historically, there was tip-to-tail review that included non-value added review – so modern work plans are being drafted to focus on more critical or novel systems. You need a set of work plans that describe what will be investigated during the validation process.
If the validating authority does not agree with the applicant, then TCCA is now doing stronger management oversight to identify whether the concerns reflect safety issues (as opposed to merely a different approach being used by the certifying authority). TCCA will therefore disconnect the issue from the project, and look at it objectively. Problems should be taken up with the certifying authority to eliminate the issue for future projects.
The metric should not be efficiency – it should be effectiveness. Validation is all about “what you do with what you’ve learned.” Harmonizing approaches to means of compliance, such as accepting the other authority’s means of compliance even when you do not approach the showing of compliance the same way (test v. analysis, for example).
Retaining knowledge is another important issue. Creating contact reports that can be referenced by engineers in the future in a useful manner to preserve knowledge and experience.
David Hempe, FAA
FAA has 600 engineers, and needs to be “smart” about allocating those resources. Good communication is important – being open and up-front. It should be initiated as early as possible. That is the mechanism for achieving the “trust” that many regulators agree is important.
The regulators need to familiarize themselves with new technologies, and with new ways of making findings of compliance (even if the technology is not intrinsically new). The regulators / validators need a good conflict resolution process for whenthey have different ways of finding compliance.
There are no established international metrics for validation. FAA is ISO 9000 certified, and his office has 20 sets of processes, one of which is type validation. FAA is on their third iteration of trying to develop useful metrics for their procedures (including type validation).
One of the things FAA has done is to try to learn from its past validation processes, in order to carry ideas forward about what is important and what is not important when conducting validation. The FAA understands that they have to be smart about how they pick and choose the items for focus in a validation process. This cannot be arbitrary; there must be procedures in place. The procedures should be based on risk-based analysis, with a focus on the issues that affect safety. New technologies, new means of compliance, and conflicts over whether test or analysis are appropriate for the certification process are all areas where there may be risk that causes special focus in the validation process.
FAA is working on validation ideas with countries who are not represented on the speaking panel, like China and India. In the validation process, it is important to get beyond shared principles, and get to shared certification processes and practices. He is finding that sometimes it is easier to start working on procedures for validation related to parts, in order to gain an understanding of how each authority performs certification and how the authorities can work together. When working on emerging technologies, it is useful to create global teams to set international standards for how to approach the emerging technology. The FAA thinks that there is a lot of value in having more global efforts on how to approach certification.
Norbert Lohl, EASA
Efficiency is not the basis for measuring success. As regulators, the job is not to perform the job efficiently – it is to perform the job correctly. When you trust another system you can use that as the basis for validation. It is a work-sharing exercise, where EASA refers to the county of the manufacturer, but EASA brings their own unique experience to the project. He does not like to see his experts too tightly bound to standardized procedures and process for validation. His experts should identify the safety issues that they personally feel are important to the project.
EASA continues to monitor the airworthiness of the types it has certificated, including those certificated by validation. He is not completely satisfied with the current occurrence reporting standards; but EASA coordinates with TCCA and FAA through frequent meetings and through direct links to the TC holders as well as the other authorities. This is an area that has achieved significant improvement over the past few years. He would like to see us move to a system that monitors worldwide occurrences in a single database to permit better use of the available information for performing risk-based analysis.
There is good communication among TCCA, EASA and the FAA that permits better information to all three about airworthiness conditions / issues.
There needs to be an understanding of the design in order to consider continued airworthiness issues, like ADs. This is part of the reason that the validation process needs to convey an understanding of the type design.
Chris Erickson, Erickson Air Crane
Air Crane is a Part 29 transport category helicopter. It is validated by TCCA and JAA(ENAC). They transferred the ENAC certificate to EASA when EASA came into being. Air Crane was therefore an early user of the validation process.
As EASA has grown and the personnel have stabilized, the validation process has become easier. Erickson finds that manufacturer-produced STC seems to be more readily accepted by EASA in the validation process than third party STCs.
Doug Lane, Boeing
Doug Lane agrees that the measure of validation success should not be efficiency. But performing simultaneous certification / validation provides benefits because it provides additional regulator input into the safety issues surrounding the type design.
Boeing and Airbus probably have type certificates in over a hundred countries. Usually, there is a primary responsible authority and the other authorities need to rely on the primary responsible authority. Validation and compliance is a separate issue from acquiring product knowledge. We want the authority to be familiar with the aircraft, so they can support continued airworthiness issues, but that is not the same as making a finding of compliance.
Harmonization is a key to validation. This does not mean that the regulatory codes must be identical – it is sufficient that there is a harmonization in understanding. The bilateral agreements and their inherent reciprocal system acceptance are also important foundations to validation. It is also important for the regulators to recognize that not everyone can be involved in everything. In a validation process, a risk-based approach allows the validators to focus on the important issues while avoiding wasting time on issue that do not require review in the validation process.
Boeing works with both BASA countries, and non-BASA countries. Assistance from the certifying authorities is useful because the validating authorities will look to the original certification authority for assistance on understanding the issues associated with the type during the validation process.
Lane suggested that when you begin a project, and know where the aircraft is likely to be sold, that you can contact the authorities and find out from them what standards will apply. This can either be a set of standards, or you can seek to find out the differences from the rules of the initial validating authority (the likely areas of focus for validation). This can support effective compliance demonstrations to multiple authorities.
Claude Schmidt, Airbus
Validation allows an importing authority to be convinced that the product meets its own certification standards.
Validation allows an importing authority to be a “second set of eyes” during the validation process. There has been tremendous progress in the process of encouraging validation among the authorities. Harmonization has been a critical part of this progress.
Joint certification processes provide an opportunity for the authorities to gain confidence in each other, and to discuss and resolve differences about means of finding compliance. Streamlining the validation processes based on risk-based analysis is another important way to improve effectiveness of the validation process.
It would be nice is all authorities could come up with a list of all of the special standards that they have – a global list of significant regulatory difference or special requirements.