Archives

FAA

This category contains 145 posts

FAA Issues Statistical Analysis Guidance – Are You Testing Enough Parts?

The FAA has issued a new advisory circular for statistical analysis.  This AC only applies to engine and APU parts.

The new advisory circular uses statistical analysis to arrive at correct sample sizes.  This sample size formula is introduced for persons who are trying to correlate two populations of parts.  MARPA had pointed out to the FAA that typically a PMA applicant does not correlate two different populations of parts – instead they derive the reasonable tolerances on one population of parts, and then design and produce within those tolerances.  The FAA would like PMA applicants to arrive at their appropriate sample size, test the parts being reverse-engineered, and then produce an equivalent number of pre-PMA parts to test for the same properties (and then correlate the two populations).  This is contrary to current FAA regulatory guidance, which requires the design to be approved and the  requires the production quality system to ensure that parts are all produced within the approved design parameters.  In essence, ACO engineers will now take control of the quality assurance system through the design process.

One of the problems with the AC is that it relied on statistical analysis for clinical trial sample size as the basis for assessing statistical analysis of reverse-engineering sample size.  Clinical trials for pharmaceuticals typically rely on populations of hundreds or even thousands of people.  Trying to test hundreds or thousands of parts in order to reverse-engineer them is simply not realistic.  Furthermore, the degree of part-to-part difference under modern quality assurance systems does not support such large sample sizes.
For those cases where the equations in the advisory circular give a lower number for the appropriate sample size, the FAA has also established minimum sample sizes.  The AC sets some minimum limits for the number of parts that must be tested in order to derive certain values (remember that you need that number of PC parts and also that number of reverse-engineered parts to meet the AC’s requirements):

Minimum Number of Parts to be Sampled

  • For basic material properties that are more dependent on alloy constituency than on part manufacture process – 10 approved parts from three separate lots with at least three parts per lot
  • For properties affected by how the material is processed during part manufacture such as high-cycle fatigue, low-cycle fatigue, creep, tensile strength, crack growth, etc. – 30 approved parts
  • For fatigue testing – at least 25 tested parts that are run until they crack
  • For parts exhibiting complex geometry or complex manufacturing variables, additional specimens may be required
  • For parts considered to have a high degree of criticality, greater sample sizes may be required

No statistical basis is offered for these minimum sample sizes.

The new advisory circular provides guidance for statistical analysis of sample-size despite the fact that no regulation actually requires such a broad-based sample.  The advisory circular appears to potentially change the regulations by increasing the burden on applicants.  To the extent that this is true, it is inappropriate.

Our concern is that despite warnings that this is non-mandatory guidance, this guidance may be used as if it were a regulation, with offices refusing to accept PMA applications that are otherwise valid, but that failed to use this AC as a basis for identifying sample size.

If you find that this AC effectively changes the application obligations imposed on you as a PMA applicant, STC applicant, or other FAA-approval applicant, then please contact MARPA so we can raise this concern with the appropriate personnel at the FAA.

AC 33-10 is known by its full title: Statistical Analysis Considerations for Comparative Test and Analysis Based Compliance Findings for Turbine Engine and Auxiliary Power Unit Replacement, Redesign and Repaired Parts.

Inappropriate Restrictions on the Use and Availability of ICAs

Some of you may be wondering where to find the FAA’s policy memo on design approval holder restrictions on ICA availability.  Older MARPA links to the original position of that guidance are no longer valid because it has been moved.  But it is still available if you know where to look!

Many MARPA members look for this policy memo because it clarifies that anti-competitive language in ICAs (restricting use of PMAs or third party repairs) is unacceptable to the FAA:

While not exhaustive, the FAA finds the following practices of using restrictive language in the ICA or through restrictive access or use agreements unacceptable under the provisions of 14 CFR §21.50(b) and related ICA airworthiness requirements:

1) Requiring the owner/operator to only install DAH-produced or authorized replacement parts, articles, appliances, or materials.
2) Requiring that alterations or repairs must be provided or otherwise authorized by the DAH.
3) Requiring the use of only maintenance providers or other persons authorized by the DAH to implement the ICA.
4) Establishing, or attempting to establish, any restriction on the owner/operator to disclose or provide the ICA to persons authorized by the FAA to implement the ICA.

The guidance has found a permanent home in the FAA Regulatory and Guidance Library.  A copy of that guidance is also available on the MARPA website.

New TCDS Guidance Stresses Avoidance of Anti-Competitive Language

The FAA has released Notice 8110.116, which stresses the importance of writing clear notes in the Type Certificate Data Sheet (TCDS), and of ensuring that those notes are consistent with FAA policy.  This new guidance can be very valuable to everyone in the industry.  The value to TC applicants and holders (who create draft TCDS for FAA adoption) is obvious, but less obvious are the subtle and not-so-subtle admonitions against using the TCDS for anti-competitive effect.

One valuable element of this guidance is that it stresses certain things that should not appear in the TCDS.  For example, in paragraph 6(a)(6), the guidance advises against language that limits work to a scope narrower than permitted by the regulations:

(6) Do not include requirement that is non-regulatory in the notes. For example: “This aircraft shall be maintained in accordance with the BHT-427 Maintenance manual.” This note implies that BHT-427 is the only source for maintaining the aircraft, and conflicts with 14 CFR 43.13(a) which states “Each person performing maintenance . . . shall use methods . . . or other methods acceptable to the administrator.”

This admonition is consistent with FAA guidance that has prohibited language limiting operators options in seeking repairs.  E.g. PS-AIR-21.50-01: Type Design Approval Holder Inappropriate Restrictions on the Use and Availability of Instructions for Continued Airworthiness (March 23, 2012).  In the past, there have been problems with manufacturer’s guidance that seeks to limit operators’ repair options, which in turn has had an adverse effect on independent repair stations who are qualified by the FAA to perform the same repairs.  Other language in the guidance highlights the prohibition against anti-competitive statements by stating:

Avoid language promoting a TC holder or their suppliers as the sole source for maintenance or overhaul.

and

It is contrary to 14 CFR parts 43 and 21 to include a note that all repairs or modification schemes must be approved by the TC holder prior to FAA approval.

In a similar vein, the guidance also forbids use of notes that could be interpreted as statements of FAA General Policy.  The obvious reason for this is because the TCDS should not be used as a vehicle for establishing FAA policy.  The example that is given is useful in its own right, because this is clearly not a statement of FAA policy, despite the fact that some government prosecutors have tried to promote similar policies:

(8) Avoid notes that can be interpreted as FAA general policy. For example: “Reuse of parts and assemblies that have been involved in an accident is not permitted unless approved by FAA Engineering.”

In subsection 6(a)(10), the FAA advises against the use of brand names, but in subsection (11), the FAA requires that when a specific brand name is used, then the note must also advise of the possibility of using FAA-approved alternatives:

(11) If a specific brand name material needs to be listed, then you must state that an equivalent material is also may be approved. For example: “SOHIO BIOBOR JF biocide additive is approved for use in fuel at a concentration not exceeding 270 PPM. Use of other, equivalent material, may be approved by the FAA.”

Aviation is a competitive industry, but some companies have attempted to use FAA-approved and FAA-acceptable documents as levers to inhibit competition.  The FAA has repeatedly asserted that its job is safety – not competition – and that it will not allow its approvals to be used as independent inhibitors of competition where no safety interest is served.  While the main purpose of this new guidance is to standardize the layout of TCDSs, it also includes important admonitions that should help prevent anti-competitive behavior.

MARPA and FAA Talk Policy

MARPA had a very good meeting today with Mark Bouyer and Ann Azevedo of the FAA’s Engine and Propeller Directorate (EPD).

The focus of the meeting was status on EPD policy that may affect PMA manufacturers.

Azevedo explained that she has responded to the comments on the Statistics Advisory Circular (AC), and hopes to have the final draft of that guidance available to the public by September.   The Statistics AC is meant to address FAA concerns that have been recognized in practice, such as misusing statistical methods to show equivalence, and underestimating the appropriate sample sizes.

Bouyer expects the Materials AC to go out for public comment this month.  The Materials AC will identify the essential data that is necessary when a PMA applicant is trying to replace the material used in the type design.  MARPA members should watch for this one, and be prepared to offer their comments.

The FAA had published the Geometry AC for comment.  This AC is meant to enhance awareness of how reverse engineering can introduce dimensional differences in replacement parts.  MARPA Board members have expressed that the draft of this proposed guidance appeared to be very helpful to the industy.

The Burner Rig AC, which was also previously out for comment, is expected to be issued by September. It is expected to provide a method for establishing functional equivalence for certain degradation modes in parts such as oxidation, hot corrosion, erosion, etc. The AC is expected to identify existing technology as a means of compliance.

Finally, the FAA is internally reviewing AC 33.8 with a plan to update and clarify the AC. The updates are intended to make the AC easier to use.

The FAA has been very active and diligent in preparing guidance.  In the immediate future (before the Conference), MARPA members should expect to see the release version of the Statistics AC and the Burner Rig AC, and they should expect to see the Materials AC go out for comment.

MARPA Files Comments on FAA Geometry AC

MARPA recently filed comments on the FAA Engine and Propeller Directorate’s draft Advisory Circular 33-Geometry, discussing geometry and dimensional considerations for comparative test and analysis for turbine engine and APU replacement parts.  In a previous blog post we observed that AC identified a number dimensional and geometric factors that the FAA expects to be assessed in ensuring the integrity of dimensional characteristics for the purposes of showing similarity.

We requested feedback from our members describing to what extent the FAA’s expectations were reasonable and practicable, and identifying any issues with the proposed guidance on which MARPA should comment. We received several very helpful responses from our members that helped us shape our comments to the proposed AC. Among the issues members identified were:

    •  Possible confusion over availability of existing industry guidance;
    •  Apparent inconsistency with existing FAA guidance;
    •  Inaccurate implications that product testing is the only method available to collect data for lifing methods of life limited parts; and
    •  Issues regarding the specific practice of part selection for selective assembly components.

Feedback from our members is both helpful and valuable to our comments, as it helps us to identify issues that directly affect members’ businesses, and helps us to better focus our resources on those matters that are important to the PMA community.  The result is more detailed and on-point responses to the FAA to better help shape the guidance material that will ultimately be issued.

We greatly appreciate the feedback we received from our members on this Advisory Circular, and we hope that our members will continue to answers our requests for responses as additional guidance and rulemaking documents are issued.  Together we can work with the FAA to develop the best possible guidance for our industry.

Your Feedback Needed on Engine Endurance Test Draft AC

The FAA is currently seeking comments on its Draft Advisory Circular Engine Overtorque Test, Calibration Test, Endurance Test, and Teardown Inspection for Turbine Engine Certification (§§ 33.84, 33.85, 33.87, 33.93).

As the title suggests, the AC offers guidance on compliance with the engine overtorque, calibration, and endurance tests, and teardown inspection called out in Part 33 of the Federal Aviation Regulations.  Although the guidance is directed at engine manufacturers, foreign regulatory authorities, applicants for engine type design approval, and FAA designees, it also notes that parts manufacturer approvals “may require running certain endurance testing for compliance with § 33.87″ and refers to AC 33.87-2 for guidance on showing compliance by comparative test methods.

MARPA would like to know to what extent members anticipate this AC might effect them, and whether we should submit comments.  If you plan on submitting comments, or have already done so, we would would like your feedback so that we can incorporate member concerns into our comments.

Comments on the Draft AC are due next week, so if you have feedback for us please submit them to ryan@washingtonaviation.com soon!

Revised Guidance for Rotating Parts (Continued Rotation And Rotor Locking)

The FAA is proposing to issue a revision “B” to its Advisory Circular 33.74/92-1B (Turbine Engine Continued Rotation And Rotor Locking).  This AC provides guidance on how to demonstrate compliance with the continued rotation and rotor locking requirements of 14 C.F.R. §§ 33.74 and 33.92.

Marc Bouthillier of the Engine and Propeller Directorate has said that this revision features some updated references, and clarifications of guidance.  He has said that the FAA’s compliance guidance IS NOT intended to be changed in this revision.

We’d appreciate your insights about whether you think there are any substantive changes.  We’d also appreciate hearing from PMA manufacturers that produce rotating parts, whether you find that the requirements of Advisory Circular 33.74/92-1A (the current version) have been applied to PMA applicants.

In addition, the AC has always included in this AC some requirements that appear to go beyond the strict requirements of the regulations.  Many of these are requirements that seem to have been endorsed as sound practices by engine OEMs.  We would be interested in hearing whether the PMA community finds that any of the extra-regulatory requirements of the AC are imposing a burden on PMA manufacturers.

Comments on this proposal are due to the FAA by September 12, 2014 so we would appreciate it if you could provide your comments to us at MARPA no later than the end of August.  You can also file comments directly with the FAA by emailing them to carlos.fernandes@faa.gov

New Guidance for PMA Via Tests and Computations or Identicality: Forthcoming

Today, the FAA announced the issue of AC 21.3o3-4.  The title of this AC will be “Application for Parts Manufacturer Approval Via Tests and Computations or Identicality.”

As of this time, the AC is not yet available to the public.  When it is available to the public, it is expected to be posted online at this location:

http://www.faa.gov/regulations_policies/advisory_circulars/index.cfm/go/document.information/documentID/1024227

Note: the issue date published on the FAA’s website is March 21, 2014; however the AC is not yet available through the FAA’s online advisory circulars, nor through the Regulatory and Guidance Library (RGL).  The note on the FAA’s website states “This document‘s content is not currently available.”

New Draft AC Could Affect Engine PMA Manufacturers

The FAA Engine and Propeller Directorate has issued a new draft Advisory Circular that could have a significant effect on companies seeking PMAs, STCs, or approval of repair or alteration of turbine engine and APU parts.  Draft AC 33-Geometry: Geometry and Dimensional Considerations for Comparative Test and Analysis for Turbine Engine and Auxiliary Power Unit (APU) Replacement, Redesign, and Repaired Parts is directed at applicants who use comparative techniques to reproduce dimensional characteristics of parts during the reverse engineering process.

The AC is intended to provide guidance to help PMA (and other) applicants assess their reverse engineering methods to identify possible causes of dimensional inaccuracies, as well as helping applicants develop adequate criteria for determining dimensional similarity between the PMA design and the sample.  To that end, the draft AC identifies several dimensional and geometric factors that the FAA expects should be assessed in seeking to ensure the integrity of dimensional characteristics for the purposes of showing similarity.  These factors range from geometric relationships to precision and accuracy to influence on critical parts.

The background section of the draft AC states that because reverse engineering methods vary in their measurement techniques, interpretation and combination of data, and in dimensioning systems, they do not typically produce a design with the exact same dimensional properties as the type design part.  It further explains that because dimensional differences may exist, functional assessments–which may include both test and supplemental analytical data–will be necessary to safeguard type design functional properties, as well as capabilities of interfacing and higher level assemblies.  Such explanations help to illuminate the FAA’s expectations of an applicant in making showings of dimensional similarity.  MARPA would like feedback from our members describing to what extent such test data is currently available and practical within the industry.

Although this AC, like all Advisory Circulars, is neither mandatory nor regulatory, such guidance documents do sometimes become ingrained and relied upon in considering applications.  It is therefore important to review the draft and offer comments to the FAA to ensure the guidance reflects the realities of our industry.  MARPA will be closely reviewing the guidance and offering comments on behalf of the PMA community.  If you identify any particular issues within the guidance, please bring them to our attention so we can incorporate them into our comments.  Email your concerns to ryan@washingtonaviation.com.  We also recommend that members who identify concerns file their own comments on the guidance.  Comments are due July 3, 2014 and should be submitted to Mark Bouyer of the Engine and Propeller Directorate, at mark.bouyer@faa.gov

New FAA PMA Policy Documents have been Issued

In recent weeks the FAA has released final versions of two important pieces of guidance.  FAA Order 8110.42DParts Manufacturer Approval Procedures, which cancels revision C, was published on April 10, and FAA AC 21.303-2Application For Parts Manufacturer Approval Via Tests and Computations Or Identicality, was published on April 7.  Both of these documents have a direct effect on PMA producers.

MARPA submitted comments addressing issues in these guidance documents in September, 2013.  Among the issues MARPA noted was the intended use in Order 8110.42D of the FAA’s Risk Based Resource Targeting Tool (RBRT tool) to prioritize PMA projects.  MARPA observed that the RBRT tool as intended relied on subjective assessments of project risk that could result in unfair treatment of certain PMA projects.  The FAA agreed that the RBRT tool was not quite ready for prime time and has removed it from the Order.  MARPA will remain vigilant, however, as guidance for use of the RBRT tool will be addressed in a future Order.

MARPA also worked with its members to offer comments and feedback to the FAA regarding AC 21.303-2.  This new AC consists primarily of guidance to PMA applicants that formerly appeared in the now-cancelled Order 8110.42C.  MARPA offered comments addressing the expected sample sizes needed for PMA applications based on test and computation, origins of samples, and discretion for establishing sample sizes.  The FAA’s adoption of these comments should better help square the advisory guidance with the realities of industry sampling.

MARPA also commented that references to Order 8110.119 – Streamline Process for Parts Manufacturer Approval (PMA) should be included in both documents to encourage use of the Streamlined PMA process when applicable. We are pleased that the FAA agreed and adopted references to the Order in both guidance documents.  We are hopeful that the inclusion of references to Order 8110.119 in both Order 8110.42D and AC 21.303-2 will further promote ACO acceptance and PMA applicant use of the Streamlined PMA process.

Both of these guidance documents are now active.  Members should familiarize themselves with the guidance, as it will inform the nature of the application process and the relationship with the FAA and your ACOs.  In reviewing the two documents, if you notice anything troubling or that seems out of place, please bring it to our attention so that we can work with the FAA to find a solution.  Send your thoughts on Order 8110.42 revision D and AC 21.303-2 to ryan@washingtonaviation.com.

Follow

Get every new post delivered to your Inbox.

Join 75 other followers