The FAA has issued a new advisory circular for statistical analysis. This AC only applies to engine and APU parts.
The new advisory circular uses statistical analysis to arrive at correct sample sizes. This sample size formula is introduced for persons who are trying to correlate two populations of parts. MARPA had pointed out to the FAA that typically a PMA applicant does not correlate two different populations of parts – instead they derive the reasonable tolerances on one population of parts, and then design and produce within those tolerances. The FAA would like PMA applicants to arrive at their appropriate sample size, test the parts being reverse-engineered, and then produce an equivalent number of pre-PMA parts to test for the same properties (and then correlate the two populations). This is contrary to current FAA regulatory guidance, which requires the design to be approved and the requires the production quality system to ensure that parts are all produced within the approved design parameters. In essence, ACO engineers will now take control of the quality assurance system through the design process.
One of the problems with the AC is that it relied on statistical analysis for clinical trial sample size as the basis for assessing statistical analysis of reverse-engineering sample size. Clinical trials for pharmaceuticals typically rely on populations of hundreds or even thousands of people. Trying to test hundreds or thousands of parts in order to reverse-engineer them is simply not realistic. Furthermore, the degree of part-to-part difference under modern quality assurance systems does not support such large sample sizes.
For those cases where the equations in the advisory circular give a lower number for the appropriate sample size, the FAA has also established minimum sample sizes. The AC sets some minimum limits for the number of parts that must be tested in order to derive certain values (remember that you need that number of PC parts and also that number of reverse-engineered parts to meet the AC’s requirements):
Minimum Number of Parts to be Sampled
- For basic material properties that are more dependent on alloy constituency than on part manufacture process – 10 approved parts from three separate lots with at least three parts per lot
- For properties affected by how the material is processed during part manufacture such as high-cycle fatigue, low-cycle fatigue, creep, tensile strength, crack growth, etc. – 30 approved parts
- For fatigue testing – at least 25 tested parts that are run until they crack
- For parts exhibiting complex geometry or complex manufacturing variables, additional specimens may be required
- For parts considered to have a high degree of criticality, greater sample sizes may be required
No statistical basis is offered for these minimum sample sizes.
The new advisory circular provides guidance for statistical analysis of sample-size despite the fact that no regulation actually requires such a broad-based sample. The advisory circular appears to potentially change the regulations by increasing the burden on applicants. To the extent that this is true, it is inappropriate.
Our concern is that despite warnings that this is non-mandatory guidance, this guidance may be used as if it were a regulation, with offices refusing to accept PMA applications that are otherwise valid, but that failed to use this AC as a basis for identifying sample size.
If you find that this AC effectively changes the application obligations imposed on you as a PMA applicant, STC applicant, or other FAA-approval applicant, then please contact MARPA so we can raise this concern with the appropriate personnel at the FAA.
AC 33-10 is known by its full title: Statistical Analysis Considerations for Comparative Test and Analysis Based Compliance Findings for Turbine Engine and Auxiliary Power Unit Replacement, Redesign and Repaired Parts.