Journal of Information Technology & Software Engineering
Open Access

Fundamental Principles, Processes, and Roles of Environmental Qualification Test Strategy for Complex Engineered Systems

Phillip Jaramillo^*, Marco Rascon, Charles Adams and Eric Jauregui ^*Correspondence: Phillip Jaramillo, Engineering, Raytheon Corporation, Tucson, AZ, USA, Tel: 5202504146, Email:

Author info »

Introduction

Design qualification is defined as a verification process on a design to meet particular requirements [1]. Environmental qualification is the verification of requirements specific to the environment the product design will be subjected to [2]. Examples of typical environments are temperature, shock, vibration, electro-magnetic interference, rain, humidity, and pressure [3].

Strategy is defined as a method or plan chosen to bring about a desired future, and involves setting goals, determining actions to achieve the goals, and mobilizing resources to execute the actions. A strategy describes how the ends (goals) will be achieved by the means (resources).

Environmental qualification test strategy discussed here is the method to verify the product design meets environmental requirements by 1) demonstrating test environment capabilities during risk reduction, 2) defining key stakeholders, roles and responsibilities along with methods to reduce the possibility of increased test durations due to setup inefficiencies, and 3) providing methods to ensure the required test coverage and test methodology is robust enough to detect design deficiencies and to provide a framework to facilitate data analytics [4]. Improved development team execution is one result of implementing an environmental qualification test strategy.

Environmental Qualification Test Strategy

Qualification of a design accomplishes system verification, which ensures compliance to all requirements. This paper focuses on the environmental testing performed during qualification at all levels of assembly - from component, to unit, to section, to system [5]. Three fundamental principles are followed when developing a strategy to successfully execute qualification for a product design:

1. Risk Reduction - Develop confidence prior to entering qualification that the verification of requirements will be successful.

2. Test Planning - Ensure the qualification plan is well understood to prevent test setup related failures/anomalies/ issues.

3. Test Design - Develop a test that is robust to ensure that any weaknesses in the design’s ability to meet requirements will be detected, thus eliminating escapes [6].

Fundamental principle #1 – Risk reduction: develop confidence prior to performing qualification

Perform Design Verification Testing (DVT) at component, unit, section, and system levels with Proof of Design (POD) or Proof of Manufacturing (POM) hardware to verify the design, providing confidence prior to entering qualification (Figure 1) [7].

Plan thermal and vibration testing (special tests) to assist in understanding risk as progress is made towards qualification and will increase confidence in completing a qualification successfully. It is very important to work with the structural technologists to assess the need to add/modify testing to DVT plans to provide sufficient confidence before entering qualification.

One major area of concern on conducting a successful environmental qualification test strategy is the validity of vibration/shock/thermal level requirements flowed to design specifications.

Follow these steps to validate the vibration/shock/thermal requirements to be verified during qualification:

1. Instrument production representative component, unit, section hardware and subject it to a known, low-level vibration input to determine external to internal transfer functions to critical components. Coupled with any available system data, Maximum Predicted Environments (MPE) levels can be generated and requirements updated.

2. During system level testing, instrument the test article to determine non-operational vibration levels induced on components for various user conditions.

3. Instrument hardware during test operations to obtain early thermal data to reconcile with temperature requirements in Prime Item Specifications.

The data obtained justifies updates to requirements documents and environmental qualification plans to be used during their respective tests, and should be considered carefully against the predicted product lifetime [8].

These are three characterization examples of what one could do to better understand vibration, shock and thermal environments prior to beginning qualification. However, these might not be attainable on all for all product designs, therefore consultation with the structures team and structural technologists should occur to develop the best strategy for any given design.

Fundamental principle #2 – Test planning: understand plan for qualification test execution

Qualification efforts are important “for-score” tests that are performed to verify products meet requirements. Thus, utilizing a strategy that follows a methodology to eliminate problems unrelated to the product design, such as test setup anomalies, issues and process type failures. This strategy is applied to supplier qualification as well as component/unit/section/system level qualification, wherever possible (Figure 2).

The strategy employs the following roles as part of its organizational structure (at a minimum, the organization should consider these roles to determine if they are applicable and bring value):

Qualification strategist: The goal of the Qualification Strategist is to put in place an Environmental Master Plan that describes the overall strategy for a product design (or subsystem) and answers the question, “Why is Qualification going to be successful?”.

Qualification lead: The Qualification Lead role shall have the overall responsibility for leading the entire Qualification effort. The Qualification Lead will engage with and ensure participation of both an Environments Expert and Test Expert in each test.

Environments expert: The Environments Expert will typically be the person who created the test environment (vibration, shock, temperature, humidity, etc.) and best knows the genesis, the intended use of the environment, any included margin, extrapolation limitations, and is the expert with the test type. An important role of the Environments Expert is to determine the success of test execution in terms of whether the proper environment was applied to the test article. A good rule of thumb is to have an Environments Expert who is working on the product team and an Environments Expert who is off the product team.

Test expert: Executing a Qualification test can be difficult, detailed, and involved. Many aspects of facilities, hardware, software, fixturing, test units, instrumentation, data acquisition and reduction, etc., must come together at the right time. This is best accomplished by utilizing a Test Expert. The Test Expert and their colleagues are responsible for the total execution of any particular test as requested by the Qualification Lead.

For a successful qualification strategy, “What Could Go Wrong” (WCGW) Reviews are instituted as a means to review the upcoming event and brainstorm what possible problems might occur and ways to mitigate those potential problems ahead of time. This is an opportunity to share lessons learned and what can be done to implement them. The WCGW reviews are not a review of the design and are not test planning sessions to make sure test processes are being followed. They are intended to identify possible problems prior to execution of the testing and develop an action plan to prevent these problems from happening [9].

Further efforts to ensure a successful qualification is to answer the following questions when preparing for qualification events to prevent test setup/process type failures, anomalies, or issues (Table 1):

Table 1: List of Shops in Oja-Oba Markets.

Fundamental principle #3 - Test design: define the qualification plan to eliminate possibility of escapes

While Fundamental Principle #1 is intended to identify design problems prior to entering qualification, a few issues might still exist. The sample size of the units subjected to DVT testing may not be sufficient to identify all the issues in the design. Thus, if any of these issues get through DVT, it is imperative that qualification find these issues, eliminating escapes before the design moves to production (Figure 3).

Thus, the strategy on how the hardware is examined and the framework to gather the data needed to sufficiently analyze the design: 1.) during buildup of the test article, 2.) during operational environments, 3.) after qualification, is critical to eliminating escapes.

• The number of Enhanced Stress Screening (ESS) cycles, the Acceptance test integrity, are all critical in determining the quality of the test article to be used for qualification. Thus a detailed understanding of the entire assembly and test process by the Qualification Lead and Test Expert is required.

• Developing a test process to properly operate the test article during qualification to verify the hardware will function during and after the environment is critical to ensure the elimination of all possible escapes. Consultation with the Design Leads is required to develop a test process to properly test the test article. It should be noted that performance requirements are not required to be verified during environmental qualification, but tests that demonstrate the hardware is functional during and after the environment is what is needed. The software and test equipment teams should also be brought into the conversation to ensure the best possible solution is implemented. Utilizing Built-in Test (BIT) is an option that should always be considered when determining the functional tests to be performed [10].

However, BIT may not be the total solution and supplemental test capability may be required.

• “Are there any special tests or analysis that needs to be performed post qualification to verify the requirement has been met?” should always be asked as part of the development of the overall strategy. An example is when verifying a requirement that may need energetics to be exercised to ensure operability/survivability.

• An inspection plan for during and after qualification should be written and approved by the Qualification Leads, Design Leads and Structural Technologist. The inspection plan should include any teardown required post qualification.

Conclusion

Product development execution is expected to be shorter in duration and more efficient in performance as the customer's need for quick delivery of the product is demanded. A solid Environmental

Qualification Test Strategy is imperative to achieve this goal. This paper describes a process to develop such a strategy based on three fundamentals:

1. Proving in the test environment during risk reduction,

2. Defining key roles and responsibilities along with methods to reduce the possibility of increased test durations due to setup inefficiencies, and

3. Methods to gather analytical data and to ensure testing are robust enough to detect design deficiencies. Adhering to these fundamentals will ensure a successful Environmental Qualification Test Strategy.

References

1. Baker L, Clemente P, Cohen B, Permenter L, Purves B. Model Driven System Design Working Group: foundational concepts for model driven system design. In INCOSE International Symposium. 1996;6(1):1179-1185.
2. Hoffman AR, Ross RG. Environmental qualification testing of terrestrial solar cell modules. In 13th IEEE PVSC. 1978;835-842.
3. Environmental Engineering Considerations and Laboratory Tests; U.S. Department of Defense: Washington, DC, USA. 2008.
4. Maksi L, Berryman S, Brio A, Burkhardt A, Elder S. Deriving Common Factory Test Platform Requirements Using Historical Test Data. J Inform Tech Softw Eng. 2018;8:246.
5. Kececioglu D, Sun FB. Environmental Stress Screening: Its Quantification, Optimization and Management. DEStech Publications, Inc. 2003
6. Kuo W, Chien WTK, Kim T. Reliability, yield, and stress burn-in: a unified approach for microelectronics systems manufacturing & software development. Springer Science & Business Media. 2013.
7. Ciambrone DF. Effective transition from design to production. Auerbach Publications. 2007.
8. Osterwald CR, McMahon TJ.History of accelerated and qualification testing of terrestrial photovoltaic modules: A literature review. Pro Photovol: Res and App. 2009;17(1):11-33.
9. Hollnagel E. The four cornerstones of resilience engineering. In Resilience Eng Perspect. 2016;2:139-156.
10. Drees R, Young N.Role of BIT in support system maintenance and availability.IEEE Aerospace Electron Sys Mag. 2004;19(8):3-7.