Factory acceptance testing (FAT) is a critical part of any physical product realisation. It should be the opportunity to prove product performance to the end client, is often a significant payment milestone in the contract and is the last opportunity to fine tune the product in a controlled environment.
The reality however is often very different and the FAT becomes the start of the breakdown of the relationship between supplier and client, opening the door to debate, disagreement, formal dispute and financial consequences – why?
The discussion below is not intended as a checklist for undertaking a successful FAT although many versions of such checklists exist; rather it is intended to be more of a distillation of the lessons learned, both good and bad, during a variety of such events in a range of industrial and commercial settings.
The good (often discussed, less frequently incorporated)
Experience indicates that a successful FAT is the culmination of a process which starts with the tender submission, continues through the development and manufacturing cycle and ends with the client acceptance of the equipment and the approval to ship it to the job site.
Key to this is the ability to understand and define what a successful FAT is for the equipment in question. Basically:
- Can the FAT replicate the full range of operation of the equipment i.e. is the equipment a stand-alone item or part of larger system (which may or may not be available for the FAT).
- If a full functional test is not possible how are the inferred performance criteria defined and accepted/rejected.
- What are the defined and agreed performance criteria which can be undertaken during the FAT.
- How will they be measured.
- Who will carry out these measurements to ensure the results are accepted by all parties.
And…
- Who carries the cost of the FAT including the mobilisation & demobilisation of the supporting equipment? This should be clarified as part of the contract negotiations.
- Who pays for the cost of overruns in the FAT programme and any re-tests if required.
- Who is responsible for equipment damage as a result of the testing, should this occur.
Failure to tackle these questions in advance will result in the FAT being no more than a test or tests carried out under the auspices of best endeavours, often witnessed by the client and other interested parties.
The bad (well understood but often ignored)
- The acceptance criteria have not been defined in sufficient detail and are ambiguous.
- The recorded results are disputed.
- The FAT requires to be re-run and the delivery milestone and associated payment is missed.
The ugly (seldom considered a possibility, usually results in contractual re-negotiation)
- No performance and thus test criteria have been defined and are generated during the FAT.
- The equipment does not achieve these generated test criteria.
- The equipment fails under test.
- The client has the option to reject the equipment or invoke a delay / discount / damages clause in the contract.
Getting out of jail (or more accurately, not going to jail)
- Understand and agree the performance criteria as early as possible and incorporate these into the contract. These effectively form a substantial portion of the design specification.
- If this is not possible, define at which point in the development programme / project plan it will be possible to define and agree these criteria.
- Define a mechanism for discussion, negotiation and arbitration should it be required.
- Agree the costing mechanism and the impact on the project plan timeline.
- Make this a progress stage gate and payment milestone to ensure it is not bypassed in the rush to develop the product.
- Define how these criteria will be assessed, measured and recorded. Include feasible tolerances for criteria which are not yes / no.
- Define who will carry out these measurements to ensure the results are accepted by all parties.
Where do you fit?
You need to understand where you fit in the supply chain, for example, is it a primary supplier i.e. your customer is the end client, or are you a subcontractor? This has a significant bearing on the level of responsibility carried during the FAT and provides clarity on the information flow and pre-FAT testing required by you or for you. It is critical to understand the responsibilities associated with the incorporation of pre-tested systems and sub-systems to the overall FAT. Examples of this include pipework pressure testing, electrical continuity checks etc. Clarification of these cascaded sub-system tests should ensure, as far as possible, that no potential for damage to the equipment exists during the FAT. This is not always feasible and where this is the case it is critical to define any residual risks in advance of the FAT commencing. Responsibility for subsequent damage due to untested sub-systems should also be clarified at this stage. This should cover replacement of components, time and cost associated with delays and if applicable, financial damages.
Third party approvals
Although often not recognised as FAT’s, it is essential to understand the requirement for third party testing and approval as these are in effect third party FAT’s which can have much more severe consequences associated with failure or sub optimal performance. These often include CE Marking (if not conducted in-house and self-certified), EMC testing, testing and approval under the pressure systems regulations etc.
You need to be clear on the different requirements for stand-alone systems and those incorporated into larger systems e.g. a stand-alone machining centre and a packaging machine, which is part of a high-speed production line.
These must be incorporated into the design, build and test schedules at the appropriate times to ensure that the completed equipment is fit for purpose, can be insured and sold. These are often in addition to any third-party approvals requested by the client.
The Sequel – Site Acceptance Testing (SAT)
So we have successfully completed the FAT and the product has shipped to the client. Time to relax? No, it’s time to prepare for a bigger challenge and one with many more variables and significantly less control.
What’s different? Using a football analogy…
- If the equipment is stand-alone then it is a replay of the FAT at a different ground. If it is part of a larger system then it’s an away match so no home team advantage this time.
- Different crowd, bigger and possibly a little more hostile.
- Know where the edges of the pitch are i.e. the boundary conditions.
- You are in the premier league now; the rewards are bigger, the final milestone payment.
- Failure to qualify at this stage is financially very painful indeed, and relegation is a very real possibility.
- Hope for the best, plan for the worst.
A successful FAT is the best possible preparation for a successful SAT but not a guarantee of a repeat result. If the equipment is stand-alone then the SAT will often be a repeat of the FAT in the final location with a few differences e.g. mains power in place of generators, different ambient temperature etc. If the equipment is to be incorporated into a larger system and this normally means interfacing with other equipment then the SAT is a much more complex undertaking
Interested? Then look out for The Sequel – SAT where we examine the fine detail, potential pitfalls and rewards of Site Acceptance Testing.
