Now that you know that UAT is included in Agile, I will go on to explain what other types of acceptance can be involved, who typically does the testing, the benefits and much more…<\/p>\n\n
What is acceptance testing?<\/h2>\n
Acceptance testing is similar to system testing from the perspective that you’ll be testing the entire system. However, in acceptance testing, it can form a decision if the system is viable for deployment.<\/p>\n
It is also used as a quality gate for regulatory or contract related projects (more on this later).<\/p>\n
What are the objectives of acceptance testing?<\/h2>\n
There are a number of different objectives for acceptance testing.<\/p>\n
However, the overarching objectives are to get confidence that the system has been built to the user’s expectation.<\/p>\n
- \n
- Verify the functional and non-functional aspects of the system are fit for purpose.<\/li>\n
- Verifying that the system works as specified (based on requirements).<\/li>\n<\/ul>\n
Are defects expected to be found at this stage?<\/h2>\n
Acceptance testing is typically assumed to be at the end of the SDLC process. Therefore, the assumption by all parties is that the system has been thoroughly tested. And, it is expected that there should be minimal or no new defects found.<\/p>\n
In reality, finding no defects is quite unrealistic. Because you will inevitably find some defects. But, what I would say is, if there is a large number of defects found at this stage it is definitely a red flag.<\/p>\n
In most cases, if there are too many defects, it is regarded as a high project risk. And often recommended not to deploy the code.<\/p>\n
In my experience of testing, had we have found a large number of defects during the acceptance testing stage, there would have been a lot of questions asked. We would also expect to be questioned why simple defects were not picked up during testing. Believe me, these are uncomfortable questions to answer, especially under pressure.<\/p>\n
Different types of Acceptance Testing<\/h2>\n
Now that you understand what the objectives of acceptance testing is. It is important to understand that there are various other types of acceptance testing. Each different type has different objectives and resources to perform it. They are as follows:<\/p>\n
- \n
- User Acceptance Testing (UAT)<\/li>\n
- Operational Acceptance Testing (OAT)<\/li>\n
- Contractual<\/li>\n
- Regulatory<\/li>\n
- Alpha & Beta<\/li>\n<\/ul>\n
User Acceptance Testing<\/h3>\n
![\"user](\"http:\/\/talentedtester.com\/wp-content\/uploads\/2019\/11\/TT0130-user-acceptance-testing-uat-600x338.jpg\")
User Acceptance Testing (UAT)<\/p><\/div>\n
When acceptance testing is mentioned most people will typically assume UAT. However, it is not the only form of acceptance testing. I will agree though, it is one of the most popular, from my experience.<\/p>\n
UAT is an opportunity for the end-user to verify and get confidence that the system performs as expected.<\/p>\n
This usually revolves around the user requirements (Test Basis – more on this later) that were drafted from the initial stages of the project.<\/p>\n
Who performs UAT?<\/h4>\n
This stage of testing is typically performed by business users, typically end-users who will be using the system on a day-to-day basis when it goes live. These resources are usually had picked based on their knowledge of the current system (if its an existing system, that is being upgraded).<\/p>\n
However, if it is a brand new Greenfield system, that has never been used before, then you will typically have the users who are expected to use the system in anger once it goes live into production.<\/p>\n
Operational Acceptance Testing (OAT)<\/h3>\n
Operational acceptance testing is another form of acceptance testing that is often overlooked. Essentially this involves highly technical resources, such as tech support.<\/p>\n
Objective of OAT<\/h4>\n
The focus of this acceptance testing is different from UAT because these testers want to feel confident that they can support the system when it goes live.<\/p>\n
For example, this could include tests like backing up and restoring the system in a simulated disaster (disaster recovery), etc.<\/p>\n
This is quite a technical level of acceptance testing and requires experts that have had, ideally, years of experience or acquired a high level of skills and knowledge in a short space of time.<\/p>\n
Contractual and regulatory acceptance testing.<\/h2>\n
Quite a mouthful but ultimately it comes down to two separate types of testing. but there are aspects of which overlap.<\/p>\n
Contractual Acceptance<\/h4>\n
Contractual acceptance testing involves testers (or representative testers) verifying that the system in question meets the agreed contract specification.<\/p>\n
The agreed contract specification is often used to verify that the system is operating as expected. This contract and agreed behavior is usually created at the start, once a contract is agreed, at the very beginning of the project, way before testing starts.<\/p>\n
Regulatory Acceptance Testing<\/h4>\n
Regulatory acceptance testing is slightly different because it involves testing to verify that a system meets certain regulatory standards.<\/p>\n
My example of regulatory testing<\/h4>\n
During my experience of testing, I worked for an energy company (Gas & Electric). They were launching a Customer Management System (CMS) in a new European country.<\/p>\n
However, the difference with this particular country is, they had a strict regulatory licensing quality standard that needed to be proven and verified before the system could go live and be awarded its license.<\/p>\n
Therefore, I was drafted as a test manager to prove to the regulatory representatives that the system was fit for purpose to go into production in their country.<\/p>\n
Typically for these regulatory acceptance testing, you will have a neutral third-party company that will verify what you have tested to ensure it meets their standards.<\/p>\n
Witness & Test Asset Testing<\/h4>\n
I also had experience of this, which included verifying that the test assets were created as per their specification as well as face-to-face visits to witness test what were testing.<\/p>\n
Depending on who you are working with these tests can be quite stressful. Why? Because they usually have to be done within a restricted time frame. And, there is not usually much chances to get this right.<\/p>\n
And, to make matters worse, the stakes are usually quite high because of the investment that has been made to get the software to pass the regulatory standards before the system can go live.<\/p>\n
Alpha or beta testing<\/h3>\n
Alpha or Beta testing is a strategy used to verify that the system is fit for purpose. However, it is typically used by developers of commercial off-the-shelf software (COTS).<\/p>\n
The objectives of this testing is to get an early understanding of the system and verify it is fit for purpose.<\/p>\n
Big players that use Beta Testing<\/h4>\n
You may have even noticed some of these strategies used by big players such as Google. Sometimes they roll out an early stage “Beta” and get user feedback, before rolling out the final product.<\/p>\n
The idea is to get a good dataset from the intended customers to understand if the system is working before you roll out the fully tested and production-ready version.<\/p>\n
What documents are typically used as a test basis for Acceptance Testing?<\/h2>\n
Like most phases of testing, test cases will typically be drawn up. However, a test basis is required to write the test cases against, are you with me? Typical documents that are used for test basis are as follows:<\/p>\n
- \n
- User requirements.<\/li>\n
- Business requirements.<\/li>\n
- System specifications<\/li>\n
- Use cases<\/li>\n
- etc.<\/li>\n<\/ul>\n
However, we also discussed OAT earlier. This usually requires some of the following:<\/p>\n
- \n
- User manuals.<\/li>\n
- System backup & restore procedures<\/li>\n
- Operational guides<\/li>\n
- Deployment instructions<\/li>\n
- etc<\/li>\n
- <\/li>\n<\/ul>\n
Acceptance Test objects.<\/h2>\n
As well as the test basis documents. There is obviously the actual objects that will be tested. Such as:<\/p>\n
- \n
- The full test application (System under test)<\/li>\n
- Configuration data (especially with (COTS)<\/li>\n
- Business processes<\/li>\n
- Reports<\/li>\n
- Operational processes<\/li>\n
- etc<\/li>\n<\/ul>\n
What defects can you expect to see?<\/h2>\n
At this stage, defects are expected to be very little. However, as I mentioned earlier, you will expect to see some. The types of defects that usually show up during this phase are as follows:<\/p>\n
- \n
- The system not meeting the requirements<\/li>\n
- Non-functional issues, such as security-related<\/li>\n
- Non-functional issues related to system performance.<\/li>\n
- Functionality not meeting a regulatory or contractual agreement.<\/li>\n<\/ul>\n
All of these examples are important and need to be investigated. And, at this stage in the project, there is very little time to rectify these issues.<\/p>\n
Who is usually responsible for these tests?<\/h2>\n
During this testing, depending on which type of acceptance testing, there can be various different resources testing it.<\/p>\n
UAT<\/h3>\n
For UAT, you would expect the end-users to be involved or potentially some of the stakeholders.<\/p>\n
OAT<\/h3>\n
For OAT, this is typically done by system administrators, support workers or a knowledgeable resource that has lots of working knowledge of the system in question.<\/p>\n
Regulatory or Contractual<\/h3>\n
Regulatory and contractual acceptance testing is typically done as a collaborative effort with the actual system testers and a representative (from the regulatory body or third party contract owner\/partner).<\/p>\n
When is acceptance testing typically done?<\/h2>\n
As mentioned earlier most people assume that acceptance testing is done at the end of testing after system testing has completed.<\/p>\n
In my experience, this is usually been the case. However, it does not necessarily have to happen at this stage. Especially when you are dealing with iterative development methodologies such as agile.<\/p>\n