If you are involved in UAT or functional testing, you may have wondered what are the key differences, or if they are one of the same thing. Well, let me explain…
Is UAT functional testing? No, UAT is not functional testing. UAT is a process of checking and validating that the user requirements have been met. This is typically governed by the User Requirements document. Functional testing, on the other hand, is verification that the system meets the functional specification.
To put things into perspective, it is possible to pass functional testing, with the system meeting the functional specification, but still failing UAT. But now that you know that they are different let me explain each one in more detail, the types of testing that fall into these categories and more…
UAT is a validation process which proves the contract (user requirements) between the project team and the user has been met. This is based on User requirements agreed from the beginning of the project.
This is typically conducted by end users (customers). Or in some cases, it is done by a product owner who is working as a proxy for the actual end-user.
UAT testing is important because it confirms that the customer’s requirements have been met. It is also possible to find issues in the system that could have been missed during functional testing.
It is also an important quality gate. Often the final gate before the system goes live.
Functional testing is a way of verifying that each system function meets the functional specification document.
Inputs are put into the system with the actual results (or outputs) of these functions compared against the expected results defined in the functional specification.
I have mentioned the functional specification a few times in this article. Which is a key document in the traditional v-model, used in testing? However, in the real world, from my experience, you will often find that there is no functional specification.
This is the real challenge to a tester. Because, the books ion testing, or ISTQB exam never really prepare you for this.
The main activities of functional testing use black box testing…
Black box testing is a functional based procedure which does not concern itself with the underlying code within each functional module.
Therefore, a tester running black box tests does not need to know the exact code behavior within that function. Meaning they don’t have to be a developer.
They just need to know what input is required and what to expect to get out of it, as per the defined specification. Hence the reason it’s called the “black box”.
Functional testing usually tests functions such as APIs, databases, user interfaces, etc.
In this section, I’m going to explain what different types of functional testing there is.
The following forms of testing would be regarded as Functional testing:
Integration testing is a classic example of this ( Click here to see the difference between System Integration Testing and UAT testing). During integration testing, you will be verifying if each individual system module can integrate and talk to each other successfully.
For this to be successful, inputs from one module will feed into another function. Hence the reason why it is typically called Integration testing.
So, for example, in one of my previous testing projects, we were working on a large financial system. This system was responsible for checking customer debt. Depending on the debt levels, certain actions would be triggered, etc a letter to a customer.
To gather accurate data on the customers, required multiple XML feeds into a central database. It would collect data, for example, address information and credit score data from external vendors.
Each one of these XML feeds was effectively integration tests that needed to be verified using functional testing.
Verification is what most people think of when they talk about system testing.
It is a procedure to verify that the system works as per the specification. In particular, the functional specification. In basic terms, it means – is the product built correctly?
Validation is different if you are thinking of user-related testing. Then, the chances are your thinking of validation testing.
Validation in basic terms means – are we building the right product?
It Focuses on confirming if the system that has been built and verified actually delivers on the user’s expectations.
And this is validated against the User Requirements that were drafted in the very beginning of the project.
Non-functional testing is test procedures that confirm non-functional related specifications such as reliability, performance, etc.
These tests are performed based on separate non-functional requirements. They are not covered during functional testing.
An example of a non-functional test would be: How many simultaneous users can be loaded onto the system?
Typically these tests are done by specialist test resource. They focus on non-functional testing. These testers are usually very well paid because they have rare skills that are often needed by large organizations.
If if you are ever considering a highly paid contracting role for the future, this is something to consider.
Usually, this testing is done by some form of automation such as LoadRunner, QTP, etc.
The reason for this is, it is usually quite challenging replicating these test scenarios without using test automation.
An example of non-functional testing that I was involved in, as a test manager. During a previous project for a large energy company, we needed to simulate some load on the system.
We didn’t have the budget for automation tools, and time scales were tight. So, to simulate this load, we required multiple people all in the same room.
Each person was using the system at one time to verify the system could handle the load. The reality is this method is quite hard to coordinate, and there is room for error.
Also, this is quite a costly affair because you need to have everyone coordinated in the same room at the same time. And time is money. For these non-functional tests, If the project has the budget, automation is easier. because you can click a button to replicate the load.
No, white box testing focuses on the underlying code in a function. This is typically referred to as unit testing and is usually executed by an actual developer.
It is a very important activity which typically happens before the system is passed on for functional testing. Think of it as an important prerequisite before you get into black box testing (or functional testing).
End-to-end testing is a testing method to verify that the entire system application, from start to finish, performs as expected. It verifies the flow of data works as per the specification.
Q: What is compatibility Testing?
This is classed as a non-functional test. Its main job is to verify that the application is compatible with different devices, browsers, etc. For example, let’s say you were working on a web application, such as an online sweet shop.
To perform compatibility tests, you would check the web app on multiple browsers to verify that it is compatible with the major browsers. This is just one example obviously.
These requirements, governing which browsers will need to have been documented and agreed on ahaead of time so that it is not an open-ended test on whatever browser the tester feels like testing. And to also ensure that the agreed coverage is met.
If you have ever worked in system testing, you must have seen when a system does not integrate correctly or just doesn’t work as expected, even after the system testing has passed. This is usually due to the lack of SIT (System integration testing), but how is this different from UAT? Let me explain…
What is the difference between SIT and UAT testing? SIT (System Integration Testing) focusses on proving that the system functions integrate successfully. With each input/output message invoking the next function successfully. Whereas UAT proves that the system meets the User requirements.
Now you know the difference. Let me explain who works on these phases, how they fit into the software development life cycle, the different types that are available and more.
UAT (User Acceptance Testing) testing is a method to verify that the system application meets the user requirements that were laid out before the software development began.
It verifies from a user perspective that they are happy to accept the system.
SIT stands for Software Integration Testing. It is basically a way to verify that the overall system integrates with the subsystem modules as expected.
Essentially, it makes sure that the end-to-end system hangs together as expected after system testing.
SIT typically follows system testing and is based on verifying that the system meets the functional specification. This is the process typically used with the V-model.
UAT usually happens towards the end of a project test phase. Usually, the last step before the system is accepted to go into production.
This is after its been proven in system testing, SIT and sometimes other phases such as OAT (Operational Acceptance Testing).
The reason it is usually at the end of the project is that at this point in time the system should be almost clear of all system serious system defects.
In fact, if you are following the correct procedure it will have had to have passed the exit criteria (click here to see what QA sign off is) of the other phases before it gets to UAT. So, by that time there should only be non-show stopper defects if any.
UAT testing is typically done by a designated product owner. When I say product owner, these are users that typically use the system day in day out and are deemed as product experts or SMEs (Subject matter experts).
When I say experts they are not necessarily technical people. They are usually very skilled in using the application on a day-to-day basis, so they know all the nuances or expected behaviors, are you with me?
As well as verifying the requirements have been met, they are ideal for catching ad-hoc issues that could ordinarily get missed by a less experienced individual.
Their input is valuable because there are a lot of things that they know about the system that maybe even the technical testers may overlook.
SIT is usually done by very technical and skilled testers. These are usually dedicated resource who focus on testing in their job role.
They usually have very good technical skills and understanding of the modules that will be integrated. For example, these skills may include advanced SQL query skills.
There are two types of UAT testing which are as follows:
In all honesty, these distinct types of UAT testing are not always implemented by every company. In my years of experience working on software projects, which includes some large blue-chip companies, it is very rarely split up into these two distinct phases.
But, for your benefit, I will break these two distinct approaches.
Alpha UAT is typically done on the development site by a product owner or user accepted acceptance representative. This user will be able to test the system at the control of the development or project team.
Therefore, the project team is still largely in control of the environment and user access, etc.
Beta testing, on the other hand, is typically done on the clients actual site. This is using their own environment outside of the project teams control.
The benefit is, it mimics the real-life environment more. There still may be a representative from the project team on site to lend assistance, but not always.
Let me share an example of this from my experience of testing. Whilst working for an energy software consultancy. We were tasked with providing some software for them.
After SIT testing, was a phase of UAT on the client site using their environment. Any defects were entered into our defect management system for triage.
SIT can also be broken down into two major approaches:
Let Me Explain…
The top-down approach is focusing on the main parent function. The child modules can or will be replaced by stubs.
What is a stub?….
A stub is a test harness that will return an expected result to a parent function. This is used to test an application with missing (or incomplete) functions.
For example… Cash machine…
Let’s assume we have a cash machine application and we want to test the interface that the customer uses (the parent module). In this example we want to test that the interface links to two child functions:
In this example, we could replace the whole withdrawal cash page with a stub. This could be written to return an expected value when the user selects the option from the main interface page.
This is a way of testing to see how the parent function will perform in the absence of the parent function.
The bottom up approach is essentially the complete opposite of top-down. You start with the lower level modules and use drivers instead of stubs.
A driver is the opposite of a stub. It will actually provide the input that you expect into a bottom level function to prove that the child function works.
So if we revert back to a cash machine application example…
If we want to prove that after viewing your statement you are returned to the parent menu. A driver could be used to provide a value to confirm that it performed as expected.
An information session or UAT kick-off session is a basic session to transfer information to the UAT testers before starting the UAT phase.
In this session, you will give important information such as
So, in a nutshell, it’s a way of establishing the expectation for UAT after following system integration testing.
In my experience of UAT testing, this is always been a very useful session. Because, if you do not have this initial startup information meeting you will find that you’ll get a lot of these questions asked by multiple people during the test process. Often, you will end up duplicating information again and again.
What is the difference between SIT and System Testing? SIT, as discussed earlier, is mainly focused on how the individual system application modules integrate together.
System Testing focuses on the behavior of the system functions. Hence the reason why it is often called functional testing. The goal of System testing is to prove the system meets the functional specification.
Why do you even need system integration testing? It proves the dependencies between system modules. Sometimes it is possible to get carried away focusing on which tests pass at a functional level.
But, not focusing on how they will hang together once all the functions are proven.
If you have ever heard of progression testing, then the chances are, you may have heard of regression testing. But are they different? similar, or something completely different? Let me explain…
What’s the difference between progression and regression testing? Progression testing focusses on new functionality and proving that it works as per the requirements. Whereas regression testing focuses on proving that existing functions of the application are not broken from the addition of new code.
During my years of testing, regression, and progression testing have been critical. Especially when you are dealing with systems that have serious impacts on customers. So, let me explain in further detail the major differences. In addition to this, I will give some examples to make it more understandable.
Is reality its another word for saying functional testing. Some people also refer to this as incremental testing (Click here to see what the incremental model really is), depending on what type of testing model you’re using.
Essentially, it’s testing new functionality in a methodical approach. To make sure that the new functionality matches the desired requirements.
I say sometimes referred to as the “incremental model” because it depends on what model of testing is being used. For example, you could be using the V-model.
However, whatever your model of choice is, progression testing still comes down to testing new functionality against your requirements.
Functional testing is a method of verifying the outcome of the application in comparison to the agreed requirements.
It is often referred to as a form of “black box testing”. This is commonly referred to as black box because we are not focused on the underlying code for this. Testing code modules is known as unit testing (Click here to see the difference between unit and integration testing) or “white box testing”.
So, ultimately with functional testing, you are looking at the functions of the system and making sure that expected outputs are seen, are you with me?
Regression testing is different from progression testing. Ultimately, with regression testing, we are making sure that we have not broken anything when we introduce new code. Simple as that.
Let me give you a working example here…
Let’s assume that you’re testing a calculator application. Currently, it is very basic and can only add and subtract, nothing else.
Now requirements have been drawn up to add a multiplication feature to this app. We can also assume that the application is fully functional and in production at present.
To regression test, the calculator application after the new multiplication feature is tested. You need do some additional tests to:
Regression testing is needed because mistakes or errors in code can be expensive to clean up post-go-live.
Typically you’d need to use regression techniques for the following situations:
All of these potential changes can provide a risk of damaging existing functionality which can have adverse cost effects to the business if they are not tested.
Once you have decided that regression testing is important and worth the time investment. You then need to think to yourself, how are you going to tackle it? Let me explain…
Essentially, there are three main ways to approach regression testing:
Testing everything is one of the most thorough ways of doing regression testing. Meaning you test the entire suite of tests after every new change is implemented.
You’ll often find that most companies will not want to invest in this. Simply because there is a massive cost impact for doing this.
To be perfectly honest, from my experience of testing, it just isn’t feasible to be able to do this every time a change is implemented into the system. So, it’s unlikely that this will ever be used.
A selection of test cases can be chosen instead. This is usually the approach businesses choose. But there a decision made on what to test. In some cases, a standard regression suite can be selected for this purpose. Depending on the business needs.
Prioritizing is a clever way to select which test cases should be tested first. Depending on time restraints its likely that only the high priority regression tests will be executed.
Essentially, you and your project team make a risk assessment on which test cases need to be focused first. This could be based on the risk of the system being impacted or it could be based on how often a particular function is used.
There can be a number of different reasons behind these decisions. But, essentially it comes down to testing only a subset of test cases based on their perceived priority or risk Factor.
In agile testing, you mainly use short sprints of functionality (click here to see what an Agile sprint is) to test. Therefore there are two main ways that you can tackle regression testing in agile:
At this sprint-level, you will effectively test the functionality which is relevant to that particular sprint. It is more of a scaled-down modular based approach.
The end-to-end approach Focuses on the entire application. With each sprint module talking to each other. To regression test this, a selection of test cases are run to ensure that there are no problems (defects).
In modern agile testing, there is a growing appetite to automate the majority of these tests to speed up the delivery to market.
For example, in my experience of testing, I was working for a well-known mobile telecommunication company. They had a Christmas promotion one year targeting millions of homes.
Effectively it was offering their customer base as a special offer for Christmas. This was new functionality added to an existing codebase. Therefore, a large amount of repeatable regression testing was required.
Due to the number of man-hours required to test this manually a decision was made to use selenium test automation suite.
We implemented the selenium test automation suite to go through many laborious iterations of dropdown forms on a web application. This is an example of how test automation could be used to speed up regression testing
As mentioned above Selenium is one good example. However, there are quite a few others such as QTP, winrunner, etc.
The advantages of regression testing is it gives your application higher quality delivery. It’s one of the best ways to secure the quality of your application.
Also, you have the advantage of being able to automate some of these laborious tests to speed up the delivery.
Disadvantages are the additional cost that comes along with regression testing to bolt onto a project. Unfortunately, some companies have a tight budget and cut corners. This can be a disadvantage.
Also if automation is not an option for your company due to lack of budget for the tools, you are left with quite a large burden of manual tests.
Either way, in my opinion, regression is critical. And, although it may seem expensive when you look at the costs up front, long term it can save your company some serious time and money downstream. The costs after going live can be exponentially more than if it was picked up before then.
Preparation is absolutely key to your success with the ISTQB foundation exam. It is definitely achievable if you know how to prepare correctly. let me explain…
How can you prepare for the ISTQB Foundation exam? You need to use a combination of self-study, textbooks, sample exam papers, the actual ISTQB foundation course syllabus, getting familiar with the exam time restrictions and exam format.
Now you know what you should be doing to prepare let me break down exactly how you can implement this starting today. For this, I have 9 ways that can help you prepare:
You may find that paying for a training course is quite expensive. Unless you’ve been given a budget from your workplace. For that reason, it is advisable to do self-study.
Self-study can be done in many different ways but one good way of doing it is using a study group.
This is basically a collection of people that have the same goal of completing the ISTQB exam. You can pool your resources to make sure that you can make studying a lot easier, faster and more fun.
The best tactical way to approach this is to have each person in the group responsible for a specific section or topic. Then ask them to present their findings. After the presentation, they should ask the group questions to test if they have retained the information.
Be mindful that this is just a suggestion. There are many different ways of doing this. But, this is one good way of retaining the information in preparation for the exam.
The textbook that I recommend is this (Click to see the price on Amazon) it is a great textbook to use in collaboration with the official ISTQB syllabus.
I highly recommend that you go through the textbook first in detail and then follow up with the syllabus. The reason for this is the textbook is structured in a great way that will help you understand what you need to do and the concepts within the exam.
The syllabus is crucial because it is always updated and you can download the latest version from the ISTQB website. And, this will also work hand-in-hand with the textbook.
As you are referring to the syllabus you may come across various acronyms or words that are unfamiliar to you. A good way to understand these terms is to go through the glossary to make sure that you truly understand them.
It’s important not to just gloss over unknown acronyms or words because you may get tripped up in the exam if you do not have a good understanding.
This is essential (Click here for some sample exam papers) because it will give you an idea of how the test questions are actually formatted. And, help you get a good feel for the multiple choice questions that you will get in the real exam.
Personally, I love these sample papers. When I was doing my exams for my degree, I used to love getting mock exams because it gave me a taste of what to expect. And, in most cases, the real exam papers were very similar.
When I did my ISTQB foundation exam especially, they were also essential.
However, I would say that you need to understand that these are sample papers. And, they are not actual papers taken from the actual ISTQB exam. Therefore, do not think that you can literally just study every sample paper and guarantee that the question will come up.
Because there is a good chance that you will get different questions. Reason being, the questions are always being updated each year.
During my years of study, one thing I noticed, that separated the high achievers from the people that struggled in exams was preparation.
Not just the actual study of the exam topics but organizing their time so that they didn’t run out of time during the exam, are you with me?
For example, if you have a 2-hour exam and you spend the first 30 minutes on the first couple of questions you may not get to complete all the answers. You may simply run out of time by not pacing yourself correctly.
So for the ISTQB Foundation level exam, you need to know the following: It is 40 questions and you have to get at least 65% of these correct to pass. That’s the most important thing to take in.
Therefore that means you need to get at least 26 of the questions correct in the time period that you are given to pass.
The time period is 60 minutes. By the way, it is 75 minutes in countries where English is not their first language, for example in Canada, where they have French as their first language, in some parts.
Therefore for English speaking countries, you need to make sure that your 60 minutes is used wisely.
I can’t emphasize how important it is to make sure that you revised hard for the exam. Do not underestimate the exam. Maybe you have heard from other people that it is easy?
You will fail if you turn up unprepared!
Your mind is likely to let you down if only you focus on just reading the material over and over. That’s why it’s important to take notes.
For the human brain to consume information it needs visuals, sound, and tactility.
Therefore it is a good idea to actually write down notes of important sections in the book, and then go through these notes. Read them out loud and it will stick in your mind more.
The reason for this is as you read it you take in information initially, but as you write it with your hands you then reinforce the understanding and mental retention. Finally, when you read your notes out loud, it will cement the topic in your mind.
Although this exam does not have negative questions it is designed in such a way where a lot of the answers look very similar.
Also, you may find that there are little words such as “Not” or “Never” that are nested in the sentences that could easily be overlooked if you quickly skim read it.
Therefore, I highly advise that you take the time to read through the questions at least a couple of times before taking your multiple choice answer.
Obviously, you still need to be mindful of the time that you have available. Because you only have 60 minutes (in English speaking countries) so it’s important that you take that into consideration as well.
If you take the option of doing the online exam which is typically done in one of the exam centers make sure that you use the option of the backup paper.
Most people overlook this because they don’t really want to spend the time of writing things down. They want to just put all of the answers directly into the online portal.
However, this is a mistake. The problem with this is if there is any technical issues with the online exam you will lose everything and you do not want to have to do this again.
It has been known that people have experienced technical issues during the exam, like any online portal, and they have not had a backup. And this has been detrimental. So, don’t let this happen to you.
So in this section, I’m going to provide a few areas that are usually in the exam that you should focus on
They are Concepts that will help you in the real world of testing that will come up from time to time so it’s good practice to know these pretty well anyway:
No, I wouldn’t say it’s easy I would say you stand a good chance of passing if you come prepared. If you arrive at the exam with the notion that is easy you can get complacent and overlook the basics.
This will typically end up with you having to retake the exam. So make sure you’re prepared and do not underestimate the task.
If you are focused on studying, most people can get them self prepared in 3 to 4 weeks. However, if you are not focused or you’ve got other important things to do in your life you may find that it may take longer such as 4 to 6 weeks. It may be even more.
If money is not an issue and time is a problem you may even consider going to one of the intensive training courses.
These are two or three-day courses where they work you through the syllabus in the first couple of days followed by taking the exam on the third day. With the goal of you passing your certificate within that week.
To be honest this was the route that I took and was quite beneficial. However, I had the benefit of my workplace paying for the bill and covering the costs. If this is not the case for you then that might not be a luxury you can consider.
If you are new to the industry making a choice about quality assurance could be daunting. Is it really worth pursuing? could you earn well? Let me explain…
Is quality assurance a good career path? Yes, it has many different high paying roles with great prospects. It is also something that you can feel proud about when asked “What do you do?” by a family member.
Now you know its worth it, what kind of roles are there, how do you get in, what is the typical salary, and what qualifications do you need? Let me explain…
Quality assurance is a formalized process to prove that system requirements have been delivered and that the system is fit for purpose.
It is actually twofold, firstly it focuses on confirming to the management and project team that the system has delivered as per the requirements.
And also it there is a quality gate to make sure that customers receive the highest possible quality software.
Quality assurance is important to companies for a number of different reasons. In this section, I’m going to break down a number of reasons why it is necessary:
This may seem like a funny thing to say because quality assurance in itself has costs. Such as paying for testers salaries, costs to draught up requirements by BAs and test planning, etc.
However, after many years of experience of software production, the industry has been proven time and time again that it is cheaper to spend the money upfront to get the best quality rather than trying to clean up the mess after the software has been delivered.
Once the software is live and in production it is very hard to control the costs and clear up the mess.
Apart from the actual costs of cleaning up errors once the system is live, there is also the potential damage to a companies reputation to consider.
Once a customer gets frustrated with software they never really give it a second chance. They typically move onto the competitor as soon as possible.
Therefore to avoid this happening in the first instance quality assurance is important to mitigate this risk.
If your team respects the software that they’re working with they will work harder to maintain quality, simple!
Think of it logically, if you were working in a restaurant and you know deep down the food that you’re serving is low-quality, will you feel embarrassed to serve the food? Of course!
Let’s be honest, you may not even want to continue working there, right?
It is the same concept as software. If your employees and team are passionate about the software, because it actually delivers, and is quality, then it will improve their morale and motivation. This is why quality assurance is important.
Quality assurance has two major routes that you can take in your career:
Personally, in my career, I have taken the managerial angle. And I have done this for a number of years. but let me explain these two different routes right now:
The managerial angle is not just dedicated to being a test manager, in fact, there are a few different roles in this direction such as:
Similar to the managerial roles, there are quite a few technical roles, such as:
According to cwjobs, software testing professionals in quality assurance earn the following salaries:
These are broad averages and be mindful that there are a number of different roles in these areas which could earn significantly more or even significantly less.
For example, an entry-level permanent tester may start off at £25-30k and a test manager could earn £70k+.
The purpose of this is to give you a rough average. Also, be mindful that these costs are based on the United Kingdom. Depending on where you live in the world the earnings will be different depending on your economy.
You may have noticed from the averages above the contract workers earn significantly more than the salaried employees, but why is this (Click here to see my full article why contractors earn so much)?
In summary, this is because a salaried employee has additional benefits in their pay package that is not reflected in their net salary, for example:
So, as you can see there are a lot of hidden bonuses as a permanent employee that are not reflected in this net salary.
However, depending on your circumstances, either of these is great. It all depends on your risk tolerance and your confidence levels.
As a contractor, you run the risk of being out of contract for periods of time. Some people do not have the confidence or appetite for risk to take this on. Especially if they have families to feed.
To become a QA professional, from the beginning, you will usually need a degree and this is preferred in computing or computer science.
However, there are other qualifications that are accepted such as a BTECH, GNVQ and other related qualifications.
Depending on what level you wish to join you will usually be expected to have real-life experience in the industry that you want to work for.
Especially if you want to pursue a contract position. For contractors experience is critical. To be honest, the experience is more important than any vocational qualification in this instance.
In addition to this, a professional qualification such as an ISTQB foundation in software testing (click here to see 11 benefits of the ISTQB certification) is usually required.
You will usually find that recruiters will not even contact or shortlist you for a role unless you have a professional qualification such as an ISTQB.
Other routes into quality assurance can be working your way up, through a career ladder within a company. For example, you could start off by working in a call center in a telecoms company. And then gradually working your way into a support role, and then into a UAT role. Then into an entry level system testing role, etc.
This route may seem long-winded and maybe unbelievable but this is based on my own personal experience of working with people that have taken that exact same route!
Bear in mind that this is not limited to this route. It is just one example of many different ways of doing it.
What is the difference between QA and QC? QA (or quality assurance) and QC (or quality control) are two different things which are closely related.
QA is a proactive methodology to find issues in software products before they hit the market. Whereas QC is a reactive process to ensure the quality of a product that has already been released.
If you are keen to get certified as a software tester, then no doubt you have heard of ISTQB and ISQI. On some resources, their name is used interchangeably. However, they are linked but not the same at all. Let me explain…
What is the difference between ISTQB and ISQI? ISTQB (International Software Testing Qualifications Board) is a non-profit organization that creates testing courses. ISQI (International Software Quality Institue) organize and facilitate the exams and courses provided by the ISTQB.
Now you know the difference between these two acronyms and organizations, let me explain more about the ISQI, how they work with ISTQB, what other courses they offer if they actually provide their own courses and more.
ISQI (International Software Quality Institute) is a company which actually organizes/sells examinations and test courses. However, they are not the company that actually put the courses together.
As I will reveal later on in this article they work very closely with organizations such as the ISTQB to provide an examination and course material for their customers around the world.
The acronym itself stands for the International Software Testing Qualifications Board. Essentially this non-profit organization compiles/creates software testing courses.
They work closely with companies such as ISQI, who sell and facilitate the actual exams and administer the courses that they put together.
The courses that are compiled by the ISTQB rely on hundreds of professional volunteers worldwide. This is to ensure that the standards remain high and that the courses are of high quality.
No, they only offer training courses that have been provided by other organizations. In fact, they actually promote this as a benefit.
They say this improves the validity of their courses and shows that they have no bias towards the actual courses that they are selling.
They are merely partners of the organizations that put together the courses rather than actually writing the courses themselves, are you with me?
The ISQI work with 14 different ISTQB boards around the world. This covers up to 33 different countries.
They also have over 5000 different physical venues where they provide testing examination facilities.
For example, if you would prefer to do an ISTQB Foundation level certification. And, you want to do it online, you can go to one of their test centers.
These test centers will provide a controlled environment to make sure that there is no cheating and also to improve a standard test for every participant.
At these centers, an individual will take the ISTQB course on the ISQI venue. This is how they partner up with organizations like ISTQB.
ISQI have a number of different categories of Interest. Not only software testing. In fact, they refer to these different categories as “Special Interest Groups (SIGs)”.
An example of these categories/SIGS include:
They offer many different courses, across many different categories/SIGS. As discussed earlier in this article, this can be from project management to mobile app testing. But, for your benefit I’ll give you an example of a few of the courses that they have in software testing:
This is more of an advanced level course which is aimed at extending some of these standard practices that are covered in the ISTQB foundation level (Click Here to see the benefits of the ISTQB certification ) such as equivalence partitioning, boundary value analysis, and decision tables.
Ideally, you should have the foundation level before you move onto this course.
This is the bread-and-butter entry-level Foundation level certification (CFTL) which I have discussed on this site in many different articles. This is one of the most popular and recognized certifications in software testing across the world.
This course is also based at an advanced level. To even qualify for this you need to have done the foundation level certification.
However, in addition to this, you need to have at least a minimum of 18 months of real-world testing experience in the industry to be considered for this course.
It covers all areas of software test management, planning, validation, estimation and a number of core competencies that are required to be a test manager.
This is a good one to have on your CV. But, also remember that nothing beats experience. So, while this is great on the CV, most employers will always be looking for your years of experience as a test manager rather than a certification to prove you know what to do.
As I have been in the software testing industry for nearly 20 years now, I speak from experience of actually doing a test management role. And I have seen exactly what requirements that companies are looking for.
Can I take the ISTQB exam online? Yes, you can. But, you can’t just log onto a website and do it on your mobile or in your home. You need to go to an approved test center.
This is to ensure the quality of the exam is upheld. And, to make sure that no cheating is done. Also, keep the circumstances standard for every participant.
The chances are you can find one of these test centers local to you. This is because they have over 5000 test centers around the world.
Does the ISTQB certification expire? No, it doesn’t expire. Once you have done the certification it is yours for life. Funnily enough, I remember being asked this in a recent contract within the last few years.
After looking into it, I was pleasantly surprised to find that it does not have an expiry date. Simply because at that time I had the qualification for nearly 10 years and was worried I missed an expiration period 😉
However, there are some testing certifications such as the CSTP, which do have an expiry date. For example, this one will expire after 3 years.
After these 3 years, unfortunately, you need to re-sit the exam to prove the validity of your certification. This is to ensure that you are up-to-date on all of the latest syllabus updates and changes in the industry.
So, fortunately, this is not required for the ISTQB Foundation level exam.
Is there any negative marking in the ISTQB exam? No, there is no negative marking actually. It is a simple multiple choice marking system. Just to clarify, I do not mean the exam is simple. Its complexity is subjective to each individuals skills, experience, etc.
In total there is forty multiple choice question. You need a score of 65% or more to pass the test. The exam typically takes an hour. But is extended for countries such as Canada that may have English as their second language (They also speak French there).
How do I become a software tester without a degree? You can get in with work experience and a qualification such as an ISTQB CTFL.
In all honesty, most employers will be looking for graduates with an IT related degree. Also, ideally they look for high degree passing grade (in the UK 2:1 or above).
However, this is not always necessary. If you have managed to get experience doing the actual job, that will be much more favored than just a degree.
But how can I have experience without getting the qualifications first? I know it sounds like a catch-22 (no way to win). But I have personally worked with people, that have gone from entry level testers to test management positions with 5 years, without any prior testing experience.
Also worked with people who have had no experience or degree and still doing well in testing. The key is taking a different angle into the position. Maybe you start as a call centre rep, then move to IT support, then into testing, are you with me?
System knowledge in a company is highly valued. Once you establish yourself there for a few years you have experience as a tester and can move onto more lucrative testing positions.
When you are starting out in testing, you may get bogged down wondering which certification you need. But is the ISTQB certification the right one? Well, let me explain…
What are the advantages of the ISTQB certification? You have an Internationally recognized, industry standard that can get your foot in the door for a lucrative job or contract position. It could be the difference between getting considered by a recruiter or rejected.
If you have spent any time sifting through the job sights, you will probably see a pattern. Many recruiters asking for a list of skills and certifications to even stand a chance for you to get your foot in the door. Now that you know at a high level the benefits of the ISTQB certification, let’s look at 11 advantages of the ISTQB now to give you a broader scope.:
One of the beauties of the ISTQB is the fact that it is it’s not restricted to use in your local country but it is internationally recognized.
So, if there is a need for you to go overseas and work, work for an international client or even if you need to emigrate, you’ve got an internationally recognized certification.
As we know, recruiters are not always that knowledgeable about every single term for each job role, therefore they heavily rely on searching and scanning CVs for keywords.
Whether you like it or not, without the keyword on your CV and a certificate to back it up you could be rejected from and miss out on a valuable job or contract role.
So, having the ISTQB could be the big tick in the box that could be the difference between you not having a job and watching re-runs of Friends at home on Netflix, or getting a high flying testing job to be proud of.
If you are trying to snag an important client this certification could make or break the deal. Think of it this way: If you run a consultancy and your potential client, which could be worth millions by the way, is inundated with proposals from rival bidders.
If your rival consultancy had a team of testers with ISTQB certifications and yours doesn’t, all being equal apart from this, it could be the determining factor that could cost you a lucrative contract.
I remember when I passed my ISTQB certification. I was working as a permanent employee and they offered this to me as a perk of the role.
This was quite good because it kept me in the company for longer than I planned to. It also kept me motivated because I was proud that they invested in my career.
As an employer, keeping employees motivated and feeling valued is an important part of retention (keeping them from not leaving). Therefore, this certification could help you keep hold of some of your valuable staff.
As an employer, if you are looking to recruit employees it can be difficult unless you are very skilled in that particular job role. It can be hard to understand your perspective employee really has the right skills. Especially if they are creative with their CV 😉
Therefore you can use the ISTQB to get an understanding of some standard terminology. It can help you understand if that person knows what they’re talking about by having access to standardized terminology that can be checked against a recognized curriculum.
This follows on from my point above. The ISTQB will help enforce Quality control by making sure that your employees have a standardized approach to testing, rather than off-the-cuff.
If you have performance reviews looming and you want to have a measurable and quantifiable way of checking individual employees performance, getting them to pass a certification, such as the ISTQB, is a great way to do this.
At the time of writing, in my current contract testing role, I am working with graduates who have recently joined the testing industry. It’s great because it reminds me of when I first got started decades ago.
Seeing their face as they find out that they have passed their ISTQB certification just reminds me of how satisfying it can be to feel. I can see that feel validated, making them feel worth something and that they have got something to show for their efforts. For that reason, just the sheer satisfaction of having it, is worth considering it, are you with me?
This benefit could be seen in two ways. Negatively, you could argue with no barriers to entry it opens the Floodgates for anyone off the street and boosts competition.
Or, on a positive note, in my opinion, it levels the playing field allowing anyone to progress. Increases the good people moving into the testing industry and improves the standards.
Although there are people that may argue that experience outweighs any certificate. I feel it is still important to prove that you have a benchmarkable skill.
And, the only way of proving this, especially in the absence of experience is to have a standardized certification. It is a great benefit, right?
If you are successful, break through the recruiters initial filtering and get selected for an interview in a test role. Having the ISTQB on your CV is a great way to make a good impression, let me explain.
Just having this certification will prove that you were willing to put in the effort to actually do the exam and study. Especially if your competitor does not have this. In fact, this could be the deciding factor.
As you may or may not know there are a number of different testing certification in this industry. One thing that I like in particular but the ISTQB is the price.
If you are not fortunate enough to have your employer picking up the tab for the course, then this is a great way for you to be able to cover the costs if you’re on a tight budget.
Is there a Selenium ISTQB certification? There is an advanced level automation certification, however, this is not solely focused on selenium’ and is more for advanced level professionals who have had experience before they can actually sit the exam. However, there is a certification offered by SIQI which does this.
Is it better to do a 2-day intensive course or self-study then do the exam? Personally, I don the intensive course. However, I was fortunate to get my course fee paid for by a blue-chip company with deep pockets. The question is if I had to do it today, without the company paying, what would I do?
In all honesty, I would be inclined to get some sample papers (click here for some ISTQB study papers), and self- study. This would help me reduce the cost and do it following my own pace.
Can you take the test again if you fail? Yes, there is no real limits. The only limit is your budget. I would say though, if you study carefully and really put in the work you should stand a good chance of passing. Worse case the second time around. But be mindful everyone has different skill levels so it is impossible to say for everyone.
In my early days of testing I got my ISTQB foundation certificate. It was very important to my career because a lot of recruitment agents tend to filter for keywords such ISTQB and CSTE. But what are the difference between these certifications? In this article we will focus on two in particular.
What is difference between the ISTQB foundation & CSTE? The ISTQB foundation is an entry level testing certification that does not require any prior experience. The CSTE is a higher level testing certification that requires experience before you can apply. The former has become very popular and almost a required prerequisite into testing, whilst the later is more established and still very respected.
Now that you understand at a high level what these certifications are, lets delve a bit deeper and look into the differences at a more granular level.
The Certified Software Tester (CSTE) certification is a qualification provided by the Quality Assurance Institute (QAI). Once you are certified you become a member of a professional collective, which has the potential to increase your chances of commercial recognition and potentially speed up your chances of getting a job.
The International Software Testing Qualifications Board (ISTQB) certification is an internationally recognised standard that is quite sort after when looking for jobs in software testing.
There are a number of different member boards around the world that conduct their examinations online of face to face with an exam provider. These exam providers get a licence from the member board to authorise them to examine their students.
There are different levels of the ISTQB, but for the purposes f this article we will focus on the ISTQB foundation level.
The CSTE has a bigger barrier to entry. Meaning that you need some testing experience before you can qualify for it. If you have some testing experience this can be a real advantage.
On top of this, there are other requisites that need to be satisfied before you can be accepted.
This exam focuses more on the practical side rather than being theoretical. It is also good to have on your CV. This is because recruiters tend to filter applications based on core entry level skills regardless of your other experience and skills.
In total the exam takes four and a half hours. However it is broken up into four sections. The first two has a limit of up to 50 minutes. The last two have a time limit of up to 75 mins. There are breaks in between these sections as you can imagine.
The ISTQB foundation level exam total exam takes up to 75 minutes. In this time you have 40 questions to answer. The minimum passing rate is sixty five percent. The questions are multiple choice, which allows you to get you results faster because it is either right or wrong.
One of the biggest benefits is the fact that anyone, without any prior experience in testing, can take the exam and qualify. Which means there is no barrier to entry.
The certificate is internationally recognised, meaning that you have the flexibility to work in another country if this is the direction that you wish to take with your career.
For a company, in particular a testing consultancy, if they have certified testers they are more marketable and can demand a higher day rate. This can also help to provide a quantifiable target for a companies employees.
Another benefit is no recertification is required. Meaning once you have done the exam and passed, you can don’t need to sit the exam again.
The main disadvantage is the cost. Although some would argue that the cost is negligible for the potential value it can add to your career.
You can pay for the exam directly or do a course that trains you for a couple days first, then you sit the exam. The later is what I done and worked well for me. The course helps you prepare for the exam and increase your chances of success.
The first disadvantage is the fact that you need to re-certify every 3 years, as appose to the ISTQB. Meaning that in the long term it is more costly. Secondly is the fact that you have to wait until you have experience before you can apply.
The later can be seen as a an advantage if you are already established. But for a newbie, this is a disadvantage.
The CSTE will be approx $350. This comes with a PDF version of their curriculum which is called “The Software Testing Body of Knowledge” (STBOK). If you are not a fan of PDF books, you can opt to pay a higher premium of $420. This will give you access to the CD and a hard copy.
Essentially this is a curriculum for the exam that you will tested against. The high level topics of this book are principles of testing, Managing the test project, Risk analysis, Test planning, Reporting, Test case designs and execution.
According to the ASTQB the cost is US $250. However this cost can vary from country to country because it has international exam boards. You are advised to visit you national board or an official ISTQB exam provider in your country.
In my opinion, the best preparation is a course that that includes the exam. This is what I used and is very effective. Essentially you have a couple days preparation on the course curriculum, followed by the actual exam taken on the final day. Bare in mind this is the most expensive way, but effective.
If you are not able to afford the course, the next best thing is sample exam papers. This will prepare you on what is required for the exam without the expense of an in person course.
As discussed earlier, the CSTE comes with the CBOK PDF, or hard copy if you have opted for the upgrade. This contains all the information you need to pass. It just comes down to studying and digesting the content.
If you are looking for something else to assist you, you can get example samples and mock exams to aid you.
What is the CAST certification? The Certified Associate in Software Testing (CAST) is a foundation level certification offered by the QAI. The examination cost is approximately $100. To qualify for this application you need to pass one of the following prerequisites: 3 or 4 year degree but has to be an accredited organisation, two year degree from an accredited organisation + one year IT experience or 3 years IT experience.
What is the CSQA? The Certified Software Quality Analyst (CSQA) is a software testing certification for testers that are at a more experienced level. It is more targeted for experienced testers, project mangers or leaders that want a better grasp of the testing on their projects.
Similar to the CSTE, the cost id $350 or $420 with more materials. The exam is separated into two sections and is multiple choice based. The entire examination process will take up to two and a half hours.
If you are new to testing the different types of testing can be confusing. During my career I have done unit testing as a developer, but primarily system and integration testing as a tester. Today we are going to focus on Unit and Integration testing in particular.
What’s the difference between integration testing & unit testing? Unit testing is typically performed by developers and focusses on one particular function or code module. Integration testing focusses on the communication of one or many code modules or functions to prove that the inputs and outputs work as designed.
A unit test, or white box test as it is also known, is typically done by developers. An entire application consists of many individual cod modules or functions. Typically this is created by a team of developers working together on a tight timeline.
Each developer is responsible for testing that their allocated code module meets the expected design. For example, if the application was a calculator. One developer might work on the addition function and another on multiplication.
The developer working on the addition function would need to prove that this function worked in isolation, before it is integrated into the “calculator” application.
If we extend the calculator example above. An integration test, in this context, would prove that the calculator application modules talk to each other. This testing is typically done by test analysts and can also be referred to as Black box testing.
The reason for this name “Black Box”, is because the tester does not need to have intricate knowledge of the individual code module, they just need to insure the modules talk to each other and the “calculator” application hangs together.
There are two main types of integrations testing: Incremental and non-incremental. Firstly we will focus on Incremental. With incremental the developer will test out integrations earlier in the development process.
This is done by using stubs and drivers to simulate inputs and outputs for incomplete code modules. The most important advantage is that issues can be identified very early. This will in turn reduce costs by delivering a more quality product to the dedicated Test analysts.
Within the Incremental integration approach, there are two variations: “Top-Down” and “Bottom-Up”. Both of these types are named after the flow of the testing.
In the Top-down method, if you can picture a pyramid split up into different levels. This defines the functionality, with the main function at the top, and the minor functionality at the bottom.
With the Top-Down approach, you start with the top level code modules and substitute the lower level modules with stubs to prove they work ahead of time.
What is a stub? A stub is placeholder program that will take an expected input and output an expected result. This allows the developer to code a function and test its inputs and outputs work, before the actual module intended to integrate is ready.
E.g. You have an application called calculator, and within this application you have a function called brackets, addition and subtraction. The brackets function works out the calculation when you have addition and subtraction in brackets.
However, to test the “Brackets” function you need addition and subtraction ready. So, to gut around this problem, you use a “Stub” called “addition” and another one called “subtraction”.
This way you can test “Brackets” function without waiting for the other modules being ready to test.
As you can imagine, this is opposite to the Top-Down method. In this method, the lower level code modules are completed and tested without the parent or higher level modules being available.
For this to be possible drivers are used are used to provide the necessary inputs. One of the biggest disadvantages of this method is the order of the development. Unfortunately because the priority high level code modules are developed last, there could be major issues with these key inputs down the line.
The incremental model is preferred because you can gradually bring in each nodule and deal with issues in manageable chunks. However, if it is unclear exactly how these modules will communicate you can use the “Big-Bang” approach.
Effectively taking a number of modules and integrating them all in one go and seeing what happens. As you can imagine it very rarely works straight away and is likely to be a panful process.
The biggest issue is identifying exactly what module is at fault when the integration fails. The large amount of integrations can make the process very complex.
Integration testing is performed after Unit testing but before UAT. In my experience of testing a System testing phase tends to happen first to prove the main system is functioning correctly, then integrations are tested after.
When Is Unit Testing Performed? Unit testing is performed after each module of code is developed. The idea is to improve the quality of code that is delivered to the system testers.
Stubs and drivers can be used in the absence of other required code modules. For example, if a developer was creating a “brackets” function in a calculator app. He may be waiting for the other “Multiply” and “addition” code modules to be complete, before he can unit test his function.
Instead of waiting he can use stubs and drivers to provide the expected inputs/outputs to prove that his function works, once the other modules are complete.
Can JUnit be used for integration testing? The short answer is yes, it can be used. But it depends on what you want to test with it. Essentially it is not the ideal job for this task, but you can adapt it to work for some integration tasks.
What is a Black Box Test? A black box test is basically a method of testing that does not require the tester to see or understand the inner workings of the code functions. For example a System test phase may have a selection of functional tests.
The tester does not need to know how to code to run the test, in fact they just need to know the expected result from a successful functional test.
What is a White Box Test? As appose to Black box, this test expects you to have knowledge of the inner workings of the code function. Unit testing is an example of White box testing. With this level of testing you are expected to understand the code level inputs and outputs required to complete the test.
The ultimate goal in testing is proving that the system in question is ready and fit for purpose. The official way to do this is via a formal process called a QA Sign off.
What is a QA sign off? It’s the test team’s, or quality assurance (QA) in this instance, method for formally declaring the completion of testing. This declaration is recorded and referred to if required after testing has completed.
To make the decision that testing is complete, obviously you’ll need some kind of criteria to make sure that you’re confident that the sign off has been completed. This criteria is the agreed exit gate definition that governs when testing is complete.
The answer is, yes. The exit criteria is typically listed as one of the main headings in the Test Plan, as well as the entry criteria.
So effectively, the QA sign off is the actual declaration of testing completion. And the criteria of this is typically documented in the Test Plan.
The Test Entry criteria is effectively the list of requirements to govern when testing can start.
With regards to the exit criteria, we spoke about this in greater length earlier
Following on from what we discussed earlier, when I explained the different types of exit criteria. One other example of a QA sign off criteria item is:
are all of the priority tests being executed?
I say, “Priority,” because in some cases where you have really been time-pressed to complete a test phase, you may agree that you can perform the top priority tests and the lower priority ones.
For example it might be a test that focusses on a more cosmetic test, for example checking the labels on an e-commerce order form.
Another example is:
No P1 or P2 criteria defects are in the system at the time of completion.
And maybe another example, one that i’ve used in my experience is,
Minimum of 5 P3 defects with documented and agreed plan to fix after go-live.
This sign off criteria definitely needs to be agreed upfront. It can’t be an afterthought once you started testing. Because the worst case, there’ll be an argument about when you can actually finish testing, because one party may feel that you haven’t done enough.
So , to avoid this, it’s essential that this exit criteria is agreed upfront, and documented. And the document is signed off. Now in reality, getting the Test Plant signed off before testing starts, believe it or not, doesn’t always happen.
And sometimes there’s a battle to do this, because for some unknown reasons there’s people within a project team that are frightened, in a way, to sign off a document, because they don’t want to admit any liability in case anything goes wrong down the line.
But theoretically in testing, you’d want the sign off criteria out of the way and signed off at the beginning. The agreement of the criteria is a collaboration of team members.
The responsibility is a collaboration of all interested parties. And these parties usually will be the Business Analyst (BA), the Test Manager if you were the test lead in this instance. And the Program manager or Project Manager
This is subjective, depending on where you work and how your company works. But effectively the most common way is in the form of an email that is sent out.
And who this email goes to depends on your company. For example, it might go to the project manager, and it may come from the test manager.
Whoever gets it, ultimately it is basically a simple email.
Really and truly this doesn’t have to be War And Peace. This has to be very basic and simple, and get to the point. So a very simple email just explaining that you’re happy that the testing has met the exit criteria is fine.
It’s important that you explained the exit criteria, because you don’t want to this be a subjective statement. You want it to be based on facts. So you want to relay it and relate it to the exit criteria that was agreed.
So for example you might say, “The testing has met the exit criteria as documented in the Test Plan section. Blah, blah, blah.” And then you may relist the items in the exit criteria, and then you will just explain how you’ve met that.
For example if you had 300 test cases to run, and 200 of those test cases were priority one’s. And the agreement was that you had to at least complete the priority test cases, then you would state that 200 test cases of priority one have been done as per the exit criteria.
So it’s just listing out how you have come to the conclusion that the testing has actually complete, and keep it as simple as that.
This happens quite a lot of time. Like for example, you may have a very strict deadline to meet. But then once you meet the deadline, you may find that you don’t actually complete all the tests.
You haven’t actually met all of the test exit criteria, and you have to make a call as to what happens next. And this is typically in a form of a conditional sign off. And a conditional sign off is effectively a way of agreeing that the testing will be accepted for completion, but with some conditions as in certain functionality will have to be tested at a later date.
Or you’d have an agreement for a workaround for a problem that you’ve discovered in testing, and therefore it will be accepted with those conditions.
A checklist is a catalog of items that are typically recorded for tracking. This list is ordered in a sequence. You may or may not be ordered in a sequence. But it’s effectively just the way to work out exactly what needs to be done for testing.
And it’s a checklist much the same as anything that could be used outside of the context of testing. But that effectively is what it is.
As briefly discussed earlier, the conditional sign off is ultimately a way for you to have acceptance of exiting the testing, but with a list of agreed conditions to move forward.
Why do you really need a QA sign off? Why not just stop testing and then just go into production?The sign off is quite important, because it is a formal process which is recorded and can be looked back to if there are any issues in the future.
Also, it’s a way to get buy-in that everyone accepts that testing has completed in case there are any issues. And it’s the way to prove that you’ve met the actual exit criteria.
Basically its a way to cover your back and get something documented. As projects go into production and issues crop up in the future, then there may be a need to have some traceability about how things were tested. And you may be challenged on these in the future. So it’s always good to have an agreed sign off.
Is there a sign off required for UAT testing? In the same way as you have the QA sign off, UAT testing (Click here to see the difference between UAT & SIT) definitely is required to be signed off.
This is quite critical as well, because this is the acceptance from the actual stakeholders or the customers that are gonna use the software in production.
For this reason it’s important to get their buy-in. This proves that they’re actually happy with what they seen. Believe me, this is critical. When things go into production, if things don’t go correct, if you’ve got QA sign off andUAT sign off, it makes your life a whole lot easier.
So as I said before, the UAT testing it typically signed off by a nominated product owner. And this might be someone who is very skilled in that particular function of not necessarily testing, but actually uses the product day-in, day-out.
For example, if you were working on a finance system, and it was an invoice inventory system and you had someone who’s day job was to use that every day. Then they would be the perfect candidate to actually do UAT testing. This is Because they know how the system should work. After they they are informed about the new functionality, they can make a decision if it meets their criteria.