Author Archives: Talented Tester Support
Author Archives: Talented Tester Support
Integration testing is a pivotal part of testing and one of those areas that is always covered in the ISTQB syllabus. However, who actually does this and what is it? These are the questions I plan to cover in this article…
Who does integration testing? Developers and testers perform integration testing depending on the level of testing. There is a component as well as system integration testing that will be tested by their respective resources.
Now that you know who does what, let me go on to explain exactly what it is, the objectives, what types of defects you may expect to get from the process and more…
As discussed briefly, Integration testing is a term that encapsulates system as well as component-level testing. Therefore, there are different resources that perform each of these levels.
At the component level you would expect to see a developer doing the work. This is because they need to be intimate with the component and have a deep understanding of how the component works.
At the system integration level, this is typically done by testers. They also need to have a deep understanding, but not of the component, of the architecture of the system and how these interfaces should interact, are you with me?
Integration testing is an important part of the Software Development Life-cycle (SDLC). There is a number of objectives in this process and for that reason, I will give you an overview of this now.
At the integration level, we expect a level of confidence in the systems that are to be integrated. Whether that be the components or the systems themselves.
However, integration testing reduces the risks before getting into further levels of testing because there are basic integrations that may not be quite correct.
A quick example of this could be a basic XML file interface which is misinterpreted by two separate parties (but more on this later).
The benefit of finding defects early is reducing the cost further down the line. The cost of fixing defects towards the end of a project is substantially more than at the beginning or earlier stages.
Therefore it is important to capture the defects as early as possible. During integration testing is a perfect opportunity to see glaringly obvious integration issues such as what I mentioned earlier in a previous section about XML interfaces.
As discussed earlier there are different levels of integration tests at the component and the system level. But let me give you more detail on each of these levels…
At the component level, we are focused on the interaction of the individual components. This is a lower level than the system.
This is after a developer completes their component testing (unit testing). And then take it one step further to confirm that the component that they have just unit tested can interface with the next component in line.
System-level integration testing usually happens after system testing. It is essential to confirm that the system has been tested before going into integration.
The objective here is to prove that the system integrates as defined in the specification with external systems, for example, web services.
So far we have discussed the objectives of integration testing and how it works between a component and a system level. But what will be used to confirm how the integration should work? This is where test basis items come in..
Examples of test basis documents are as follows:
During integration there are a number of different objects that can be used at both levels, as you can imagine. But, to give you an idea of these, I will list down a few of these objects to give you an idea.
Once you are in the actual integration test phase it is a good idea to find as many defects as you can, as I explained earlier. However, what kind of defects can you expect to see?
At the component level, typical defects include data issues, interface problems, or functions that may not be handled correctly. E,g, causing indefinite loops or system component errors.
At the system level, you will typically see defects around the actual system interfaces. one of the biggest issues that I personally experienced during my integration testing is XML interfaces that integrate with external systems that are hosted by third parties.
This usually comes down to ambiguous design specifications. These specs could be interpreted in two different ways. Or even data items that were not correctly defined and left open for debate.
It’s amazing the number of man-hours that can be lost just by a simple field not being defined correctly in a design specification.
However, this is the purpose of capturing these things at the integration level and not later on in the process when it cost more money.
Integration at the surface level may sound straightforward but it can get very complicated depending on how large the system is.
Also, it depends on how frequent and how far into the integration testing. Whether that be at the component or system level.
For large systems, for example, financial-based systems can be very complicated. Especially when you’re dealing with multiple third-party vendors that have to interface with different systems.
I remember dealing with a client who specialized in vehicle finance applications. During my time working for this client, I experienced a number of challenges with simple interfaces into their system.
During this process, one of the key things that helped push the project through was regular contact with the client and clearly defined specifications.
Briefly, we talked about integration testing at the system and component level but when is this usually done?
At the component-level, these tests will typically happen immediately after a component has been completed and unit tested.
However, logically speaking you need more than one component completed before the integration can be tested, obviously.
Therefore there may be a reliance on a collaboration of developers working together so that they can make integration happen.
As for the system testing, integration, as you can imagine, happens after system testing. Ideally. And again, there is a dependency on the 3rd party system being available.
Therefore, you can be at the mercy of third parties and playing the waiting game if they are not ready when you are.
You may be wondering how integration testing fits into the big scheme of things, right? Is it a black box testing activity or a white-box? The reality is, it can be white or black box testing depending on the test scenario. The best way to explain this is with an example…
Let’s start with a white box testing example. A white-box integration test could be the integration of how two web services integrate, at a component level.
An example of a black box test in this context could be, testing the integration of a vehicle finance system with an external credit referencing agencies system, in particular, making sure that the XML feed from the external agencies system integrates with your system.
So far you have learned about what integration testing is, its objective, etc. But why is it even needed? This is the question I will help to explain now…
Integration testing is essential because if dependant systems or components cannot talk to each other the system is useless, are you with me?
For example, from a system integration perspective, imagine that you have a credit checking online app, but you can log in and see your profile, but when you click the “get credit score” button you get no response because the interface to Experian is not integrated correctly, would you be happy to pay for that service? are you with me?
This is a simple but powerful example of how critical integration testing is.
In integration, there are a few different ways to actually implement it. Each one has an inherent risk associated. Let me explain what they are…
This is one of the riskiest methods. It essentially means that all integration will be tested together and fingers crossed, it will work. The problem is identifying what has caused the issue in this method. If there are so many integrations at once it is a real challenge.
As the name suggests, the testing will start at the top-level components or systems and work its way down the integration tree.
An example would be testing the login page loads the profile page of a CRM system.
As you can imagine, this is the opposite to top-down testing. The integrations at the bottom will be tested first until you reach the top. So, following on from the previous example, the login page would be tested last. This method is only really effective if the components or systems are ready at the same level.
If you are preparing for your ISTQB, you are probably keen to understand how stubs and drivers are used. For that reason, in this article, I will explain this to you…
Why are Stubs and Drivers Used in Component Testing? This is to enable a developer to unit test a code component without other dependent code modules being available. Stubs help to produce an expected output whereas a driver will send a required input to the code module.
Now that you know what they are at a high level, let me explain in more detail, with examples. Also, exactly what component testing is, the objectives and so much more.
For example, if you have an address book app and the dev wants to test the “login” code component, in the absence of a display contacts code module a stub could be used to test the login component in isolation.
In another example for a driver, if we use the address book app example again if the login page is not ready, but the “display contacts” code module is ready for testing. The developer can use a driver for the Login feature so he can test the display contacts component in isolation.
What is component testing?
This method focuses on a specific code component in isolation. It is also referred to as unit or module testing.
There is some confusion regarding unit and component testing, with other sources claiming/suggesting they are separate, but according to the latest ISTQB syllabus, they are one of the same thing. And, from my experience, they have been treated as one of the same as well.
Now that you understand what component testing is, from a high level. Let me explain how stubs and drivers are used. This will broaden your knowledge and expand on the brief explanation I covered in the earlier section.
As discussed, component testing is typically done in isolation. Meaning, to do this correctly other supporting objects will be required, such as:
Drivers are place holder (dummy) programs that call the function that you need to test. For example, earlier I used the address book analogy, this driver could be the login code module that calls the display contact component you wish to test.
These code modules take inputs from a calling function and display the required output. So, refereeing back to the address book app example, this could be used to replace the display contacts component while the developer tests the login component.
You may be wondering why do we need to use component testing. To help to make this clear I will explain the objectives of it in this section.
The main objective of component testing is to reduce the risk of the component. It is better to find defects early rather than waiting until later test levels. Why? Simple, it saves costs by detecting defects as early as possible.
It is far easier and cheaper to verify that the component meets the related specification before promoting it to integration or system testing.
I have had personal experiences in the past where code has been promoted to system test and there were glaringly obvious issues with the software that wasted many test resources time.
For example, after attempting to test the application, discovering that we are not even able to log in, are you with me?
It may not sound much to you right now. But, look at through the lens of a business. Each tester costs money per hour. If the code is not testable, the time lost could be half a day, or even worse. And, when you multiply that by the number of test resource booked to test, it gets pricey, are you with me?
Using simple techniques such as component testing at the earlier stage helps to reduce these risks.
At this component stage, another objective is verifying that the code module matches the agreed specification. This is from a functional as well as non-functional level.
If it is verified at this stage, it reduces the chance (but not eliminates) of defects going forward.
Automating regression for ever-changing components.
In some cases, when code components are changing frequently, it may be advantageous to automate some component regression tests to verify that the component(s) have not regressed since the last iteration. This is even more relevant when an Agile methodology is used.
Now that you understand what component testing is, its benefits and why we use it. It is also important to understand how it is verified. What do I mean? Well, how can you be sure it’s right? In this section, I will explain some documents that can be used to do just that. Also referred to as the “test basis”.
Here are a few documents that can be used:
In my experience, these are the most common documents used as a test basis. Typically, they clearly outline what is to be expected by the code components. Making the component testing easy to see if it is correct.
One thing to remember, these specifications can also be wrong, they are not perfect. However, that is not the objective of component testing. The concerns can be noted, but ultimately the task is to verify it meets the specification.
According to the latest ISTQB syllabus, the code and data models can also be used. To be honest, in my time I have not seen this done. But, I am not saying it’s impossible, just that I have not experienced this.
Understanding the high-level definitions of component testing is one thing, but grasping exactly, down to the code level, is another. What do I mean? Well, what type of code will be testing in this phase? Let me explain…
During component testing here are some examples of code that can be tested:
These are examples, in reality, there are more. But this is a good flavor of what can be expected.
As you can imagine, during component testing you will encounter many different issues. But, that’s a good thing, right? But, what kind of issues typically rear their ugly head? Let me explain…
During component testing here are some of the most common issues detested:
These errors are a lot easier to pick up these days. Why? Because most up to date code management software will do this automatically. But, there is always room for error.
These errors are more complex when compared to syntax errors. Why? Because they are not always obvious. The challenge with these errors is, it may look ok on the surface (passes syntax checks, etc.), but it won’t give you the expected result. This is where an experienced developer will save the business a lot of time.
Data issues can cause code errors where it is least expected. It could be a weird combination of data, that was not expected, that is actually used in some obscure scenario, that can trip up the said program.
So, now we know the type of issues we can expect. But, which resource is usually responsible for this testing? Let me explain…
Usually, the developer that actually coded the component is responsible for this testing. And from my experience, this is the case. However, I have seen many cases where the developers attempt to avoid this task and effectively promote the code directly to testers. Not ideal.
When new methodologies are being used, such as agile, with automation. There can be additional tasks thrown upon the developer to produce automated, component level, automated tests before even coding the module.
Personally, I think this is good practice. And, it saves the business wasted time and money on defective code.
You may be wondering how component integration fits into things, is it the same as component testing, or a completely different phase. In this section, I plan to explain exactly what this is.
Component integration should follow component testing, in an ideal world. At this level, it is assumed that the individual components are tested.
The objective here is to verify that the component talks to each other, as per the agreed specification. This is done to verify before moving onto the next level of testing.
Earlier we talked about component integration, and when it is expected to be done and established that it is its own unique phase. But, when is the actual component testing done? In this section, I will explain.
Component testing is typically done before component integration testing. In fact, logically, it has too. Why? Because the code components need to be verified first, before moving onto integration. Think of this as a necessary gate.
Ideally, once the code is written, competent testing will commence, to verify the code meets the detailed design or component specification.
If you are preparing for your ISTQB, or genuinely curious about the best white-box techniques prior to starting an up-coming test phase, then I have prepared some info to help you decide the best white-box testing approach.
Which white-box testing is best? The best white box testing technique is a combination of a decision based which is also run dynamically (more on this later). Because you have the potential of high test coverage with the ability to analyze how the code actually behaves.
Now that you understand which technique is best, in my opinion, let me go on to explain exactly what white-box testing is, the advantages, disadvantages, types, how it can be measured and much more.
White-box testing is effectively the opposite of black-box testing. It focuses on the internal structure of a test object, rather than just analyzing the inputs and outputs.
This comes down to analyzing the actual code of the software-based on a detailed design or equivalent specification document.
Now that you have a general idea of what this technique is, you may be wondering what the point of it is. Why should you even care about it, right? Well, let me explain some of the
This testing technique is ideal for improving the efficiency of the code, as well as spotting clear errors before getting into further phases of testing.
In my experiences of testing, developers are sometimes hesitant to do this, with the attitude that they are not testers. But, in my opinion, this is the mark of a great developer who truly checks his work before throwing it over the fence to the testers (so to speak).
Like most things, there are always set-backs with every technique. So, in this section, I will explain some of the disadvantages of this testing technique.
Firstly, whilst it’s a great way to weed out erroneous code early on in the Software Development lifecycle (SDLC). It relies on your organization having skilled developers that can perform the tests.
Usually, it the actual developer who wrote the code in the first instance that actually executes the tests, so you can guess the level of expertise required here.
The point I am getting at here is the cost. To do this correctly you need a decent budget for a skilled resource.
Impossible to catch everything
Whilst this technique helps to improve the code, prior to getting into later stages of testing, such as system testing, etc. It needs to be said, to set your expectation, that it is impossible to catch every potential defect at this level. In fact, ever, throughout any testing stage for that matter.
Don’t get me wrong, it’s worth the investment, but it is worth explaining this upfront.
In this section, I will now explain some of the actual white-box techniques that can be used. According to the ISTQB syllabus, this testing can occur at any test level. However, I will explain some of the most common techniques used.
White-box testing is typically performed by a developer at a code unit test level using one of the following techniques:
The statement based technique focuses on the individual lines of code to be delivered. The idea is to analyze the actual lines of code in the software program and have tests that verify these lines.
Decision-based techniques are slightly different. Instead of focussing on the lines of code it revolves around the conditions in the code. When I say conditions, I mean the decisions, for example, IF or case statements.
As you can imagine there are many conditions within code that will create many permutations.
Whether you use statement or decision-based techniques there is another choice to make. This is how you will actually execute these tests. For this there are two choices:
Static is a manual approach. This relies on the developer going through the code and eyeballing the code to see obvious errors. This can be done by the creator of the code.
But, it is more effective if it is done by a different developer with a similar skill set. These reviews are typically called peer reviews.
The benefit of the peer review is; it introduces fresh eyes. Often a developer will overlook obvious issues because they have simply spent too much time steering at the code. Whereas a new pair of fresh eyes will look at it from a different angle.
This is an opportunity for the reviewer to not only spot obvious errors but also suggest smarter or more efficient ways of doing things.
Dynamic is the opposite. This comes down to running the code and analyzing the outputs. This gives the developer the opportunity to compare the outputs to what’s expected from the specification.
This is a really effective method because the software can be complicated and even the most experienced developer may be surprised with output that seems to look perfect until it is dynamically run.
Understanding the concept of white-box testing is one thing, but grasping how to measure it is another. And, arguably one of the most important factors that will govern the next stage of your test object. For that reason, I will explain how this is done for statement & decision-based techniques.
For statement-based testing, the lines of code is the main focus, as discussed earlier. Therefore, for this method, the total lines of code (statements) are compared to the total tests that execute these lines.
The coverage is typically expressed as a percentage of statements executed to total statements of code.
For decision-based testing, the conditions (decisions) in the code is the main focus, as discussed earlier. Therefore, the coverage is based on this metric.
In particular, the total number of decisions executed by the tests is expressed as a percentage of the total number of decisions.
With this intricate testing, you can expect that it will require a certain level of skills to get it right. For that reason, in this section, I will explain exactly what level of skills you will expect to have.
White-box testing requires a resource that fully understands the code structure of the test object. In reality, every code object is different, so each skilled resource will develop their own unique expertise in one particular area of the system.
Either way, to begin with, you need a technical low-level coding resource that can do this task correctly. In reality, it doesn’t necessarily have to be a developer, but in my experience is nearly always is.
In this section, I will answer some questions related to white-box testing. If you have some additional questions that are not answered here please drop a comment below.
White-box focuses on the internal structure of a test object. Whereas in black-box testing, the internal structure is not important. All that matters are the inputs and outputs meet the specification.
Also, with black-box testing, you would expect testers to take on this task. However, with white-box testing, it is typically developers that do this.
This is typically measured by the total number of integrations executed by your tests as a percentage of total integrations.
Integrations are different. The expectation at this stage is that the individual component (click here to see why stubs and drivers are used in component testing) has already been tested and fit for purpose. Therefore, this stage focuses on how the outputs of component A interface with component B.
These integrations are usually in the form of a file (typically XML) with various fields and data. These data points need to match the specification to pass. Hence the need for the component level integrations. And, even more, an important way to measure its coverage.
If you are preparing for your ISTQB or just curious, you may be wondering what non-functional testing (NFT) is and how it fits into the testing life-cycle. Let me explain…
What is Non-functional Testing? It tests how well a software system behaves, based on an agreed specification. In particular, it focusses on certain aspects of the system, for example, performance, usability or security.
Now that you know what it is, let me explain a bit more detail Such as when is it used, how it can be measured if special skills are required, the types of non-functional testing and more. Keep reading…
Understanding what Non-functional testing is, is one thing. But, understanding when to use it is another. For that reason, in this section, I will explain when it is a good idea to use this. The answer may shock you because it’s not what most people assume or do.
Even though most software testers will assume that non-functional testing should be scheduled after system testing. The reality is, it should be done at all test levels. In fact, the earlier the better.
Why is this? Simply because in the event that you find a serious “show-stopping” non-functional defect. It is better to detect it sooner rather than later. The cost of finding it later is nearly always significantly more.
While planning non-functional tests you need a method to work out the test casts. For that reason, in this section, I am going to explain how you can use some known Black-box testing techniques to draw up these cases.
One of the known Black-box techniques, called boundary value analysis (BVA) can actually be used to derive non-functional test cases. IN BVA, the agreed boundaries of fields and inputs are tested to make sure that they meet the agreed specification.
Without the ability to report and quantify testing would effectively make Non-functional testing worthless. But, you may be wondering, with that being said, how can it actually be measured, right?
The coverage of non-functional tests relies on clear requirements/specifications. For each non-functional element specified, it is possible to measure the coverage based on the number of tests run per element.
This can then be used to help to find any gaps in the testing. Such gaps may be deemed as acceptable based on the risk it poses to the system, assuming a risk-based analysis is used.
Non-functional tests cover a large set of possible tests. But, you may be interested to know if any resource can run or design these tests, or does it need a special level of expertise?
In most cases, non-functional tests, for example, performance tests, typically need a specialized resource with a great level of experience. This experience could be deep knowledge of a tool, e.g. win-runner. Or specialised security-related skills to make sure the system is tested correctly.
In the last section, we touched on what type of skill set is required to do non-functional testing. However, I did not explicitly state who would be the typical resource member to do it.
Non-functional is quite a broad categorization, so there will be different roles for every different particular testing activity. For example, for performance testing, you would typically have a dedicated performance test analyst. If the testing required is quite significant, this could be a team of testers. With one of these resources being a Test lead or senior tester.
If you have security testing in the scope of your project, or you are just curious, you may be wondering what this is classified as. In this section, I will explain this.
Security testing is classed as non-functional. It is required to verify that the system in question is secure. Such as resilience to external infiltration.
To make sure that it is clear, I want to draw a distinction between functional & non-functional testing. This will help you to understand what NFT is.
Functional testing (Click here to see why Functional Testing is Required) is concerned with the functional elements of the system. Typically, this is defined in a functional spec as “features”. At this stage, we are not concerned with code-level testing. It is mainly black-box tests that focus on the inputs and expected outputs of each function. And, verifying that they meet the agreed specification.
Non-functional testing is different because it focuses on aspects of the system that are not usually addressed during functional testing. Such as how many users can use the system simultaneously (max load) before it stops responding, are you with me?
Then, to follow on with this example. Once we establish the max load, how does this compare to the documented non-functional specification? Does it support as many users as we expected, or not?
Reading about NFT is easy, but understanding it is another. Sometimes a real life example will help to put things into perspective. For that reason, in this section, I will give you an example of my first-hand experience using this method.
In one of my contracts in the past, the client was responsible for providing vehicle finance. Therefore, they have a large amount of critical data that needs to be secure and resilient. In particular, in the event of a disaster, e.g. office fire, damage to servers, etc.
I was tasked with leading an annual task they had. Which was to verify that they can recover all of their core systems following a disaster. To do this we emulated the situation by creating a “disaster day”
This testing is known as Disaster recovery. And, is a known example of non-functional testing. The objective was to prove that we can prove that each core system can be used following the disaster.
To prove this, we had systems located on another site. Their sole purpose was to provide a different backup environment that we could switch to following the disaster, are you with me?
On the recovery site we had two goals:
To prove this, we had a selection of Subject Matter Experts (SME’s). Their role was to test the recovered systems and verify that the core functionality was operational, following an agreed test plan. The time was limited (one day on site for all tests), so we had to be efficient.
The second goal, and arguably one of the most important, was to prove that the data was recovered successfully. This was done by taking a snapshot of the data before the disaster day. And then, comparing that snapshot on the recovered system.
To clarify, although this sounds complicated, it was done as simply as possible (due to time constraints. The data snapshots were as simple as taking some screenshots before the disaster and then comparing them after the data was restored on the backup systems, are you with me?
You may be wondering what is so important about these non-funtional requirements. And, you may even consider cutting corners and leaving them as a “nice to have”. But, in this section I will explain why they are required.
One of the main reasons why it is important is the performance boundaries. Once you understand these boundaries, e.g. how many users can use the system simultaneously. It can help your organization understand what they need to do to scale up. It can give you an idea of the cost of this and the impacts it may have on other projects, resources, etc.
Also, it can give an insight to see how long the system is useable in its current form. This can help with planning for future quarters, etc.
If you are preparing for your ISTQB (Click here for my prep guide), you may need to understand the nature of functional testing, and how it works. Let me explain…
Why is functional testing required? It verifies that the agreed behavior of each function works as specified in the related requirements. It is an important part of software testing and is covered throughout all levels during the software development lifecycle.
Now that you understand what functional testing is, at a high level. Let me now explain how it is measured, what skills are required, the different types and much more. Keep reading…
Functional testing is a Test type that focuses on the functions that a system has been designed to do. It can be used at all test levels and uses techniques such as Black-box testing (more on this later) to derive its tests.
The test cases that form this are based on functional requirements. These functional requirements can be across varying different levels and stages. For example, functional specifications, user stories, etc.
You may be wondering if these functional tests can cover such a broad amount of levels, how can the actual coverage be measured? In reality, this is done quite easily, assuming you have a documented test basis to work with, e.g. functional specification. Let me explain…
The coverage is measured based on the functions documented against the number of test cases created. This can be done in the form of a traceability matrix at a high level. These gaps that are defined (if found) can help to tighten up the test coverage.
In this section, I will discuss what skills are usually required when designing or executing functional tests. As you can imagine, depending on your specific requirements, it can be quite a technical process.
As I stated it is a technical process and for that reason, it requires skills or experience with the system or business process. This isn’t a deal-breaker if the individual resource does not have this, but in the absence of this, there will be a steep learning curve until it is mastered.
As you can imagine there are various different types of functional tests, as I mentioned earlier this testing is conducted at multiple levels. For that reason, I will break down some of these types to help you understand.
The following is a list of functional test types:
Component testing is a testing technique that focuses on the individual test component (or code function), for example in a calculator application, the “subtract” component (or code function) is responsible solely for subtraction.
Therefore, in component testing, you would have a test that verifies this component performs as per the specification. At this level, there is no regard for whether or not it integrates with the calculator application correctly.
This is closely related to component testing as you may have assumed. However, the key difference is how these components talk to each other.
At this stage, we assume that the component has been tested and works as expected. We only focus on how the output of component A integrates with the inputs of function B.
e.g. In the calculator App, that we referred to earlier. The substation component would be tested to see how it integrates with other components in the calculator app.
System testing ( is concerned with the system as a whole. It primarily looks at the end-to-end tasks or features that a system is expected to perform.
The main objective of this is to find system defects before they get into a live production environment. As well as verify that the system behaves as specified.
System Integration Testing (SIT – Click here to see What the difference is between SIT and UAT testing) is closely related to system testing, as you can imagine. However, the objective of this testing is to verify how the system integrates with other systems.
For example, in a parking application, there is an interface to the vehicle database that matches number plates scanned with real vehicle information.
The system integration test will verify that a number plate requested from system A will return the vehicle data successfully from System B.
This is, broadly speaking, testing that validates if the whole system is fit for purpose before going into production. It can be done by various different users of differing skill levels, for example, Operational Acceptance Testing (OAT), is typically run by very skilled support staff. Who typically have deep knowledge of the existing system (or product).
If you are wondering what an actual functional test looks like, then this section is for you. I will be explaining more about how these tests are structured and the common attributes of these tests.
Depending on what type of functional test you are executing, the actual look of the test can vary. However, there are some common attributes that you would expect. Even if the tests are very informal. These include the following:
This is the expected setup before executing the test. For example, it could be a previous test is done first, so that the data is in the correct state, etc.
This is a high-level description of the test purpose, objective, etc. It explains what this test is about.
These are the actual, step-by-step instructions to execute the test. Depending on what company you work for they can be really low-level detail (e.g. right-click on the “ok” button to submit the data”) or high-level tests that assume a level of experience (e.g. “submit the data”).
This is what we expect to see once the test has been executed. This is important because it will let us know if the test was successful or not. You may fee that this should be obvious, but, understand this, some tests could be negative, meaning the expected result could be an error.
This is the actual result of the test. Typically, this will be “Pass or Fail”. But, there can be other statuses such as “Blocked” that can be used.
If you have some interest in automation or are just thinking of ways to speed up test execution, then you may be wondering if functional tests can be automated. So, in this section, I will be explaining if this is possible and how this is typically handled at this stage.
Yes, automation can and is used during functional testing. The reality is, functional testing is quite a broad term that encapsulates many testing techniques, as you have seen already.
Therefore, there is quite a lot of scope to automate at different levels. If we take system testing, for example, this is an example of a stage where this can be used. I have used this personally at this stage. It is beneficial if you have a large set of tests that are quite similar/repetitive. This will help to speed up the process.
As we have discussed functional testing covers quite a few stages. So, you may be wondering if performance and functional are one of the same things. I will discuss this further in this section.
The short answer is no, performance and functional testing are different. Why? Because performance testing is classed as a non-functional test (Click here to see what is Non-Functional Testing).
These tests usually incorporate tests which are based on testing the performance of the system against the specification/requirements. An example of this is the number of concurrent users that a system can have at one time.
Earlier we talked about automation and it is a part of functional testing. But what about manual testing? Is that also included? This is what I will clarify in this section.
Yes, manual testing is used in functional testing. In fact, I would go as far as to say that it is used extensively. Obviously, these days there is more and more automation being used. However, manual testing is still used a lot across all test phases.
Manual testing is a method of test execution. It means that a tester will manually run test cases by supplying the necessary inputs, pre-requisites, etc. and then analyzing the results to confirm if the test passed or failed.
Once a system is promoted to live it is not the end of its updates or changes. In fact, there can be many iterations after it goes live, such as defects, new functionality, etc. But what is Change-related testing and what has that got to do with this?
What is change related testing? Change-related testing is a Test Type that has a specific purpose. It is designed to verify that a change to the system has been fixed correctly and does not have any adverse effects on the system as a consequence.
Now that you know what change related testing is let me explain what types of tests are performed during this time, how it relates to the overarching “Test Types” if some of these tests can be automated and more. Keep reading…
A test type is a group of testing activities that have specific objectives to test the system. Here is an example of other Test Types that are used:
You may be thinking this is all good, but when would you need to actually use change-related testing? In this section, I explain when and why you will use it.
This testing type is typically used following a change to a system. Usually, this is after a system is live, but this is not always the case.
These changes can be as follows:
If the system has just been upgraded due to defect fixes it is important to verify that the defect fixes have been fixed correctly and also check that their inclusion has not caused any unexpected issues to the current system (aka Regressions – more on this later).
If the system has been implemented and for whatever reason, it needs to be moved. Then this can be a risk. This risk needs to be mitigated with change-related tests to ensure that it is safe to use.
If the system has had modifications to improve or even change the behavior of the system, it needs to be tested to verify it. This will include tests to confirm that the changed functionality works as per the upgraded specification and also that it does not introduce any issues.
Similar to the section above, new functionality added will be treated much the same as changed functionality. The new functionality will be tested to verify that it works as specified, but it will also need to be checked for any regressions.
As discussed earlier change-related testing is an over-arching group of test activities. So, you may be wondering what specific techniques are used to test this. Let me explain…
This change-related test technique focuses on verifying that any new builds or code changes do not adversely affect the system. For example, a defect fix may be released to fix a calculation error in a timesheet system. The fix was designed to fix a miscalculation on weekend work rates.
After implementation the defect worked, however, the users have now discovered that their standard day rates are now being incorrectly calculated. This is an example of a system regression (Click here for Regression Testing Best Practices). And this is exactly what regression testing will deal with.
This technique confirms that the intended change meets its specification. So if we refer back to the example in the last section, the timesheet system defect fix. In this context, confirmation testing will be used to verify that the release fixed the issue with weekend rate calculations, which it did.
With all these potential tests that could be required on an ongoing basis, e.g. regression testing that we discussed earlier. You may be wondering if it would make sense to automate these tests. In this section, I will explore this for you.
When we look at the two main techniques in change-related testing the obvious one to automate is regression testing. Why? Well, confirmation testing is effectively a moving target, right?
Think about it, each time the system is changed for a defect or functionality change it could literally be anything. However, with regression you have some common tests that a run over and over again to check that the system responds in the same manner as it did pre-release, are you with me?
e.g. If we go back to our timesheet system we used earlier…. Every time an amendment is made to this system, it would be a good idea to check that the standard day-to-day calculations still work as designed.
This would be a perfect example of a group of tests that could be easily automated.
If you have heard of smoke testing before, you may be curious to see if it is used during Change-related testing as well. For that reason, I will clarify this in this section.
Smoke testing evolved from industrial engineering. When physical changes were done it was deemed as fit for purpose if no smoke was seen.
For that reason, it has evolved into a technique used in software testing. The purpose of smoke testing is to establish if the system passes some very basic tests, to confirm that the system is even remotely ready to test. This can be basic system functions that are mandatory to move forward.
Yes, smoke testing can and is used during change-related testing. Although it is not necessarily documented in exam syllabuses, in the real world it is used. I know from my own personal experience of using it in anger.
It helps to reduce wasted time and does not take much time to run the tests. In reality, these tests are so basic, one could argue that they are bordering on common sense tests that do not need a formal name or phase.
Earlier we mentioned Smoke testing and the inclusion in change-related testing. However, what about sanity testing? Could this be included also? Let me explain…
Firstly what is Sanity testing? Sanity testing is essentially a subset of regression testing. It aims to verify, with some basic tests, that the function in question is working. It does not confirm it, but it gives us an early view if something really basic is not working, which could render the regression testing a waste of time.
No, it’s similar, but not the same. Sanity testing focuses on the function under test. Whereas Smoke testing has a broad set of tests, which can be run in a short space of time.
Yes, it can and, like smoke testing, it usually is. Whether that is formally or informally. It is a great way to eliminate any time-wasting for a code fix that clearly has not addressed the problem.
In this section, I am going to answer some questions related to change-related testing. If you feel you have more questions you need answering, feel free to drop me a comment below.
Re-testing is effectively testing test cases that have previously failed. Whereas regression testing is verifying that the system has not “regressed” by re-running a subset of previously run tests.
Although they sound similar in action they are very different. Both are just as important as the other by the way, just have different objectives. Obviously, as discussed regression is one of the key activities in change-related testing.
If you are interested in an ISTQB course but have no idea what’s best for you, you are in the right place. I am going to recommend my best certifications in the ISTQB portfolio, let me explain…
What is the best Istqb course? For beginners the Certified Testing Foundation Level is ideal. For Test managers, The Advanced Level Test Managers certification is ideal. And finally, for Agile, The foundation level Agile certification is best.
Now that you know what courses I recommend, let me go on to explain how the ISTQB portfolio of courses is structured, the different streams they have, and what that even means in simple terms and more.
I remember going for a walk one lunchtime, with a tasty subway sandwich in hand. As I sat on the park bench reflecting on life and planning for my weekend, it made me appreciate life and take in a breath of fresh summertime air. However, it then dawned on me, that not everything is a walk in the park.
Same as choosing the best ISTQB course, nothing is always that straight forward, especially when you have multiple options. For that reason, let me explain now, how the courses are structured to help you.
ISTQB courses are structured based on different levels. In total there are three different levels:
Ultimately all of these levels higher levels are only possible if you have first done the foundation first.
In addition to these levels, they have another dimension which is called “Streams”.
These streams are grouped as follows:
As discussed in the previous section there are three different streams. Let me explain…
The core stream is based on the original ISTQB core fundamentals. These are applicable across all of the ISTQB certifications. Hence the reason why it is important to do the core foundation course before moving on to any other levels.
As the title of this Stream suggests it Focuses particularly on Agile. As software testing evolves, agile is becoming a critical part of software development and testing. So this Agile stream focuses primarily on agile practices.
The specialist stream focuses on specific approaches or testing activities such as mobile, automation, pharmaceutical or even medical devices based testing.
Yes, across all of these certifications. The foundation core is an absolute must and cannot be skipped.
In addition to this, any advanced level certification can only be considered if the foundation level is done first. And as you may expect, any expert level will require that the advanced is done.
In this section, I’m going to recommend what I believe to be the best ISTQB course. However, with so many different levels and streams, your best choice largely depends on what level you are at in your career.
Or, what qualifications you have. For example, if you already have a CTFL, then my recommendation for this course is obviously not applicable to you.
Therefore my recommendations are based on a selection of a few different skill levels. In particular:
The best ISTQB course for beginners is definitely the certified tester Foundation level (CTFL). This is the prerequisite for other levels if you want to progress.
This course is aimed at anyone that is actually a tester that hasn’t been formally certified. Alternatively people interested in getting into the industry that has not had any experience.
For example, this could include:
It covers a number of different core modules including the fundamentals of testing. Such as, “what is testing?”. “Why do we test”? etc.
Also covers test techniques such as:
Other areas include Software Development Life-cycle (SDLC).
The advanced level test manager certification is for testers that have reached an advanced level in testing. They are either working as a Test Manager and require a certificate for their CV or career progression.
Alternatively, they are aspiring test managers that want to get to test manager level, but have adequate experience.
It is also helpful for advanced level IT workers that wish to get a better understanding of the requirements of a test manager.
This course covers a deep understanding of test planning, estimation and it also goes into people skills. People skills, in particular, is something that some courses tend to overlook.
This includes motivating staff, how to get the best out of your team, etc.
Prerequisites for this course is a CTFL and you also require a certain amount of industry experience to be eligible for the certification.
Once you have completed this certification you can then move on to other levels such as the expert level certifications.
The foundation level agile certification is one of the newer offerings. There is often confusion about Agile. In particular, how it should be applied and used in anger during a software project.
In my experience of working with agile, I find that I have never seen Agile implemented the way across any two companies.
The agile certification is an attempt to formalize the understanding of agile and to create a solid foundation of the core concepts and practices.
It covers such topics as the fundamentals of agile. Such as what agile is, how it is handled in a software testing environment, etc. It also goes into detail about the tools that can be used within agile and more.
The prerequisites for this course is the CTFL, which is required across all of the certifications as discussed earlier.
The next levels after this particular certification are currently in development. This includes plans to release an advanced level technical agile tester certification which will be a perfect add-on to this.
How long does it take to get an ISTQB? For the CTFL exam, it will typically be 60 minutes. However, for non-English speaking countries, such as in French-speaking parts of Canada, it is extended to 75 minutes.
The course is structured into 40 multiple choice questions. Each question scoring one point. To pass the exam you need to get at least 65% right. Which means you have to get 26 out of 40 to get a passing mark.
Is the ISTQB certification useful? Yes, in my opinion. It is useful for some people, depending on what area of work you are in. If you are a tester who is just starting out, this certification is good. And will be a booster on your CV. This can help you to get more recognition and jobs in the future.
However I would say, the certification does not necessarily make you a good tester. It just proves that you understand the fundamentals of testing and what is required in the industry.
I remember turning up for football training as a young lad. I was amazed to see one of the guys on the team had the brightest, most expensive football boots on.
I knew they were expensive and he let everyone know it. After I saw him get tackled and embarrassed for the third time in a row, it reminded me that just having the gear, does not mean you have all the ideas. It’s the experience and skill that matters, not the cost of your boots.
Similar to testing, you can take the best ISTQB course. However, real life test execution, office politics, and the real world of testing is a whole new kettle of fish. This can only be mastered with experience, not by sitting an expensive exam.
If you are involved in UAT or functional testing, you may have wondered what are the key differences, or if they are one of the same thing. Well, let me explain…
Is UAT functional testing? No, UAT is not functional testing. UAT is a process of checking and validating that the user requirements have been met. This is typically governed by the User Requirements document. Functional testing, on the other hand, is verification that the system meets the functional specification.
To put things into perspective, it is possible to pass functional testing, with the system meeting the functional specification, but still failing UAT. But now that you know that they are different let me explain each one in more detail, the types of testing that fall into these categories and more…
UAT is a validation process which proves the contract (user requirements) between the project team and the user has been met. This is based on User requirements agreed from the beginning of the project.
This is typically conducted by end users (customers). Or in some cases, it is done by a product owner who is working as a proxy for the actual end-user.
UAT testing is important because it confirms that the customer’s requirements have been met. It is also possible to find issues in the system that could have been missed during functional testing.
It is also an important quality gate. Often the final gate before the system goes live.
Functional testing (Click here to see Why Functional Testing is Required) is a way of verifying that each system function meets the functional specification document.
Inputs are put into the system with the actual results (or outputs) of these functions compared against the expected results defined in the functional specification.
I have mentioned the functional specification a few times in this article. Which is a key document in the traditional v-model, used in testing? However, in the real world, from my experience, you will often find that there is no functional specification.
This is the real challenge to a tester. Because, the books ion testing, or ISTQB exam never really prepare you for this.
The main activities of functional testing use black box testing…
Black box testing is a functional based procedure which does not concern itself with the underlying code within each functional module.
Therefore, a tester running black box tests does not need to know the exact code behavior within that function. Meaning they don’t have to be a developer.
They just need to know what input is required and what to expect to get out of it, as per the defined specification. Hence the reason it’s called the “black box”.
Functional testing usually tests functions such as APIs, databases, user interfaces, etc.
In this section, I’m going to explain what different types of functional testing there is.
The following forms of testing would be regarded as Functional testing:
Integration testing is a classic example of this ( Click here to see the difference between System Integration Testing and UAT testing). During integration testing, you will be verifying if each individual system module can integrate and talk to each other successfully.
For this to be successful, inputs from one module will feed into another function. Hence the reason why it is typically called Integration testing.
So, for example, in one of my previous testing projects, we were working on a large financial system. This system was responsible for checking customer debt. Depending on the debt levels, certain actions would be triggered, etc a letter to a customer.
To gather accurate data on the customers, required multiple XML feeds into a central database. It would collect data, for example, address information and credit score data from external vendors.
Each one of these XML feeds was effectively integration tests that needed to be verified using functional testing.
Verification is what most people think of when they talk about system testing.
It is a procedure to verify that the system works as per the specification. In particular, the functional specification. In basic terms, it means – is the product built correctly?
Validation is different if you are thinking of user-related testing. Then, the chances are your thinking of validation testing.
Validation in basic terms means – are we building the right product?
It Focuses on confirming if the system that has been built and verified actually delivers on the user’s expectations.
And this is validated against the User Requirements that were drafted in the very beginning of the project.
Non-functional testing is test procedures that confirm non-functional related specifications such as reliability, performance, etc.
These tests are performed based on separate non-functional requirements. They are not covered during functional testing.
An example of a non-functional test would be: How many simultaneous users can be loaded onto the system?
Typically these tests are done by specialist test resource. They focus on non-functional testing. These testers are usually very well paid because they have rare skills that are often needed by large organizations.
If if you are ever considering a highly paid contracting role for the future, this is something to consider.
Usually, this testing is done by some form of automation such as LoadRunner, QTP, etc.
The reason for this is, it is usually quite challenging replicating these test scenarios without using test automation.
An example of non-functional testing that I was involved in, as a test manager. During a previous project for a large energy company, we needed to simulate some load on the system.
We didn’t have the budget for automation tools, and time scales were tight. So, to simulate this load, we required multiple people all in the same room.
Each person was using the system at one time to verify the system could handle the load. The reality is this method is quite hard to coordinate, and there is room for error.
Also, this is quite a costly affair because you need to have everyone coordinated in the same room at the same time. And time is money. For these non-functional tests, If the project has the budget, automation is easier. because you can click a button to replicate the load.
No, white box testing focuses on the underlying code in a function. This is typically referred to as unit testing and is usually executed by an actual developer.
It is a very important activity which typically happens before the system is passed on for functional testing. Think of it as an important prerequisite before you get into black box testing (or functional testing).
End-to-end testing is a testing method to verify that the entire system application, from start to finish, performs as expected. It verifies the flow of data works as per the specification.
Q: What is compatibility Testing?
This is classed as a non-functional test. Its main job is to verify that the application is compatible with different devices, browsers, etc. For example, let’s say you were working on a web application, such as an online sweet shop.
To perform compatibility tests, you would check the web app on multiple browsers to verify that it is compatible with the major browsers. This is just one example obviously.
These requirements, governing which browsers will need to have been documented and agreed on ahaead of time so that it is not an open-ended test on whatever browser the tester feels like testing. And to also ensure that the agreed coverage is met.
If you have ever worked in system testing, you must have seen when a system does not integrate correctly or just doesn’t work as expected, even after the system testing has passed. This is usually due to the lack of SIT (System integration testing), but how is this different from UAT? Let me explain…
What is the difference between SIT and UAT testing? SIT (System Integration Testing) focusses on proving that the system functions integrate successfully. With each input/output message invoking the next function successfully. Whereas UAT proves that the system meets the User requirements.
Now you know the difference. Let me explain who works on these phases, how they fit into the software development life cycle, the different types that are available and more.
UAT (User Acceptance Testing) testing is a method to verify that the system application meets the user requirements that were laid out before the software development began.
It verifies from a user perspective that they are happy to accept the system.
SIT stands for Software Integration Testing. It is basically a way to verify that the overall system integrates with the subsystem modules as expected.
Essentially, it makes sure that the end-to-end system hangs together as expected after system testing.
SIT typically follows system testing and is based on verifying that the system meets the functional specification. This is the process typically used with the V-model.
UAT usually happens towards the end of a project test phase. Usually, the last step before the system is accepted to go into production.
This is after its been proven in system testing, SIT and sometimes other phases such as OAT (Operational Acceptance Testing).
The reason it is usually at the end of the project is that at this point in time the system should be almost clear of all system serious system defects.
In fact, if you are following the correct procedure it will have had to have passed the exit criteria (click here to see what QA sign off is) of the other phases before it gets to UAT. So, by that time there should only be non-show stopper defects if any.
UAT testing is typically done by a designated product owner. When I say product owner, these are users that typically use the system day in day out and are deemed as product experts or SMEs (Subject matter experts).
When I say experts they are not necessarily technical people. They are usually very skilled in using the application on a day-to-day basis, so they know all the nuances or expected behaviors, are you with me?
As well as verifying the requirements have been met, they are ideal for catching ad-hoc issues that could ordinarily get missed by a less experienced individual.
Their input is valuable because there are a lot of things that they know about the system that maybe even the technical testers may overlook.
SIT is usually done by very technical and skilled testers. These are usually dedicated resource who focus on testing in their job role.
They usually have very good technical skills and understanding of the modules that will be integrated. For example, these skills may include advanced SQL query skills.
There are two types of UAT testing which are as follows:
In all honesty, these distinct types of UAT testing are not always implemented by every company. In my years of experience working on software projects, which includes some large blue-chip companies, it is very rarely split up into these two distinct phases.
But, for your benefit, I will break these two distinct approaches.
Alpha UAT is typically done on the development site by a product owner or user accepted acceptance representative. This user will be able to test the system at the control of the development or project team.
Therefore, the project team is still largely in control of the environment and user access, etc.
Beta testing, on the other hand, is typically done on the clients actual site. This is using their own environment outside of the project teams control.
The benefit is, it mimics the real-life environment more. There still may be a representative from the project team on site to lend assistance, but not always.
Let me share an example of this from my experience of testing. Whilst working for an energy software consultancy. We were tasked with providing some software for them.
After SIT testing, was a phase of UAT on the client site using their environment. Any defects were entered into our defect management system for triage.
SIT can also be broken down into two major approaches:
Let Me Explain…
The top-down approach is focusing on the main parent function. The child modules can or will be replaced by stubs.
What is a stub?….
A stub is a test harness that will return an expected result to a parent function. This is used to test an application with missing (or incomplete) functions.
For example… Cash machine…
Let’s assume we have a cash machine application and we want to test the interface that the customer uses (the parent module). In this example we want to test that the interface links to two child functions:
In this example, we could replace the whole withdrawal cash page with a stub. This could be written to return an expected value when the user selects the option from the main interface page.
This is a way of testing to see how the parent function will perform in the absence of the parent function.
The bottom up approach is essentially the complete opposite of top-down. You start with the lower level modules and use drivers instead of stubs.
A driver is the opposite of a stub. It will actually provide the input that you expect into a bottom level function to prove that the child function works.
So if we revert back to a cash machine application example…
If we want to prove that after viewing your statement you are returned to the parent menu. A driver could be used to provide a value to confirm that it performed as expected.
An information session or UAT kick-off session is a basic session to transfer information to the UAT testers before starting the UAT phase.
In this session, you will give important information such as
So, in a nutshell, it’s a way of establishing the expectation for UAT after following system integration testing.
In my experience of UAT testing, this is always been a very useful session. Because, if you do not have this initial startup information meeting you will find that you’ll get a lot of these questions asked by multiple people during the test process. Often, you will end up duplicating information again and again.
What is the difference between SIT and System Testing? SIT, as discussed earlier, is mainly focused on how the individual system application modules integrate together.
System Testing focuses on the behavior of the system functions. Hence the reason why it is often called functional testing. The goal of System testing is to prove the system meets the functional specification.
Why do you even need system integration testing? It proves the dependencies between system modules. Sometimes it is possible to get carried away focusing on which tests pass at a functional level.
But, not focusing on how they will hang together once all the functions are proven.
If you have ever heard of progression testing, then the chances are, you may have heard of regression testing. But are they different? similar, or something completely different? Let me explain…
What’s the difference between progression and regression testing? Progression testing focusses on new functionality and proving that it works as per the requirements. Whereas regression testing focuses on proving that existing functions of the application are not broken from the addition of new code.
During my years of testing, regression, and progression testing have been critical. Especially when you are dealing with systems that have serious impacts on customers. So, let me explain in further detail the major differences. In addition to this, I will give some examples to make it more understandable.
Is reality its another word for saying functional testing. Some people also refer to this as incremental testing (Click here to see what the incremental model really is), depending on what type of testing model you’re using.
Essentially, it’s testing new functionality in a methodical approach. To make sure that the new functionality matches the desired requirements.
I say sometimes referred to as the “incremental model” because it depends on what model of testing is being used. For example, you could be using the V-model.
However, whatever your model of choice is, progression testing still comes down to testing new functionality against your requirements.
Functional testing is a method of verifying the outcome of the application in comparison to the agreed requirements.
It is often referred to as a form of “black box testing”. This is commonly referred to as black box because we are not focused on the underlying code for this. Testing code modules is known as unit testing (Click here to see the difference between unit and integration testing) or “white box testing”.
So, ultimately with functional testing, you are looking at the functions of the system and making sure that expected outputs are seen, are you with me?
Regression testing (Click here to see What Change Related Testing is?) is different from progression testing. Ultimately, with regression testing, we are making sure that we have not broken anything when we introduce new code. Simple as that.
Let me give you a working example here…
Let’s assume that you’re testing a calculator application. Currently, it is very basic and can only add and subtract, nothing else.
Now requirements have been drawn up to add a multiplication feature to this app. We can also assume that the application is fully functional and in production at present.
To regression test, the calculator application after the new multiplication feature is tested. You need do some additional tests to:
Regression testing is needed because mistakes or errors in code can be expensive to clean up post-go-live.
Typically you’d need to use regression techniques for the following situations:
All of these potential changes can provide a risk of damaging existing functionality which can have adverse cost effects to the business if they are not tested.
Once you have decided that regression testing is important and worth the time investment. You then need to think to yourself, how are you going to tackle it? Let me explain…
Essentially, there are three main ways to approach regression testing:
Testing everything is one of the most thorough ways of doing regression testing. Meaning you test the entire suite of tests after every new change is implemented.
You’ll often find that most companies will not want to invest in this. Simply because there is a massive cost impact for doing this.
To be perfectly honest, from my experience of testing, it just isn’t feasible to be able to do this every time a change is implemented into the system. So, it’s unlikely that this will ever be used.
A selection of test cases can be chosen instead. This is usually the approach businesses choose. But there a decision made on what to test. In some cases, a standard regression suite can be selected for this purpose. Depending on the business needs.
Prioritizing is a clever way to select which test cases should be tested first. Depending on time restraints its likely that only the high priority regression tests will be executed.
Essentially, you and your project team make a risk assessment on which test cases need to be focused first. This could be based on the risk of the system being impacted or it could be based on how often a particular function is used.
There can be a number of different reasons behind these decisions. But, essentially it comes down to testing only a subset of test cases based on their perceived priority or risk Factor.
In agile testing, you mainly use short sprints of functionality (click here to see what an Agile sprint is) to test. Therefore there are two main ways that you can tackle regression testing in agile:
At this sprint-level, you will effectively test the functionality which is relevant to that particular sprint. It is more of a scaled-down modular based approach.
The end-to-end approach Focuses on the entire application. With each sprint module talking to each other. To regression test this, a selection of test cases are run to ensure that there are no problems (defects).
In modern agile testing, there is a growing appetite to automate the majority of these tests to speed up the delivery to market.
For example, in my experience of testing, I was working for a well-known mobile telecommunication company. They had a Christmas promotion one year targeting millions of homes.
Effectively it was offering their customer base as a special offer for Christmas. This was new functionality added to an existing codebase. Therefore, a large amount of repeatable regression testing was required.
Due to the number of man-hours required to test this manually a decision was made to use selenium test automation suite.
We implemented the selenium test automation suite to go through many laborious iterations of dropdown forms on a web application. This is an example of how test automation could be used to speed up regression testing
As mentioned above Selenium is one good example. However, there are quite a few others such as QTP, winrunner, etc.
The advantages of regression testing is it gives your application higher quality delivery. It’s one of the best ways to secure the quality of your application.
Also, you have the advantage of being able to automate some of these laborious tests to speed up the delivery.
Disadvantages are the additional cost that comes along with regression testing to bolt onto a project. Unfortunately, some companies have a tight budget and cut corners. This can be a disadvantage.
Also if automation is not an option for your company due to lack of budget for the tools, you are left with quite a large burden of manual tests.
Either way, in my opinion, regression is critical. And, although it may seem expensive when you look at the costs up front, long term it can save your company some serious time and money downstream. The costs after going live can be exponentially more than if it was picked up before then.