If you are new to testing the different types of testing can be confusing. During my career I have done unit testing as a developer, but primarily system and integration testing as a tester. Today we are going to focus on Unit and Integration testing in particular.
What’s the difference between integration testing & unit testing? Unit testing is typically performed by developers and focusses on one particular function or code module. Integration testing focusses on the communication of one or many code modules or functions to prove that the inputs and outputs work as designed.
A unit test, or white box test as it is also known, is typically done by developers. An entire application consists of many individual cod modules or functions. Typically this is created by a team of developers working together on a tight timeline.
Each developer is responsible for testing that their allocated code module meets the expected design. For example, if the application was a calculator. One developer might work on the addition function and another on multiplication.
The developer working on the addition function would need to prove that this function worked in isolation, before it is integrated into the “calculator” application.
If we extend the calculator example above. An integration test, in this context, would prove that the calculator application modules talk to each other. This testing is typically done by test analysts and can also be referred to as Black box testing.
The reason for this name “Black Box”, is because the tester does not need to have intricate knowledge of the individual code module, they just need to insure the modules talk to each other and the “calculator” application hangs together.
There are two main types of integrations testing: Incremental and non-incremental. Firstly we will focus on Incremental. With incremental the developer will test out integrations earlier in the development process.
This is done by using stubs and drivers to simulate inputs and outputs for incomplete code modules. The most important advantage is that issues can be identified very early. This will in turn reduce costs by delivering a more quality product to the dedicated Test analysts.
Within the Incremental integration approach, there are two variations: “Top-Down” and “Bottom-Up”. Both of these types are named after the flow of the testing.
In the Top-down method, if you can picture a pyramid split up into different levels. This defines the functionality, with the main function at the top, and the minor functionality at the bottom.
With the Top-Down approach, you start with the top level code modules and substitute the lower level modules with stubs to prove they work ahead of time.
What is a stub? A stub is placeholder program that will take an expected input and output an expected result. This allows the developer to code a function and test its inputs and outputs work, before the actual module intended to integrate is ready.
E.g. You have an application called calculator, and within this application you have a function called brackets, addition and subtraction. The brackets function works out the calculation when you have addition and subtraction in brackets.
However, to test the “Brackets” function you need addition and subtraction ready. So, to gut around this problem, you use a “Stub” called “addition” and another one called “subtraction”.
This way you can test “Brackets” function without waiting for the other modules being ready to test.
As you can imagine, this is opposite to the Top-Down method. In this method, the lower level code modules are completed and tested without the parent or higher level modules being available.
For this to be possible drivers are used are used to provide the necessary inputs. One of the biggest disadvantages of this method is the order of the development. Unfortunately because the priority high level code modules are developed last, there could be major issues with these key inputs down the line.
The incremental model is preferred because you can gradually bring in each nodule and deal with issues in manageable chunks. However, if it is unclear exactly how these modules will communicate you can use the “Big-Bang” approach.
Effectively taking a number of modules and integrating them all in one go and seeing what happens. As you can imagine it very rarely works straight away and is likely to be a panful process.
The biggest issue is identifying exactly what module is at fault when the integration fails. The large amount of integrations can make the process very complex.
Integration testing (click here to see who typically does Integration Testing) is performed after Unit testing but before UAT. In my experience of testing a System testing phase tends to happen first to prove the main system is functioning correctly, then integrations are tested after.
When Is Unit Testing Performed? Unit testing is performed after each module of code is developed. The idea is to improve the quality of code that is delivered to the system testers.
Stubs and drivers can be used in the absence of other required code modules. For example, if a developer was creating a “brackets” function in a calculator app. He may be waiting for the other “Multiply” and “addition” code modules to be complete, before he can unit test his function.
Instead of waiting he can use stubs and drivers to provide the expected inputs/outputs to prove that his function works, once the other modules are complete.
Can JUnit be used for integration testing? The short answer is yes, it can be used. But it depends on what you want to test with it. Essentially it is not the ideal job for this task, but you can adapt it to work for some integration tasks.
What is a Black Box Test? A black box test is basically a method of testing that does not require the tester to see or understand the inner workings of the code functions. For example a System test phase may have a selection of functional tests.
The tester does not need to know how to code to run the test, in fact they just need to know the expected result from a successful functional test.
What is a White Box Test? As appose to Black box, this test expects you to have knowledge of the inner workings of the code function. Unit testing is an example of White box testing. With this level of testing you are expected to understand the code level inputs and outputs required to complete the test.
Agile without the Scrum master is like an orchestra without a conductor. In my years of Agile, this person is always the central person in the team.
Who is the scrum master in agile? The scrum master is the organisor of an Agile development. Think of this person as the process manager of Agile or if you like, the linchpin that brings all the related project team together to follow the Agile process.
Now that you understand from a high level what a scrum master does, you need to understand exactly what they do and how they fit into the process. If this interests you, please read on for more information.
During an Agile scrum all project team members huddle together for a quick “stand-up” meeting. The reason it’s a stand-up meeting is so it physiologically reduces the expected duration.
Meaning it is likely to be shorter and concise if you do not sit down, well that is the idea.
The Scrum Master is effectively the organiser of the scrum and really wants to establish three key peieces of information from each scrum meeting attendee:
Even though this may sound like the scrum Master is similar level to a project or programme manager, in reality this is not the case. The scrum master is just responsible for facilitating the scrum, not the outcome of the project or programme o work.
During the scrum, we just established what the scrum masters role is. But what is their objective you may be thinking? Well they have a challenging task of keeping the team focused and most importantly avoiding blockers that could prevent progress within the current, or future sprints.
As well as this, they have to make sure the team is in agreement with the direction of the day. For example, if you are asked “what is your plan today?”. Your response may be: “Work on the second drop down list data feed”.
There is a chance that your response my cause an objection or suggestion to this a bit different. This is where scrum master is valuable to keep everyone on track and reach an agreement.
As discussed earlier, the scrum master is not a project or programme manager. And in fact they may not have any real project level authority at all. However they do own the “process”.
What does this mean? Basically they can decide how the scrum is organised, who speaks next. Or even if they want to make alterations to the sprint duration.
Bare in mind, all of this is in theory. However, in my experience of testing throughout the years, the real authority to make these decisions stick depends on the politics of the company you work for and how respected that individual really is. But on paper, they should have the ability to do this.
Apart from the actual scrum itself, the Scrum Master also ads value to the Product owner. In particular making sure that the team fully understand the product backlog and importance of having very clear and understandable backlog items designated to each sprint.
The value add here is helping the development team come up with the highest quality products that they can. This is mainly by helping to remove any blockers that may be preventing their progress within the current sprint. Also making making sure they understand the sprint plan and how that related to the product backlog.
In general the scrum master works to help the organisation understand and adopt the scrum process. Initially it can be hard to get a project team to conform because they are not used to the new process. This is where the scrum master really adds value.
According to glassdoor, a scrum master in the United states earns, at the time of writing, $115.992 and in the UK, approx £50,000. Obviously these are averages and depending on where you live, the company you work for, this can vary quite significantly.
Do you need a degree to be a Scrum Master?
You don’t need a degree to become one, but you do need to have a good understanding of Agile Scrum. In addition to this, it is advisable that you have a good idea of how your company works, the products they use and the culture of the organisation.
This is because the Scrum Master needs to work with a lot of people of varying levels of skills and hierarchy, and to gain their respect and do well, you will need these skills.
To become a qualified Scrum Master you will need a qualification. For example the Certified Scrum Master. However, this is a certificate, it will be the stepping stone in your carrer, much like an ISTQB foundation qualification is a stepping stone in testing.
My point is, you will still need some experience in the company to really be considered for the role. Also, bare in mind in reality there are people out there now fulfilling this role of Scrum Master without this qualification, much the same as there are testers without ISTQB qualifications.
What is the Certified Scrum Master (CSM) Exam? CSM is a certification provided by the Scrum Alliance. Once you have complete the CSM training you can can take the exam and have the opportunity to be certified. The course training is typically done over two days with a qualified trainer. These trainers are referred to as “Certified Scrum Trainers”.
How long is the CSM exam? The exam is about an hour and is done using an online system. It is possible to pause, resume during the exam, so the duration is the actual time. The elapsed time could be longer if you pause and take longer to complete it, if that makes sense.
To pass the exam you need to get at least 24 out of the total 35 questions that are asked. The great thing about this exam is that once you submit your answers you can get an instant score. This is because it is a multiple choice question exam, meaning you have fixed answers.
What happens if you fail the CSM Exam? If you fail the exam you get the chance to re-take it again. As long as you do this within 60 days you won’t be charged. However, if you fail it again there will be a fee tat you’ll need to pay to try again.
The ultimate goal in testing is proving that the system in question is ready and fit for purpose. The official way to do this is via a formal process called a QA Sign off.
What is a QA sign off? It’s the test team’s, or quality assurance (QA) in this instance, method for formally declaring the completion of testing. This declaration is recorded and referred to if required after testing has completed.
To make the decision that testing is complete, obviously you’ll need some kind of criteria to make sure that you’re confident that the sign off has been completed. This criteria is the agreed exit gate definition that governs when testing is complete.
The answer is, yes. The exit criteria is typically listed as one of the main headings in the Test Plan, as well as the entry criteria.
So effectively, the QA sign off is the actual declaration of testing completion. And the criteria of this is typically documented in the Test Plan.
The Test Entry criteria is effectively the list of requirements to govern when testing can start.
With regards to the exit criteria, we spoke about this in greater length earlier
Following on from what we discussed earlier, when I explained the different types of exit criteria. One other example of a QA sign off criteria item is:
are all of the priority tests being executed?
I say, “Priority,” because in some cases where you have really been time-pressed to complete a test phase, you may agree that you can perform the top priority tests and the lower priority ones.
For example it might be a test that focusses on a more cosmetic test, for example checking the labels on an e-commerce order form.
Another example is:
No P1 or P2 criteria defects are in the system at the time of completion.
And maybe another example, one that i’ve used in my experience is,
Minimum of 5 P3 defects with documented and agreed plan to fix after go-live.
This sign off criteria definitely needs to be agreed upfront. It can’t be an afterthought once you started testing. Because the worst case, there’ll be an argument about when you can actually finish testing, because one party may feel that you haven’t done enough.
So , to avoid this, it’s essential that this exit criteria is agreed upfront, and documented. And the document is signed off. Now in reality, getting the Test Plant signed off before testing starts, believe it or not, doesn’t always happen.
And sometimes there’s a battle to do this, because for some unknown reasons there’s people within a project team that are frightened, in a way, to sign off a document, because they don’t want to admit any liability in case anything goes wrong down the line.
But theoretically in testing, you’d want the sign off criteria out of the way and signed off at the beginning. The agreement of the criteria is a collaboration of team members.
The responsibility is a collaboration of all interested parties. And these parties usually will be the Business Analyst (BA), the Test Manager if you were the test lead in this instance. And the Program manager or Project Manager
This is subjective, depending on where you work and how your company works. But effectively the most common way is in the form of an email that is sent out.
And who this email goes to depends on your company. For example, it might go to the project manager, and it may come from the test manager.
Whoever gets it, ultimately it is basically a simple email.
Really and truly this doesn’t have to be War And Peace. This has to be very basic and simple, and get to the point. So a very simple email just explaining that you’re happy that the testing has met the exit criteria is fine.
It’s important that you explained the exit criteria, because you don’t want to this be a subjective statement. You want it to be based on facts. So you want to relay it and relate it to the exit criteria that was agreed.
So for example you might say, “The testing has met the exit criteria as documented in the Test Plan section. Blah, blah, blah.” And then you may relist the items in the exit criteria, and then you will just explain how you’ve met that.
For example if you had 300 test cases to run, and 200 of those test cases were priority one’s. And the agreement was that you had to at least complete the priority test cases, then you would state that 200 test cases of priority one have been done as per the exit criteria.
So it’s just listing out how you have come to the conclusion that the testing has actually complete, and keep it as simple as that.
This happens quite a lot of time. Like for example, you may have a very strict deadline to meet. But then once you meet the deadline, you may find that you don’t actually complete all the tests.
You haven’t actually met all of the test exit criteria, and you have to make a call as to what happens next. And this is typically in a form of a conditional sign off. And a conditional sign off is effectively a way of agreeing that the testing will be accepted for completion, but with some conditions as in certain functionality will have to be tested at a later date.
Or you’d have an agreement for a workaround for a problem that you’ve discovered in testing, and therefore it will be accepted with those conditions.
A checklist is a catalog of items that are typically recorded for tracking. This list is ordered in a sequence. You may or may not be ordered in a sequence. But it’s effectively just the way to work out exactly what needs to be done for testing.
And it’s a checklist much the same as anything that could be used outside of the context of testing. But that effectively is what it is.
As briefly discussed earlier, the conditional sign off is ultimately a way for you to have acceptance of exiting the testing, but with a list of agreed conditions to move forward.
Why do you really need a QA sign off? Why not just stop testing and then just go into production?The sign off is quite important, because it is a formal process which is recorded and can be looked back to if there are any issues in the future.
Also, it’s a way to get buy-in that everyone accepts that testing has completed in case there are any issues. And it’s the way to prove that you’ve met the actual exit criteria.
Basically its a way to cover your back and get something documented. As projects go into production and issues crop up in the future, then there may be a need to have some traceability about how things were tested. And you may be challenged on these in the future. So it’s always good to have an agreed sign off.
Is there a sign off required for UAT testing? In the same way as you have the QA sign off, UAT testing (Click here to see the difference between UAT & SIT) definitely is required to be signed off.
This is quite critical as well, because this is the acceptance from the actual stakeholders or the customers that are gonna use the software in production.
For this reason it’s important to get their buy-in. This proves that they’re actually happy with what they seen. Believe me, this is critical. When things go into production, if things don’t go correct, if you’ve got QA sign off andUAT sign off, it makes your life a whole lot easier.
So as I said before, the UAT testing it typically signed off by a nominated product owner. And this might be someone who is very skilled in that particular function of not necessarily testing, but actually uses the product day-in, day-out.
For example, if you were working on a finance system, and it was an invoice inventory system and you had someone who’s day job was to use that every day. Then they would be the perfect candidate to actually do UAT testing. This is Because they know how the system should work. After they they are informed about the new functionality, they can make a decision if it meets their criteria.
If you are interested in Agile, no doubt you have heard of Sprints. But what are they? Well in this article I plan to reveal all and explain more.
What is a Sprint in Agile? Sprint is an iteration of time boxed work that is used within the scrum framework of Agile. Typically, these sprints are short, modularised pieces of development/testing which aims to meet the sprint goal.
To truly understand how Sprints fit into the whole Agile methodology, you need to read on. I will explain the different between sprints and scums, how sprints fit into the Agile Scrum framework and more detail about sprints in Agile.
In some cases people may get these confused, but effectively they're very different. A scrum is a meeting, that is typically done on a daily basis and it's an opportunity for the team members to get together to make sure they're still on track for the sprint.
It is an opportunity to work through any challenges they may have had the day before, and to focus their attention on what needs to be done for that particular day.
It's usually a very short 15 to 30 minutes get together, and then you continue with your rest of your day, and the main focus is on the sprint itself.
The Sprint on the other hand. Is an agreed iteration of work to be delivered in a agreed timeframe.
A sprint cycle is really just another way to say a Sprint. As discussed earlier, a sprint is a timed boxed piece of work used in the scrum framework.
Sprint planning is effectively planning the up coming sprint. It is a collaborative effort which typically involves a scrum master.
For those of you that are familiar with the framework of scrum. The idea is, the scrum master facilitates the planning of upcoming sprints, and agrees, with the project team, all of the backlog items in the product backlog that are going to be used in each sprint.
As discussed earlier, a sprint it is an agreed selection of backlog items that will be delivered in an agreed time frame. Obviously the very specific tasks that are delivered are subjective to the project that you're working on.
For example, let's say that you are building a new banking application, then maybe you'll have two weeks to work on just the graphical user interface and that will be the focus of the first two weeks.
The day-to-day scrums will make sure that you're on track to deliver that sprint goal.
As you can imagine, there is a bit of subjectivity here because it depends on your individual project, or how your company works, but the average is about two weeks.
It can be shorter, it can be a week or it could even be longer. It's not recommended to have really long sprints because the whole beauty of Agile is to have quick sprints, and then get quick feedback and quick iterations.
To cut a long story short, two weeks is the average, but you could be looking anywhere from one to four weeks, but most people go for two weeks.
What is a sprint backlog? A sprint backlog is effectively a list of tasks that have been identified to be delivered throughout the entire Agile project.
Each sprint consists of the agreed list of work items, for example, the two week sprint, that are taken from the sprint backlog. Effectively, the sprint backlog could literally be a document which has all of the pieces of work and development that are gonna be done throughout the entire project.
What is a sprint forecast? A sprint forecast is a scrum guide that refers to what will actually be delivered throughout the project itself.
This is largely reliant on the product backlog and the items that are selected for each sprint. It gives you a forecast of where you're going and where you're going to get to with your actual project.
What is a confluence sprint planning template? If you're into the world of Agile, then maybe you've come across confluence before.
Effectively, confluence is a collaboration tool that is used to help teams to collaborate and share their knowledge. The sprint planning template is just a template within this tool which enables Agile teams to work more effectively together, and effectively that's what it is.