2 Testing Techniques
A Testing Technique specifies the strategy or method to verify the software under test, by comparing the actual and expected results. Different quality aspects of software can be revealed through different testing techniques and the techniques can be categorized in the following two ways:
2.1 Black Box Technique:
Black Box test technique treats the system or the software program under test as a ‘Black Box’. Through this technique a tester is testing a software without knowledge of the internal workings or coding of the program under test and focuses solely on the outputs generated in response to selected inputs and execution conditions. It is solely based on analysis of business requirements, user specification etc. Black box testing is often referred as ‘Behavioral Testing’, ’Functional Testing’ or ‘ Opaque Box Testing’.
There are several methods that the testers follow during Black Box testing and some of the methods are listed below:
2.1.1 Equivalence Partitioning
Equivalence Partitioning is a black box testing method that divides the domain of acceptable inputs into classes of data from which test cases can be generated. The set of valid or invalid states for input conditions is termed as Equivalence Class and the guidelines for Equivalence classes are given below:
- If input condition specifies a range of input parameters, one valid and two invalid Equivalence Classes are defined for the test. For example consider a scenario to test an input box which accepts numbers between 1 to 10. Here any value within 1 to 10 will be considered as valid and two input value less than 1 and greater than 10 would work as invalid Equivalence Classes for the scenario.
- If an input condition accepts a specific value then one valid and two invalid Equivalence Classes are defined for the test. For example consider a scenario to test an input box which only accept alphabet ‘C’ and a valid pop up is generated . Here the valid equivalence class will be ‘C’ and the two invalid equivalence class should contain alphabet before and after ‘C’
- If an input condition requires a member of a set then one valid and one invalid equivalence classes are defined for the test. Again consider an input box which accepts only even number. In this case any even number will work as valid equivalence class and any odd number will work as an invalid equivalence class for the test.
- If an input condition requires a Boolean value then one valid and one invalid equivalence class is defined for the test.
By applying the above mentioned guidelines test cases for each input domain can be derived.
2.1.2 Boundary value Analysis
It is a well observed truth that a greater number of errors occur at the boundaries rather than in the center of the input domain for a test and that is why Boundary Value Analysis has become an important Black Box Testing method. Test cases can be mainly prepared by the following guidelines for boundary value analysis:
- If input condition specifies a range within values a and b, test cases should include a and b, values just above and below a and b respectively
- If an input condition specifies multiple values then the test cases should be produced to exercise the minimum and maximum numbers as well as values just above and below the maximum and minimum values respectively
2.1.3 Orthogonal Array testing
The Orthogonal Array Testing is a systemic method of testing pair wise interactions and this technique enables the design of a reasonably small set of test cases that provide maximum test coverage. Through this technique tester can categorize faulty logic which likely to be present within a program without traversing through the designed code and it provides a means to effectively generate high test coverage to validate the test domain.
Advantages of Black Box testing
- Testers need to have knowledge only on the functional requirements of the program and any knowledge on the internal workings of a program is not required
- Any gap between the system specification and customer specification can be identified easily
- Developers and Testers can work independently
Disadvantages of Black Box testing
- Test design can be hard and faulty as the testers do not have in depth knowledge of the application
- Only a small number of possible inputs can be tested
2.2 White Box testing
White Box Testing is the strategy of software testing in which the tester has an excellent knowledge of the internal workings of a program and it examines the program structure based on the program logic. White Box testing is also referred as ‘Glass Box testing’, ‘Structural testing’, ’Clear Box testing’ or ’ Open Box testing’ and it provides a complementary function to Black Box testing
Some of the techniques used in White Box testing are shortly mentioned below:
2.2.1 Basis Path Testing
Basis Path Testing is a white box testing technique which was first proposed by Tom McCabe and the aim of this method is to derive a logical complexity measure of a procedural design. This measure works as a guide for defining the basis set of execution path. This method assures that every statement of the program is tested at least one time.
For obtaining the basis set , the simple notation used for the representation of control flow is called a ‘Flow Graph’. Main components of a flow graph are :
‘Flow Graph Node’ which represents one or more procedural statements and ‘Edge’ or ‘Link’ which represents flow of control. Areas bounded by edges and nodes are called regions. The following figure is an example of ‘Flow Graph’
In this context we will discuss about one of the most important software metric ‘ Cyclomatic Complexity’ which gives a quantitative measure of the logical complexity of a program. Cyclomatic Complexity defines the number of independent paths in the basis set of a program. For a given graph G Cyclomatic Complexity V(G) is computed in one of the three methods:
- The number of regions of the flow graph
- Cyclomatic Complexity V(G)=E-N+2 where E is the number of flow graph edges and N is the number of flow graph nodes
- Cyclomatic Complexity V(G) =P+1 where P is the number of predicate nodes contained in the flow graph G
For the above figure the Cyclomatic Complexity V(G)=3 (6 edges, 5 nodes and 2 predicate nodes)
2.2.2 Control Structure testing
These are the White Box test techniques which focus on the control structures present within the software program. These broaden testing coverage and improve the quality of white box testing. Some important techniques used for control structure testing are mentioned below:
- Condition Testing
Condition Testing aims to exercise all the logical conditions contained in a program module. Compound conditions are divided into two or three simple conditions and tested through this process. The condition testing method not only focuses on testing each condition but also ensures that the whole program does not contain errors
- Data Flow testing
The data flow testing selects the test paths according to the locations of definitions and use of variables.
- Loop Testing
Loop testing is an important white box testing technique which focuses the validation of loop construction. In the software technology four types of loops are defined – simple, concatenated, nested and unstructured loops . Different set of tests can be applied on different loops to find out the errors.
Advantages of White box Testing
- As the testers have complete knowledge on the internal coding of the program, test data preparation becomes more easy
- Code optimization can be performed after white box testing
- All independent paths in a module is tested at least once
- Any hidden errors due to coding can be identified through these techniques
The main disadvantage of white box testing is the test cases are mainly planned to verify the code as designed instead of checking how the system will behave in a real life environment. As a result this method does not ensure the coverage on user requirements.
2.3 Differences Between Black Box and White Box Testing
Main differences between black box and white box testing are given below:
- Black Box Testing is a software testing method in which the internal structure /logic of the software program being tested is NOT known to the tester. Where as in White Box Testing the tester is aware of the internal structure/ programming logic of the software.
- Generally software testers perform the black box testing but the white box testing is mainly done by the developers
- Through black box testing testers can identify the errors based on the customer specified requirements. Through white box testing testers detect all logical and design errors
2.4 Other types of Testing
Quality of software is assessed in terms of many quality factors. Different type of software testing is designed to validate the software for one or more quality factor. Some of the most popular software testing types are discussed below:
2.4.1 Unit Testing
The testing is done to a unit or to a smallest piece of a software program. This type of testing is mostly performed by the software developers and they generally validate units of source code like statements, branches, functions , methods etc. JUNIT, NUNIT are the most common unit testing frameworks used by the unit testing team.
2.4.2 Integration Testing
Integration Testing is one of the important types of software testing. Integration Testing is a systemic technique to construct the software and at the same time conduct tests to find out errors associated with different unit of the program. The aim is to take unit tested components and develop the structure as per design. There are different approaches taken for Integration Testing like Top Down Integration Testing, Bottom Up Integration testing and the approach which consists of both is termed as Sand Witch Testing
2.4.3 Regression Testing
Every time a new module/unit is added/modified to a software program after integration testing, the behavior and the control logic of the software system changes. These changes may create problems for the existing functionalities tested earlier. Regression Testing is the re execution of some subset of tests conducted earlier to verify whether the existing functionalities are working properly or not. The scope of regression testing increases as there is any new module added to the software system successfully. The Regression Test Suite contains three different classes of test cases:
- A subset of test cases that will validate main software functions
- Set of test cases that focus on software functions that can be affected by the change of adding any new unit into the system
- Set of test cases focusing on the component of the software that has been added or changed
2.4.4 System Testing
System Testing is actually a series of different tests that validate the software as a whole against the customer specified requirements. Different types of testing like Recovery Testing, Security Testing, Stress Testing, Performance Testing etc. are carried out to complete System testing
2.4.5 Acceptance Testing
This type of software testing is performed by the end customers to verify if the software developed is satisfying the business requirements provided earlier. The customer performs a series of testing to produce errors before accepting the software and many cases it becomes time consuming
2.4.6 Performance Testing
Performance Testing is designed to test some of the quality attributes like stability, run time performance etc. of the developed software. Performance testing is carried out throughout all steps in the testing process. Performance Testing is performed to validate the non functional requirements and a separate performance engineering team exists to track the report of this testing
2.4.7 Stress Testing
Through stress testing the developed software is executed in a manner that demands inputs in abnormal quantity, frequency or volume. It involves testing beyond normal point in order to observe the robustness of the designed software. Stress testing also tests the behavior of the software with insufficient resources like memory usage, CPU usage, Disk space issue, abnormal shut down etc. A variation of stress testing is known as ‘sensitivity testing’
2.4.8 Load Testing
The main purpose of load testing is to determine the behavior of the software under normal and over loaded condition. As an example for a web application the determining factors for load testing are number of concurrent users, number of online transactions, data load processed by the host server for each transaction etc. Load testing is generally carried out by some automated testing tools to create the load artificially to find out the performance issues.
2.4.9 Smoke Testing
Smoke testing is a shallow and wide approach of testing whereby all areas of the software under test is tested without getting into too deep. It is conducted to verify whether the most important functions or modules are working or not. Usually a smoke test is scripted using a automation tool.
2.4.10 Sanity Testing
Sanity Test is a narrow and deep approach of testing that focuses on one or few areas of application to determine whether the function or module is still working after a small change. It is a cursory testing and it is usually unscripted.
2.4.11 Alpha Testing
The alpha testing of a software product or system is conducted by a representative group of end users at developer’s site. Most of the software builders performs this process of testing to find out errors that only the end user, who has a detailed requirement knowledge, seems able to find.
2.4.12 Beta Testing
Beta Testing is a formal type of software testing that is conducted by the end user at one or more customer sites. Unlike alpha testing the development team of the software is not present during this testing. Successful completion of this testing derives to the acceptance of the software by the end customers.
2.4.13 GUI Testing
GUI is the abbreviation for Graphical User Interface. A user driven software or program must be user friendly and the end user should be comfortable while using all the components on the screen and the components should also perform their functionality properly. GUI Testing is not only just ensuring the look and feel of the application but also to verify the functionality of the involved components. For example URL present in user screen validation, validation of the length of an input present in a form , pop up message validation etc. are some example of GUI testing
2.4.14 Ad Hoc Testing
Ad Hoc testing is an informal approach taken by software testers, business analyst or an end user without pointing to any specific test case or document. Testers, having sound knowledge on the application developed, perform this kind of testing to uncover errors which was not revealed through methodical test cases.