In the last post, we learnt about software testing levels and types. In this section, we are going to learn about Static testing techniques. Static test technique is a type of testing technique. Testing is of two types:

• Static test technique
• Dynamic test technique

Here, we will discuss more about static test technique and how it is different from dynamic testing. Static test techniques play a fundamental role in improving the quality and productivity of software development.

So, let’s look into the definition of static testing, how it is different from dynamic testing and why static testing is necessary.

Static testing is the testing of a component or system at specification or implementation level without execution of that software, e.g. reviews and static code analysis.

Dynamic testing is the testing that involves the execution of the software of a component or system.

Static testing is the testing of the software work products manually, or with a set of tools, but they are not executed.

• It starts early in the life cycle and so it is done during the verification processes.
• It does not need computer as the testing of program is done without executing the program, e.g., inspection, reviewing, walkthroughs.

Static testing is performed in the early stages of development to avoid errors as it can be fixed easily in the early stages. The errors that can’t be found in dynamic testing will easily be found in static testing.

With dynamic testing methods, the software is executed using a set of input values and its output is then examined and compared to what is expected.

Static v/s Dynamic Testing

Static testing and dynamic testing are complementary methods, as they tend to find different types of defects effectively and efficiently. One powerful technique that is used in static testing is reviews.

Reviews often represents project milestones, and support the establishments of a baseline for a software product. The type and quantity of defects found during reviews help testers focus on selecting their classes of tests effectively and their testing.

Sometimes, users/customers attend the review meeting and provide feedback to the development team, so reviews are also a means of user/ customer communication.

Static testing is a very useful method for improving the quality of software work products. There are two types of static testing: Reviews and Static Analysis by tools.

First, we are going to learn about Reviews and then we will look into the static analysis.

Review Process

            Reviews are informal and formal. Informal reviews are applied during the early stages of the life cycle of a document. A two-person team can conduct an informal review, as the author can ask a colleague to review a document or code.

In later stages, these reviews often involve more people and a meeting. The goal is to help the author and to improve the quality of the document. Informal reviews have types but there is one characteristics that is common in all informal reviews is that they are not documented.

The formal review process is related to the factors like the maturity of the development process, any legal or regulatory requirements or the need for an audit trail. Inspection is the most documented and formal review, yet it is not the only one.

Informal review is a review not based on a formal(documented) procedure.

And, Formal review is a review characterized by documented procedures and requirements, e.g., inspection.

Now, that we know that formal reviews are documented and informal reviews are not, there are differences between the formal and informal review.

Let’s see the difference between the formal and the informal review.

Difference between formal and informal reviews

So far, we learned what is static testing and how it is different from dynamic testing. Similarly, we learnt what are formal and informal reviews and how they are different from each other.

Unlike, informal reviews, formal reviews follow a process, called, phases of a formal review.

Phases of a Formal Review

The typical formal review follows a six-phase process:

Planning

The review process begins with a ‘request to review’ for a particular review by the author to the moderator. A moderator is usually assigned to take care of the scheduling of the review.

Moderator (inspection leader) is the leader and the main person responsible for an inspection or other review process.

For more formal reviews, the moderator always performs an entry check and exit criteria. The entry check is carried out to ensure that the reviewer’s time is not wasted on a document that is not ready to review.

Although there are more other steps for entry criteria but there is a minimum set for performing the entry check:

• A short check of a product sample by the moderator (or expert) does not reveal a large number of major defects. For example, after 30 minutes of checking, no more than 3 major defects are found on a single page or fewer than 10 major defects in total in a set of 5 pages.
• The document to be reviewed is available with line numbers.
• The document has been cleaned up by running any automated checks that apply.
• The document author is prepared to join the review team and feels confident with the quality of the document.

If the document passes the entry check, the moderator and author decide which part of the document to review. For a review, the maximum size is usually between 10-20 pages. In formal Inspection, only a page or two is sufficient to describe the depth and seriousness of defects.

After the document size is set and the pages to be checked have been selected, the moderator determines the composition of the review team, in cooperation with the author.

The team usually consists of 4-6 members, including moderator and author.

Kick-off

A kick-off meeting is an optional step in a review process. The goal of this meeting is to get everybody on the same wavelength regarding the document under review and to commit to the time that will be spent on checking.

During the kick-off meeting, the reviewers receive a short introduction on the objectives of the review and the documents.

The relationship between the document under review and the other documents (sources) are explained, especially if the number of related documents is high. Role assignments, checking rate, the pages to be checked, process changes and possible other questions are also discussed in this meeting.

 Apart from these, the distribution of document under review, source documents and other related documentation, can also be done during the kick-off.

Preparation

The participants work individually on the document under review using the related documents, procedures, rules and checklist provided. The participants identify defects, questions and comments, according to their understanding of the document and the role. All issues are recorded, using a logging form.
Checking rate is a critical success factor for thorough preparation. Checking rate is the number of pages checked per hour, which is usually in the range of 5-10 pages per hour.

During the preparation phase, the member should not exceed the checking rate they have been asked to use. Criteria for checking rate and document size can be set by collecting data and measuring the review process.

Review Meeting

This meeting consists of three phases: the logging phase, the discussion phase and the decision phase.

Logging Phase: In the logging phase, the defects which are identified during the preparation phase are mentioned page by page and are logged by the author or by a scribe.

To ensure progress and efficiency, no real discussion is allowed. If an issue needs discussion, then the issue is logged and discussed in the discussion phase.
Every defect and its severity should be logged.

Severity is what is the impact of a defect on the system/software under test.
Severity can be categorized further:

• Critical: defects will cause a complete shutdown of the process; we cannot proceed further.
• Major: defects will cause severe defect and collapses the system. However, certain parts of the system remain functional.
• Medium: It causes some undesirable behavior but the system is still functional.
• Low: It will not cause any major breakdown of the system.

The focus of the logging phase is mainly on logging as many defects as possible within a certain timeframe. To ensure this, the moderator tries to keep a good logging rate (number of defects logged per minute).

The logging rate should be between one and two defects logged per minute.

Discussion Phase: Informal reviews will often not have a separate logging phase and will start immediately with discussion. Members can take part in the discussion by putting their comments and reasoning. The moderator takes care of the people’s issues, like getting too personal, rephrasing remarks if necessary and calling for a break.

Reviewers who are not required to be in the discussion may leave, or stay as a learning exercise.

The outcome of the discussions is documented for future references.

Decision Phase: At the end of the meeting, a decision has to be taken by the participants for the document under review, sometimes based on the exit criteria. The most exit criterion is the average number of critical and/or major defects found per page. If the number of defects found per page exceeds a certain level, then the document under review must be reviewed, after it’s been reworked.

If the document complies with the exit criteria, the document will be checked during follow-up by the moderator or one or more participants.

Rework

Based on the defects found, the author will improve the document under review step by step. If the number of defects found per page exceeds a certain level, then the document has to be reworked.

Not every defect needs rework. It is the author’s responsibility to verify which defect requires rework and need a fix. If nothing is done about a defect for a certain reason, it should be reported to indicate that the author has considered the issue.
The author is responsible to indicate that where the changes have made because doing so it would be easy to identify the changes that are made to the document during the follow-up phase.

Follow-up

The moderator is responsible for ensuring that satisfactory actions are taken on all the logged defects, process improvement suggestions and change requests.

The moderator checks on the author to check whether he has taken all the necessary actions, it is not necessary for the moderator to check everything in detail. The moderator checks for the distribution of the updated document and collects the feedback.

It is the responsibility of the moderator to ensure that the information is correct and stored for future analysis.

Roles and responsibilities of the participants

The Moderator:

• Known as review leader
• Performs entry check
• Follow up on the rework
• Scheduling the meeting
• Coaches other team
• Leads the possible discussion and stores the data that is collected

The Author:

• Illuminate the unclear areas and understand the defects found.
• Basic goal is to learn as much as possible with regard to improving the quality of the document.

The Scribe:

• Also known as the recorder.
• During the logging meeting, the scribe has to record each defect mentioned and any suggestions for process improvement.
• Scribe is a separate person responsible for logging the defects found during the review

The Reviewers:

• Also known as the checkers or inspectors.
• Task is to check any material for defects, mostly prior to the meeting.
• Manager can also be involved in the review depending on the background.
• The level of domain knowledge or technical expertise needed by the reviewers also depends on the type of the review.

The Manager:

• Involves in deciding the execution of reviews, allocation of time in the project.
• Determines whether the review process objectives have been met.
• Manager will also take care of any review training requested by the participants.
• Involved in the review depending on his/her background.

Types of review

The main review types, their characteristics and common objectives are described in this section. Let’s look into the reviews.

Walkthroughs

Walkthroughs are characterized by the author of the document under review guiding the participants through the document and his or her thought processes, to achieve a common understanding, and to gather the feedback.

Within a walkthrough, the author is responsible for most of the preparation. A walkthrough is usually useful for higher-level documents, such as the requirement specifications and architectural documents.

Goals of walkthroughs:

• To present the document to the stakeholders, in order to gather the information regarding the topic under documentation.
• To explain and evaluate the content of the documents.
• To establish the understanding of the document and to gather feedback.

Key characteristics of walkthroughs:

• Meeting is led by authors, often a separate scribe is also present.
• Separate pre-meeting preparation for reviewers is optional.
• Scenarios and dry runs may be used to validate the content.

Technical Reviews

It is a less formal review and led by a trained moderator but can also be led by a technical expert. In practice, technical reviews vary from quite informal to very formal. It is often performed as a peer review without management participation. Defects are found by the experts (e.g. architects, designers, key users) who focus on the content of the documentation.

Goal of technical review is:

• Assess the value of technical concepts and alternatives in the product
• Inform participants of the technical content of the document.
• Ensure that technical concepts are used correctly in the early stages itself.
• Establish consistency in the use and representation of technical concepts.

Key characteristics are:

• Led by a trained moderator.
• Often performed as a peer review.
• It is documented defect-detection process that involves peers and technical experts.
• Use of formal characteristics such as checklists, logging lists or issue log are optional
• Separate preparation is carried out during which the product is examined and the defects are found.

Inspection

It is the most formal review type. A document under inspection is prepared and checked thoroughly by the reviewers before the meeting. In the discussion meeting, the defects found are logged and any discussion is postponed until the discussion phase.

Goals of inspection are:

• Remove defects efficiently, as early as possible.
• Improve product quality.
• Help the author to improve the quality of the document.
• Train new employees.
• Create a common understanding by exchanging information.
• Learn from the defects found and improve processes.

Key characteristics of inspection are:

• Led by a trained moderator.
• Uses defined roles in the process.
• Rules and checklists are used.
• The defects found are documented in a logging list and issue log.
• It involves peers to examine the product.
• Metrics are gathered and analyzed to optimize the process.
• Separate preparation is carried out during which the product is examined and defects are found.

Static Analysis by Tools

Static analysis is just another form of testing. It is focused on finding defects rather than failure and the goal is to find the defects whether or not they may cause failure.

Static analysis is the analysis of the software artifacts, e.g. requirements or code, carried out without execution of those software artifacts.

Static analysis tools are an extension of compiler technology – in fact, some compilers do offer static analysis features. It is worth checking what is available from existing compilers or development environments before looking at purchasing a more sophisticated static analysis tool. These tools are mostly used by developers as part of the development and component testing process.

Static analysis tools for code can help the developers to understand the structure of the code, and can also be used to enforce coding standards. Other than software code, static analysis can also be carried out on things like, static analysis of requirements or static analysis of websites (for example, to assess for proper use of accessibility tags or the following of HTML standards).

Features or characteristics of static analysis tools are:

• To calculate metrics such as cyclomatic complexity or nesting levels (which can help to identify where more testing may be needed due to increased risk).
• To enforce coding standards.
• To analyze structures and dependencies.
• Help in code understanding.
• To identify anomalies or defects in the code.

Static code analysis tools are as follows:

Coding Standards 

Usually a coding standard consists of a set of programming rules, naming conventions and layout specifications. The main advantage of this is that it saves lots of effort. The added advantage of adapting this approach is that if we take a well-known coding standard there will probably be checking tools available that support that standard.

Without such tools the enforcement of coding standard in an organization is likely to fail because the number of rules in the coding standard is so large that nobody can remember them all. Another reason is that if people spend time checking coding standards in reviews that will distract them from other defects that might otherwise, find and makes the review process less effective.

Code Metrics

Code metrics are basically the measurement of the depth of nesting, cyclomatic number and number of lines of code.

This information can be computed not only as the design and code are being created but also during the changes that are made to the system, to see if the design or code is becoming bigger, more complex and more difficult to understand and maintain. The measurement also helps us to decide between several design alternatives.

There are many different types of structural measures. One of them is Cyclomatic complexity metric. The Cyclomatic complexity metrics based on the number of decisions in a program.

It is important to tester because it provides an indication of the amount of testing.

There are many ways to calculate cyclomatic complexity but the easiest way is to sum the number of binary decision statements (e.g. if, while, for, etc.) and add 1 to it.

Complexity is the degree to which a component or system has a design and/or internal structure that is difficult to understand, maintain and verify.

Cyclomatic complexity is the number of independent paths through a program. It is defined as: L-N+2P.

            Where, L= number of edges/ links in the graph.

                        N= number of nodes in the graph.

                        P=number of disconnected parts of the graph.

            Below is a simple program as an example:

                                IF A=354

                                                THEN IF B>C

                                                                THEN A=B

                                                                ELSE A=C

                                                                ENDIF

                                                ENDIF

                                PRINT A

In the program mentioned above has 2 IF conditions. Thus just add 1 to it and the cyclomatic complexity is 2+1=3.

We can also calculate the cyclomatic complexity using the control flow.
In the control flow shown below there are 7 nodes (shapes) and 8 edges (lines). Thus by formula ((no. of edges-no. of nodes) +2) that is (8-7) +2 = 1+2 = 3.

Code Structure 

Code structure tells us about the effort required to write the code in the first place, to understand the code while making the change, or to test the code using particular tools or techniques. There are several aspect of code structure to consider

• Control flow structure: It addresses the sequence in which the instructions are executed.
• Data flow structure: It follows the track of the data item as it is accessed and modified by the code.
• Data structure: It refers to the organization of the data itself, independent of the program.

Now, let’s summarize what we learned in this section.

Summary

•  Static techniques.
• Differences between static and dynamic testing.
• Reviews.
• Phases of reviews.
• Roles and responsibilities of the participants.
• Types of reviews.
• Static analysis by tools.