Manual Testing Tutorial-Part-1

Types of Software

Software components of a computer system have no physical presence, they are stored in digital form within computer memory. There are different categories of software, including system software, utilitiesand applications software. Software can also be described as being either generic or bespoke.

System software is the software used to manage and control the hardware components and which allow interaction between the hardware and the other types of software. The most obvious type of system software is the computer's operating system but device drivers are also included within this category.

Fig 1: a screenshot from a software application

Utility software is software such as anti-virus software, firewalls, disk defragmenters and so on which helps to maintain and protect the computer system but does not directly interface with the hardware.

Applications software (also known as 'apps') are designed to allow the user of the system complete a specific task or set of tasks. They include programs such as web browsers, office software, games and so on. They are usually the reason you bought the computer system in the first place and aren't concerned with the management or maintenance of the system itself.

Any individual software package, whichever of the above types it falls into, can be either generic (or 'off-the-shelf') or it can be bespoke (custom-built). Generic software is mass produced with the intention that it will be used by a wide variety of different users in a range of different situations. Bespoke software is created for a specific purpose which will be used in a known environment.

Often generic software is used when there is a package available that meets the needs of the user. Because it is used by many more people the cost of creating it is spread over a greater number of people or organizations and so the cost to the individual is much lower. It also has the advantage that it is available immediately there's no waiting time involved while the software package is designed, created and tested. Bespoke software has the advantage that it can be tailor made to exact 

Software components of a computer system have no physical presence, they are stored in digital form within computer memory. There are different categories of software, including system software, utilitiesand applications software. Software can also be described as being either generic or bespoke.

System software is the software used to manage and control the hardware components and which allow interaction between the hardware and the other types of software. The most obvious type of system software is the computer's operating system but device drivers are also included within this category.

Utility software is software such as anti-virus software, firewalls, disk defragmenters and so on which helps to maintain and protect the computer system but does not directly interface with the hardware.

Applications software (also known as 'apps') are designed to allow the user of the system complete a specific task or set of tasks. They include programs such as web browsers, office software, games and so on.

Characteristics of a Good Software

While developing any kind of software product, the first question in any developer's mind is, “What are the qualities that a good software should have ?" Well before going into technical characteristics, I would like to state the obvious expectations one has from any software. First and foremost, a software product must meet all the requirements of the customer or end-user. Also, the cost of developing and maintaining the software should be low. The development of software should be completed in the specified time-frame.

Well these were the obvious things which are expected from any project (and software development is a project in itself). Now lets take a look at Software Quality factors. These set of factors can be easily explained by Software Quality Triangle. The three characteristics of good application software are :-
 1)  Operational Characteristics
 2)  Transition Characteristics
 3)  Revision Characteristics

Software Quality Triangle

Software Quality Triangle with characteristics

What Operational Characteristics should a software have ?

These are functionality based factors and related to 'exterior quality' of software. Various Operational Characteristics of software are :

a) Correctness: The software which we are making should meet all the specifications stated by the customer.
b)  Usability/Learnability: The amount of efforts or time required to learn how to use the software should be less. This makes the software user-friendly even for IT-illiterate people.
c)  Integrity: Just like medicines have side-effects, in the same way software may have a side-effect i.e. it may affect the working of another application. But quality software should not have side effects.
d)   Reliability: The software product should not have any defects. Not only this, it shouldn't fail while execution.
e)   Efficiency: This characteristic relates to the way software uses the available resources. The software should make effective use of the storage space and execute command as per desired timing requirements.
f)   Security: With the increase in security threats nowadays, this factor is gaining importance. The software shouldn't have ill effects on data / hardware. Proper measures should be taken to keep data secure from external threats.
g)  Safety: The software should not be hazardous to the environment/life. 

What are the Revision Characteristics of software?

These engineering based factors of the relate to 'interior quality' of the software like efficiency, documentation and structure. These factors should be in-build in any good software. Various Revision Characteristics of software are :-

a) Maintainability: Maintenance of the software should be easy for any kind of user.
b) Flexibility: Changes in the software should be easy to make.
c) Extensibility: It should be easy to increase the functions performed by it.
d) Scalability: It should be very easy to upgrade it for more work(or for more number of users).
e) Testability: Testing the software should be easy.
f) Modularity: Any software is said to made of units and modules which are independent of each other. These modules are then integrated to make the final software. If the software is divided into separate independent parts that can be modified, tested separately, it has high modularity.

Transition Characteristics of the software :

a) Interoperability: Interoperability is the ability of software to exchange information with other applications and make use of information transparently.
b) Reusability: If we are able to use the software code with some modifications for different purpose then we call software to be reusable.
c) Portability: The ability of software to perform same functions across all environments and platforms, demonstrate its portability.

Importance of any of these factors varies from application to application. In systems where human life is at stake, integrity and reliability factors must be given prime importance. In any business related application usability and maintainability are key factors to be considered. Always remember in Software Engineering, quality of software is everything, therefore try to deliver a product which has all these characteristics and qualities. 

 "Quality" represents the properties of products and/or services that are valued by  the consumer

What is software quality?

The quality of software is assessed by a number of variables. These variables can be divided into external and internal quality criteria. External quality is what a user experiences when running the software in its operational mode. Internal quality refers to aspects that are code-dependent, and that are not visible to the end-user. External quality is critical to the user, while internal quality is meaningful to the developer only.

Some quality criteria are objective, and can be measured accordingly. Some quality criteria are subjective, and are therefore captured with more arbitrary measurements.

The table below lists the most obvious software quality criteria, as well as some lesser known.

	User	Developer	Measurable
External quality
features	x		yes
speed	x	x	yes
space	x	x	yes
network usage	x	x	yes
stability	x	x	yes
robustness	x	x	somewhat
ease-of-use	x		subjective
determinism	x	x	yes
back-compatibility	x		yes
security	x		difficult
power consumption	x		difficult
Internal quality
test coverage		x	yes
testability		x	hard
portability		x	somewhat
thread-safeness		x	hard
conciseness		x	somewhat
maintainability		x	hard
documentation		x	subjective
legibility		x	subjective
scalability		x	somewhat

By definition the internal quality (code characteristics) is a concern to the developer only, while all the external quality aspects (coming from using the software) are critical to the end user. However the developer has also interests in performances (speed, space, network usage) and determinism, because they make testing the software easier. Developers treat ease-of-use, back-compatibility, security, and power consumption as requirements.

It is important to consider how difficult it is to measure each of these criteria. It can be difficult because there is no simple variable to look at, or because the measurement process is costly, or because it requires a complex infrastructure. For instance, speed has an objective measurement that is easy to measure. Power consumption has a simple measurement (how many µW the application consumes), but it is complex to measure. Security is difficult and costly to estimate.

Features. This is the very reason for the software to be written: to provide a service. By feature we really mean the output produced by the software –e.g., a numerical result, a string, a screen shot, a web page, an audio, etc–, regardless of the performances (speed, memory).

Speed. How quickly does the application provide the service? The user experiences the actual time elapsed between the moment she request the service, and the moment the service is delivered. The real elapsed time, or wall time, is the sum of the CPU time, system time, and network latency. Thus the developer should not only focus on the CPU time (how much time the CPU actually spends on executing the program). The CPU time can easily be overshadowed by disk access (a write on the disk is very costly), swapping (due to an excessive virtual memory size), or time spent by the network (latency issue, or too many round trips).

Space. How much RAM and disk space is taken by the application? The aggregate numbers are important –peak memory, virtual memory size, etc. But even more so, how often do we move data that triggers a CPU cache miss or a disk write, has a dominant impact on the speed of the application. A mediocre data design can lead to very poor performances.

Network usage. It is a matter of bandwidth and latency. Mismanaging sockets and channels can lead to unnecessary extra time spent in opening and closing sockets, handshakes, and round trips. As for memory, caching techniques can be used to reduce consuming network resources.

Stability. How often does one need to patch the software to fix problems? For the user, this is an inconvenience. For the developer, it means that the code is fragile and might benefit from better testing or partial rewrite.

Robustness. How often does the application stale, freeze, or crash? How tolerant is it to extreme conditions –limited CPU and memory/disk/network resources, corner cases, system failure or unresponsive 3^rd-party resources? This aspect is strongly related to testability and coverage.

Ease-of-use. It can be a very subjective factor, hard to quantify. It includes user documentation, clarity of the error message, management of exceptions, and recovery after failure.

Determinism. Also known as repeatability: does the program produce the very same result given the same input? There are many reasons for which a program can exhibit a non-repeatable behavior. A non-repeatable behavior is confusing and frustrating for the user. This also makes the program very difficult to test and debug. Repeatability is strongly dependent on a good data model design.

Back-compatibility. Can a new version of the application be used with an older version’s data? It is essential to the user, because a new version should not require a costly migration of the existing data.

Security. Who is authorized to access the data? Can the data processed by the application be compromised? This is a crucial aspect of many applications, and it is getting more and more difficult to assess with the dissemination of mobile and web-based software.

Power consumption. It is increasingly important with mobile applications, as a program may have to consider how it manages the device’s power producers and consumers (battery, cores, wireless, screen, audio), and not to rely entirely on the operating system.

Test coverage. What is the proportion of code that is executed by some unit or regression test? This is measured by the number of lines, number of functions, and number of control branches that are exercised by the tests. Usually one expects coverage of at least 85% for any moderately complex application. In practice reaching high coverage can be achieved only if testability is high, which has deep implication on the architecture and development methodology.

Testability. An often overlooked or simply ignored aspect of code development, testability is the ability to trigger any specific line of code or branching condition. Highly testable code requires a discipline of architecture and development that is difficult to find. It very costly to fix poorly testable software, as this requires major redesign. This justifies major investment in software architecture, design, and development methodologies.

Portability. Can the application run on 32 and 64 bits machines? Should it run on a mobile phone? Does it run on multiple OS (e.g., Windows, Linux, Mac OS-X, Solaris, iOS, Android, RIM)? Does it run smoothly on all web browsers (IE, Firefox, Chrome, Safari, Opera)?

Thread-safeness. Is a specific component thread-safe? Can two threads collide on non-atomic operations? Can the application get into a deadlock? As concurrency is still mostly the result of a manual process (there no compiler that automatically parallelizes the code), these questions are critical to ensure the good functioning of a program, as well as its performance –it is not rare to see the a program running slower when two many threads are available, as the cost of synchronization can become dominant.

Conciseness. Also known as compactness. Is there any dead code, or duplicated code? Is the code shared and factorized properly? A compact code usually means faster compilation and smaller binary size. Also compactness naturally leads to fewer bugs, because the number of bugs is historically constant w.r.t. code size.

Industry accepted definitions of quality are “conformance to requirements” (from Philip Crosby) and “fit for use” (from Dr. Joseph Juran and Dr. W. Edwards Deming). These two definitions are not inconsistent.

The Different Views of Quality

Meeting requirements is a producer’s view of quality. This is the view of the organization responsible for the project and processes, and the products and services acquired, developed, and maintained by those processes. Meeting requirements means that the person building the product does so in accordance with the requirements. Requirements can be very complete or they can be simple, but they must be defined in a measurable format, so it can be determined whether they have been met. The producer’s view of quality has these four characteristics:

§  Doing the right thing

§  Doing it the right way

§  Doing it right the first time

§  Doing it on time without exceeding cost

Being fit for use is the customer’s definition. The customer is the end user of the products or services. Fit for use means that the product or service meets the customer’s needs regardless of the product requirements. Of the two definitions of quality, fit for use, is the more important. The customer’s view of quality has these characteristics:

§  Receiving the right product for their use

§  Being satisfied that their needs have been met

§  Meeting their expectations

§  Being treated with integrity, courtesy and respect

In addition to the producer and customer views of quality, the organizational infrastructure also includes a provider and a supplier view.

These views are as follows:

§  Provider view – This is the perspective of the organization that delivers the products and services to the customer.

§  Supplier view – This is the perspective of the organization (that may be external to the producer’s company, such as an independent vendor) that provides either the producer and/or the provider with products and services needed to meet the requirements of the customer.

The Two Quality Gaps
Most Information Technology (IT) groups have two quality gaps: the producer gap and the customer gap as shown in the figure below. The producer gap is the difference between what is specified (the documented requirements and internal standards) versus what is delivered (what is actually built). The customer gap is the difference between what the producers actually delivered versus what the customer wanted.


Closing these two gaps is the responsibility of the quality function. The quality function must first improve the processes to the point where the producer can develop the products according to requirements received and its own internal standards. Closing the producer's gap enables the IT function to provide its customers consistency in what it can produce. This has been referred to as the "McDonald's effect" - at any McDonald's in the world, a Big Mac should taste the same. It doesn't mean that every customer likes the Big Mac or that it meets everyone's needs, but rather, that McDonald's has now produced consistency in its delivered product.


Closing the second gap requires the quality function to understand the true needs of the customer. This can be done by customer surveys, Joint Application Development (JAD) sessions, and more user involvement through the process of building information products. The processes can then be changed to close the customer gap, keeping consistency while producing products and services needed by the customer.

Difference between software Verification and Validation:

Verification	Validation
Are we building the system right?	Are we building the right system?
Verification is the process of evaluating products of a development phase to find out whether they meet the specified requirements.	Validation is the process of evaluating software at the end of the development process to determine whether software meets the customer expectations and requirements.
The objective of Verification is to make sure that the product being develop is as per the requirements and design specifications.	The objective of Validation is to make sure that the product actually meet up the user’s requirements, and check whether the specifications were correct in the first place.
Following activities are involved inVerification: Reviews, Meetings and Inspections.	Following activities are involved inValidation: Testing like black box testing, white box testing, gray box testing etc.
Verification is carried out by QA team to check whether implementation software is as per specification document or not.	Validation is carried out by testing team.
Execution of code is not comes underVerification.	Execution of code is not comes underValidation.
Verification process explains whether the outputs are according to inputs or not.	Validation process describes whether the software is accepted by the user or not.
Verification is carried out before the Validation.	Validation activity is carried out just after the Verification.
Following items are evaluated duringVerification: Plans, Requirement Specifications, Design Specifications, Code, Test Cases etc,	Following item is evaluated duringValidation: Actual product or Software under test.
Cost of errors caught in Verification is less than errors found in Validation.	Cost of errors caught in Validation is more than errors found in Verification.
It is basically manually checking the of documents and files like requirement specifications etc.	It is basically checking of developed program based on the requirement specifications documents & files.

Conclusion on difference of Verification and Validation in software testing:

1.     Both Verification and Validation are essential and balancing to each other.

2.     Different error filters are provided by each of them.

3.     Both are used to finds a defect in different way, Verification is used to identify the errors in requirement specifications & validation is used to find the defects in the implemented Software application.

If you like this article, consider subscribing to get all Testing Updates in your Inbox for FREE. To get testing tips Click Here or enter email id below:

Testing, Quality Assurance and Quality Control

Most people are confused with the concepts and difference between Quality Assurance, Quality Control and Testing. Although they are interrelated and at some level they can be considered as the same activities, but there is indeed a difference between them. Mentioned below are the definitions and differences between them:

S.N.	Quality Assurance	Quality Control	Testing
1	Activities which ensure the implementation of processes, procedures and standards in context to verification of developed software and intended requirements.	Activities which ensure the verification of developed software with respect to documented (or not in some cases) requirements.	Activities which ensure the identification of bugs/error/defects in the Software.
2	Focuses on processes and procedures rather then conducting actual testing on the system.	Focuses on actual testing by executing Software with intend to identify bug/defect through implementation of procedures and process.	Focuses on actual testing.
3	Process oriented activities.	Product oriented activities.	Product oriented activities.
4	Preventive activities.	It is a corrective process.	It is a preventive process.
5	It is a subset of Software Test Life Cycle (STLC).	QC can be considered as the subset of Quality Assurance.	Testing is the subset of Quality Control.