There are 3 principle
documents in Testing:
1) Test Policy
2) Test strategy
3) Test Plan
1.
Test Policy (Organization Level)
Test Policy Document
What is a Test Policy Document?
A Test Policy is a high
level document and is at the top of the hierarchy of the Test Documentation
structure. The purpose of the Test Policy document is to represent the testing
philosophy of the company as a whole and to provide a direction which the testing
department should adhere to and follow. It should apply to both new projects
and maintenance work.
Setting an appropriate
test policy by senior managers, provides a robust framework within which
testing practitioners can then operate. This will help to ensure the
maximisation of the strategic value inherent in every project.
Contents of a Test Policy
1. Definition of Testing
Organisations need to be clear why they are testing. This will influence the remainder of the policy document and also the detailed testing techniques that are selected by test managers at the programme and project level.
From the understanding of why testing is required it is possible to specify what the purpose of testing is within the organisation. Without this fundamental linkage the test effort is destined to fail.
Organisations need to be clear why they are testing. This will influence the remainder of the policy document and also the detailed testing techniques that are selected by test managers at the programme and project level.
From the understanding of why testing is required it is possible to specify what the purpose of testing is within the organisation. Without this fundamental linkage the test effort is destined to fail.
Example: “ensuring the software fulfills its requirements”
2. Description of the test process
It is vital to establish a solid view towards the test process. We should address questions like, which phases and subtasks will the test process include. Which roles will be involved and the document structure associated with each tasks, as well as what test levels need to be considered.
It is vital to establish a solid view towards the test process. We should address questions like, which phases and subtasks will the test process include. Which roles will be involved and the document structure associated with each tasks, as well as what test levels need to be considered.
Example: “all test plans are written in accordance with company
policy”
3. Test Evaluation:
How are we going to evaluate the results of testing, what measures will we use to ensure test effectiveness in the project?
How are we going to evaluate the results of testing, what measures will we use to ensure test effectiveness in the project?
Example: “effect on business of finding a fault after its release”
4. Quality Level to be achieved:
Which quality criteria are going to be tested and which quality level is the system required to achieve prior to its release with regards to these criteria?
Which quality criteria are going to be tested and which quality level is the system required to achieve prior to its release with regards to these criteria?
Example: “no outstanding high severity faults prior to products
release”
5. Approach to Test Process Improvement
How often and when are we going to assess the usefulness of the current processes in place and what elements need improving and techniques that shall be used to improve the processes.
How often and when are we going to assess the usefulness of the current processes in place and what elements need improving and techniques that shall be used to improve the processes.
Example: “project review meetings to be held after project
completion”
2)
Test Strategy:
It is a company level or Programme Level document and developed by QA category people like QA and PM. This document defines "Testing Approach" to achieve testing objective. Test strategy is the freezed part of BRS (Business requirement specification) from which we get Test Policy and Test Strategy.
Components in the Test Strategy are as follows:
It is a company level or Programme Level document and developed by QA category people like QA and PM. This document defines "Testing Approach" to achieve testing objective. Test strategy is the freezed part of BRS (Business requirement specification) from which we get Test Policy and Test Strategy.
Components in the Test Strategy are as follows:
1. Applicable for a
Programme/System which covers multiple projects
2. Scope and objective of
Testing : The purpose of testing in an organization
3. In scope/Out of scope
Items for testing
4. Business issues : Budget
control for testing
5. Roles and
responsibilities : Names of jobs in the testing team and their responsibilities
6. Communication and status
reporting : Required negotiation between two jobs in a team
7. Test Levels
(Unit/Module/System/Integration)
8. Test Types (
Functional/Non-Functional)
9. Entry/Exit/Stop/Resumption
Criteria for testing (for different Levels/Phases)
10. Test Environment
11. Test case design
methodology (Specification driven/BVA/EQ Partition)
12. Test methodology (
Top-down/bottom-up/risk based)
13. Test deliverable
14. Test approach : That is
the TRM ( Test responsibility Matrix)
15. Test automation Approach
and tools : Availability of testing tools and purpose of automation
16. Testing measurements and
metrices
17. Test Tools to be used
18. Defect Management
approach : required negotiation between testing and development teams
19. Defect classification
20. Retesting &
regression approach
21. Risks to be addressed
22. Risks and mitigation :
Expected failures during testing and solutions to overcome
23. Change and configuration
management: How to handle sudden changes in customer requirements.
24. Training plan
3) Test Plan:
Test plan is a Project Level document. Test plan is the freezed document developed from SRS (Software requirement specification), FS (Functional specification), and UC (use case). After completion of testing team formation and risk analysis, Test Lead is preparing test plan document in term of:
1) What to test?
2) How to test ?
3) When to test ?
4) Who to test?
2) How to test ?
3) When to test ?
4) Who to test?
There
is one Master Test Plan consists of reviewed Project
Test Plan and Phase Test Plan. So there is general
talk about Project Test Plan.
Components are as follows:
1 ) Test plan ID : Unique number or name
2) Introduction : About project
3) Test Items: Names of all modules in the project.
4) Features to be tested:
5) Features not to be tested:
6) Testing approach: finalized TRM, selected testing techniques
7) Entry Criteria : When testing can be started
8) Exit Criteria : when testing can be stopped
9)Fail or Pass criteria:
10) Test Environment : The environment where the test is to be carried out.
11) Test Deliverable:
12) Staff and training needs
13) Roles and Responsibilities
14 ) Project Schedule : starting and End dates
15) Risks and mitigation.
Components are as follows:
1 ) Test plan ID : Unique number or name
2) Introduction : About project
3) Test Items: Names of all modules in the project.
4) Features to be tested:
5) Features not to be tested:
6) Testing approach: finalized TRM, selected testing techniques
7) Entry Criteria : When testing can be started
8) Exit Criteria : when testing can be stopped
9)Fail or Pass criteria:
10) Test Environment : The environment where the test is to be carried out.
11) Test Deliverable:
12) Staff and training needs
13) Roles and Responsibilities
14 ) Project Schedule : starting and End dates
15) Risks and mitigation.
16) Approach
Test
strategy VS test plan (Summary):-
The test strategy is a road map which
tells how you are going to address testing for the project from the start to
end. Some companies have a strategy or approach section in the test plan,
others have a separate document.
The
test plan will detailed that out because it’s written at a later stage of the
project. IT describe what to test?, how to test ? , when to test?
Who to test?
A Test Plan describes the
approach, Features to be tested, Testers assigned, and whatever you plan for
your project.
In
other way the following points must be covered in some way either in testing
reference documents:
•Vision -> vision, goals, objectives
•Mission -> Organization's intention, strategy, tactics, policies, rules
•Baseline -> baselines (SWOT), maturity, capability, influencers (roles)
•Requirements -> constraints, assumptions, out of scope, dependencies, risks, issues, problems, incidents, defects
•Governance -> decision making, follow up, communication plan, meeting plan
•Resources -> QMS (processes, activities, tasks, templates, guidelines, checklist, standards, glossaries, tools, ...)
•Practice -> pragmatic approach (who does what, when , where and how)
•Improve -> problem solving, defect solving (how do we handle problems, defects)
•Measure -> definition of what we will measure and why
•Track-> based on measurement we define what we track and how we can pave the track (road map)
•Guide-> training, mentoring, coaching, e-learning, handover, ...
•KPI-> uniform management information gathering. GQM - each goal from the vision part raises a number of questions that lead to measurements. A KPI is a function of all these measurement (sum, average, footprint (radar), etc.)
•End Result->Final conclusions, GO/NOGO criteria/reporting, lessons learned
•Vision -> vision, goals, objectives
•Mission -> Organization's intention, strategy, tactics, policies, rules
•Baseline -> baselines (SWOT), maturity, capability, influencers (roles)
•Requirements -> constraints, assumptions, out of scope, dependencies, risks, issues, problems, incidents, defects
•Governance -> decision making, follow up, communication plan, meeting plan
•Resources -> QMS (processes, activities, tasks, templates, guidelines, checklist, standards, glossaries, tools, ...)
•Practice -> pragmatic approach (who does what, when , where and how)
•Improve -> problem solving, defect solving (how do we handle problems, defects)
•Measure -> definition of what we will measure and why
•Track-> based on measurement we define what we track and how we can pave the track (road map)
•Guide-> training, mentoring, coaching, e-learning, handover, ...
•KPI-> uniform management information gathering. GQM - each goal from the vision part raises a number of questions that lead to measurements. A KPI is a function of all these measurement (sum, average, footprint (radar), etc.)
•End Result->Final conclusions, GO/NOGO criteria/reporting, lessons learned
Challenge of software testing
1) Testing the
complete application:
Is it possible? I think impossible. There are millions of test combinations. It’s not possible to test each and every combination both in manual as well as in automation testing. If you try all these combinations you will never ship the product
Is it possible? I think impossible. There are millions of test combinations. It’s not possible to test each and every combination both in manual as well as in automation testing. If you try all these combinations you will never ship the product
2) Misunderstanding
of company processes:
Sometimes you just don’t pay proper attention what the company-defined processes are and these are for what purposes. There are some myths in testers that they should only go with company processes even these processes are not applicable for their current testing scenario. This results in incomplete and inappropriate application testing.
Sometimes you just don’t pay proper attention what the company-defined processes are and these are for what purposes. There are some myths in testers that they should only go with company processes even these processes are not applicable for their current testing scenario. This results in incomplete and inappropriate application testing.
3) Relationship
with developers:
Big challenge. Requires very skilled tester to handle this relation positively and even by completing the work in testers way. There are simply hundreds of excuses developers or testers can make when they are not agree with some points. For this tester also requires good, troubleshooting and analyzing skill.
Big challenge. Requires very skilled tester to handle this relation positively and even by completing the work in testers way. There are simply hundreds of excuses developers or testers can make when they are not agree with some points. For this tester also requires good, troubleshooting and analyzing skill.
4) Regression
testing:
When project goes on expanding the regression testing work simply becomes uncontrolled. Pressure to handle the current functionality changes, previous working functionality checks and bug tracking.
When project goes on expanding the regression testing work simply becomes uncontrolled. Pressure to handle the current functionality changes, previous working functionality checks and bug tracking.
5) Lack of skilled testers:
I will call this as ‘wrong management decision’ while selecting or training testers for their project task in hand. These unskilled fellows may add more chaos than simplifying the testing work. This results into incomplete, insufficient and ad-hoc testing throughout the testing.
I will call this as ‘wrong management decision’ while selecting or training testers for their project task in hand. These unskilled fellows may add more chaos than simplifying the testing work. This results into incomplete, insufficient and ad-hoc testing throughout the testing.
6) Testing always under time
constraint:
Hey tester, we want to ship this product by this weekend, are you ready for completion? When this order comes from boss, tester simply focuses on task completion and not on the test coverage and quality of work. There is huge list of tasks that you need to complete within specified time. This includes writing, executing, automating and reviewing the test cases.
Hey tester, we want to ship this product by this weekend, are you ready for completion? When this order comes from boss, tester simply focuses on task completion and not on the test coverage and quality of work. There is huge list of tasks that you need to complete within specified time. This includes writing, executing, automating and reviewing the test cases.
7) Which tests to
execute first?
If you are facing the challenge stated in point no 6, then how will you take decision which test cases should be executed and with what priority? Which tests are important over others? This requires good experience to work under pressure.
If you are facing the challenge stated in point no 6, then how will you take decision which test cases should be executed and with what priority? Which tests are important over others? This requires good experience to work under pressure.
8 ) Understanding
the requirements:
Sometimes testers are responsible for communicating with customers for understanding the requirements. What if tester fails to understand the requirements? Will he be able to test the application properly? Definitely No! Testers require good listening and understanding capabilities.
Sometimes testers are responsible for communicating with customers for understanding the requirements. What if tester fails to understand the requirements? Will he be able to test the application properly? Definitely No! Testers require good listening and understanding capabilities.
9) Automation testing:
Many sub challenges – Should automate the testing work? Till what level automation should be done? Do you have sufficient and skilled resources for automation? Is time permissible for automating the test cases? Decision of automation or manual testing will need to address the pros and cons of each process.
Many sub challenges – Should automate the testing work? Till what level automation should be done? Do you have sufficient and skilled resources for automation? Is time permissible for automating the test cases? Decision of automation or manual testing will need to address the pros and cons of each process.
10) Decision to
stop the testing:
When to stop testing? Very difficult decision. Requires core judgment of testing processes and importance of each process. Also requires ‘on the fly’ decision ability.
When to stop testing? Very difficult decision. Requires core judgment of testing processes and importance of each process. Also requires ‘on the fly’ decision ability.
11) One test team
under multiple projects:
Challenging to keep track of each task. Communication challenges. Many times results in failure of one or both the projects.
Challenging to keep track of each task. Communication challenges. Many times results in failure of one or both the projects.
12) Reuse of Test
scripts:
Application development methods are changing rapidly, making it difficult to manage the test tools and test scripts. Test script migration or reuse is very essential but difficult task.
Application development methods are changing rapidly, making it difficult to manage the test tools and test scripts. Test script migration or reuse is very essential but difficult task.
13) Testers
focusing on finding easy bugs:
If organization is rewarding testers based on number of bugs (very bad approach to judge testers performance) then some testers only concentrate on finding easy bugs those don’t require deep understanding and testing. A hard or subtle bug remains unnoticed in such testing approach.
If organization is rewarding testers based on number of bugs (very bad approach to judge testers performance) then some testers only concentrate on finding easy bugs those don’t require deep understanding and testing. A hard or subtle bug remains unnoticed in such testing approach.
14) To cope with
attrition:
Increasing salaries and benefits making many employees leave the company at very short career intervals. Managements are facing hard problems to cope with attrition rate. Challenges – New testers require project training from the beginning, complex projects are difficult to understand, delay in shipping date!
Increasing salaries and benefits making many employees leave the company at very short career intervals. Managements are facing hard problems to cope with attrition rate. Challenges – New testers require project training from the beginning, complex projects are difficult to understand, delay in shipping date!
What
is difference between Error, Defect, Failure
Error: A human mistake or issue occurred in program before or
during Compilation of program.
Ex.
If (a!=b)
{
c = a/b(;)
}
If
programmer missed the semicolon at the end of 3rd line of code and tries to
compile, then an error message is occurred. This type of mistake called ERROR.
Defects: When an issue, unexpected result or deviation in actual
functionality is found out by a moderator (Not an Author of code) during any
phase (At during development, Beta version or at production sites) of testing
is called Defect.
Ex. Author used the variable ‘d’ instead of variable ‘b’ or use divide symbol instead of plus symbol, due to this there is no compilation error
is occurred but it produce some unaccepted result but here issue is found out
by Moderator.
Failure: It is a state which rose due to a fault and affected a
single part of functionality.
Ex. Due to this Fault state, any other single line code or
single module is affected, this state is called Failure.
No comments:
Post a Comment