Contents
Preface 5
Recommendations 13
GENERAL
1 Introduction 23
1.1 The history of TMap 24
1.2 TMap evolves in step 25
1.3 What TMap offers 28
1.3.1 Where TMap helps 28
1.3.2 Where TMap can be applied 29
1.4 Reading guide and the most important changes 30
1.4.1 Structure of the book 30
1.4.2 Reading guide 31
1.4.3 The most important
changes 33
2 Framework
and importance of testing 35
2.1 What is testing? 35
2.2 Why test? 38
2.3 The role of testing 40
2.3.1 Testing and quality management 40
2.3.2 Testing: how and by whom 41
2.3.3 Test and system development process 43
2.3.4 Test levels and responsibilities 46
2.3.5 Test types 50
2.4 What is structured testing? 51
3 The
essentials of TMap 55
3.1 Business driven explained 56
3.2 Structured test process 61
3.2.1 Process: master test plan, managing the
total test process 62
3.2.2 Process: acceptance and system tests 63
3.2.3 Process: development tests 67
3.3 Complete tool box 69
3.3.1 Techniques 69
3.3.2 Infrastructure 73
3.3.3 Organisation 74
3.4 Adaptive and complete method 77
3.4.1 Respond to changes 78
3.4.2 (Re)use products
and processes 78
3.4.3 Learn from experience 78
3.4.4 Try before use 79
PROCESSES
4 Introduction
to the processes 81
4.1 Structure and contents of the process
chapters 81
4.2 Chapters 5, 6 and 7: the TMap phases 84
4.3 Chapter 8: supporting processes 85
5 Master
test plan, managing the total test process 87
5.1 Introduction 87
5.2 Planning phase of the total test process 89
5.2.1 Establishing the assignment 96
5.2.2 Understanding the assignment 100
5.2.3 Analysing the product risks 102
5.2.4 Determining the test strategy 104
5.2.5 Estimating the effort 111
5.2.6 Determining the planning 113
5.2.7 Defining the test products 116
5.2.8 Defining the organisation 118
5.2.9 Defining the infrastructure 126
5.2.10 Organising the management 129
5.2.11 Determining test process risks (&
countermeasures) 132
5.2.12 Feedback and consolidation of the plan 133
5.3 Control phase of the total test process 135
5.3.1 Management 137
5.3.2 Monitoring 138
5.3.3 Reporting 142
5.3.4 Adjusting 146
5.4 Generic Test Agreements 148
6 Acceptance
and system tests 151
6.1 Introduction 151
6.2 Planning phase 155
6.2.1 Establishing the assignment 159
6.2.2 Understanding the assignment 165
6.2.3 Determining the test basis 170
6.2.4 Analysing the product risks 172
6.2.5 Determining the test strategy 174
6.2.6 Estimating the effort 179
6.2.7 Determining the planning 182
6.2.8 Allocating test units and test techniques 189
6.2.9 Defining the test products 196
6.2.10 Defining the organisation 199
6.2.11 Defining the infrastructure 209
6.2.12 Organising the management 212
6.2.13 Determining test process risks (&
countermeasures) 219
6.2.14 Feedback and consolidation of the plan 221
6.3 Control Phase 224
6.3.1 Management 226
6.3.2 Monitoring 228
6.3.3 Reporting 237
6.3.4 Adjusting 249
6.4 Setting up and maintaining infrastructure phase 251
6.4.1 Specifying the infrastructure 258
6.4.2 Realising the infrastructure 261
6.4.3 Specifying the infrastructure intake 263
6.4.4 Intake of the infrastructure 265
6.4.5 Maintaining the infrastructure 266
6.4.6 Preserving the infrastructure 269
6.5 Preparation phase 271
6.5.1 Collection of the test basis 275
6.5.2 Creating checklists 280
6.5.3 Assessing the test basis 282
6.5.4 Creating the testability review report 283
6.6 Specification Phase 285
6.6.1 Creating test specifications 287
6.6.2 Defining central starting point(s) 295
6.6.3 Specifying the test object intake 305
6.7 Execution Phase 307
6.7.1 Intake of the test object 309
6.7.2 Preparing the starting points 312
6.7.3 Executing the (re)tests 314
6.7.4 Checking and assessing the test results 318
6.8 Completion Phase 322
6.8.1 Evaluating the test process 324
6.8.2 Preserving the testware 326
7 Development
tests 329
7.1 Introduction 329
7.2 Development testing explained 330
7.2.1 What is development
testing? 330
7.2.2 Characteristics 331
7.2.3 Advantages and disadvantages of improved
development tests 332
7.2.4 Context of development testing 334
7.2.5 Unit test 337
7.2.6 Unit integration test 338
7.2.7 Quality measures 339
7.2.8 Test tools for development tests 350
7.3 Test activities 352
7.3.1 Planning phase 353
7.3.2 Control phase 361
7.3.3 Setting up and maintaining infrastructure
phase 362
7.3.4 Preparation phase 363
7.3.5 Specification phase 364
7.3.6 Execution phase 366
7.3.7 Completion phase 367
8 Supporting
processes 369
8.1 Introduction 369
8.2 Test policy 370
8.3 Permanent test organisation 373
8.3.1 Introduction 373
8.3.2 Permanent test organisation explained 373
8.3.3 Benefits, conditions and points of concern 374
8.3.4 Supplying test services 379
8.3.5 General process model 386
8.3.6 Two common types of test organisation 389
8.3.7 Test Expertise Centre (TEC) 390
8.3.8 Test Factory (TF) 395
8.3.9 Role of a permanent test organisation in
outsourcing 400
8.3.10 Setting up a test organisation 401
8.4 Test environments 406
8.4.1 Introduction 406
8.4.2 Test environments explained 406
8.4.3 Setting up test environments 408
8.4.4 Problems in test environments 411
8.4.5 DTAP model 412
8.4.6 Processes in test environments 416
8.4.7 Two special test environments 418
8.4.8 Test environments when outsourcing 420
8.4.9 Setting up and maintaining test environments
as a service 421
8.5 Test tools 429
8.5.1 Introduction 429
8.5.2 Test tools explained 430
8.5.3 Types of test tools 431
8.5.4 Advantages of using test tools 440
8.5.5 Implementing test tools with a tool policy 442
8.5.6 Initiation phase 444
8.5.7 Realisation phase 446
8.5.8 Operation phase 451
8.6 Test professionals 453
8.6.1 Introduction 453
8.6.2 Points of concern 453
8.6.3 Characteristics 455
8.6.4 Career path 457
8.6.5 Positions 460
8.6.6 Training 468
COMPONENTS
9 Product
risk analysis 471
9.1 Introduction 471
9.2 Approach 473
9.3 Determining participants 475
9.4 Determining the PRA approach 476
9.4.1 Organisation of the PRA 476
9.4.2 Determining risk classification method 479
9.5 Preparing session/interviews 482
9.6 Collecting and analysing product risks 483
9.7 Completeness check 493
10 Quality
characteristics and test types 495
10.1 Introduction 495
10.2 Quality characteristics 495
10.3 Test types 501
10.3.1 Regression 501
10.3.2 Usability 503
10.3.3 Performance 508
10.3.4 Portability 513
10.3.5 Information security 515
11 Estimation
techniques 521
11.1 Estimating 521
11.2 Estimation based on ratios 525
11.3 Estimation based on test object size 526
11.4 Work Breakdown Structure 528
11.5 Evaluation estimation approach 529
11.6 Proportionate estimation 530
11.7 Extrapolation 531
11.8 Test point analysis 531
11.8.1 Input and starting conditions 536
11.8.2 Dynamic test points 538
11.8.3 Static test points 544
11.8.4 Total number of test points 544
11.8.5 Primary test hours 545
11.8.6 The total number of
test hours 548
11.8.7 Distribution over the phases 551
11.8.8 TPA at an early stage 552
12 Defects
management 553
12.1 Introduction 553
12.2 Finding a defect 554
12.3 Defect report 561
12.4 Procedure 566
13 Metrics 569
13.1 Introduction 569
13.2 GQM method in six steps 570
13.3 Hints and tips 573
13.4 Practical starting set of test metrics 573
13.5 Metrics list 576
14 Test
design techniques 579
14.1 Introduction 579
14.2 Essential test design concepts 581
14.2.1 Test situation, test case and test script 581
14.2.2 Coverage, coverage type and coverage ratio 587
14.2.3 Test design technique and basic technique 593
14.3 Coverage types and basic techniques 595
14.3.1 Introduction 595
14.3.2 Paths 598
14.3.3 Decision points 602
14.3.4 Equivalence classes 611
14.3.5 Orthogonal arrays and
pairwise testing 612
14.3.6 Boundary value analysis 623
14.3.7 CRUD 625
14.3.8 Statistical usage: Operational profiles and
Load profiles 627
14.3.9 Right paths / Fault paths 632
14.3.10 Checklist 634
14.4 A basic set of test design techniques 635
14.4.1 Introduction 635
14.4.2 Decision Table Test (DTT) 639
14.4.3 Data Combination Test (DCoT) 648
14.4.4 Elementary Comparison Test (ECT) 654
14.4.5 Error Guessing (EG) 661
14.4.6 Exploratory Testing (ET) 664
14.4.7 Data Cycle Test (DCyT) 670
14.4.8 Process cycle test (PCT) 675
14.4.9 Real-Life Test (RLT) 681
14.4.10 Semantic
Test (SEM) 687
14.4.11 Syntactic
Test (SYN) 690
14.4.12 Use
Case Test (UCT) 696
15 Evaluation
techniques 705
15.1 Introduction 705
15.2 Evaluation explained 706
15.3 Inspections 710
15.4 Reviews 713
15.5 Walkthroughs 715
15.6 Evaluation technique selection matrix 717
16 Test
roles 719
16.1 Introduction 719
16.2 Roles that are described as a position 720
16.3 Roles not described as a position 720
Glossary 729
References 743
About Sogeti 747
Index 749