Part 1: Test principles and process

1  Testing Fundamentals 13
    1.1 Test principles and definitions 13
    1.2 Test process 18
    1.3 Testing and the software life cycle 23
    1.4 Test levels 26
    1.5 Traditional versus new 29

2  TMap test process 31
    2.1 Testing as a process 32
    2.2 Test process model 33

3  Testing and standards 39
    3.1 Standards 39
    3.2 Software testing in context 41
    3.3 A software testing model 44
    3.4 Software testing framework 50
    3.5 Final remarks 51

Part 2: Test and risk management

4  Risk based testing 57
    4.1 The bad game 58
    4.2 Understanding necessary quality levels 59
    4.3 Setting priorities in testing 60
    4.4 More effective testing 68
    4.5 Making testing cheaper 70
    4.6 Cutting testing work 72
    4.7 Strategies for prevention 74
    4.8 Priority rules 74

5  Good enough testing 77

6  Measuring Software Quality  81
    6.1 Project context 81
    6.2 Quality model: ISO 9126 82
    6.3 Selection of quality characteristics 83
    6.4 Identification of quality metrics 86
    6.5 Defining completion criteria 87
    6.6 Real-life measurements 88
    6.7 The release decision 90

7  Test Point Analysis: a method for test estimation 93
    7.1 Philosophy 93
    7.2 Basic procedure 96
    7.3 Principles 97
    7.4 The technique in detail 98
    7.5 Test Point Analysis at an early stage 111
    7.6 An example calculation 111

Part 3: Reviews

8  Formal review types 117
    8.1 Document reviews 117
    8.2 Participants, roles and responsibilities 120
    8.3 Inspections 121
    8.4 Technical reviews 129
    8.5 Walkthroughs 133
    8.6 Combining the review types 136
    8.7 Success factors of implementation 139

9  Reading Techniques 143
    9.1 Current practices 143
    9.2 A reading procedure template 145
    9.3 Example 1: The requirements customer perspective 148
    9.4 Example 2: The requirements test perspective 151
    9.5 Customizing reading techniques 153
    9.6 Experiences 161

10  Making inspections work 165
    10.1 Basic definitions 165
    10.2 Benefits and current best practice 166
    10.3 Improving inspections 167

Part 4: Test techniques

11  Static Analysis 181
    11.1 Static fault versus dynamic failure 181
    11.2 When faults cause failures 182
    11.3 Early versus late detection 185
    11.4 Measurement for static analysis 186
    11.5 Coverage: how much is enough? 187
    11.6 Approaches to static analysis 188

12  Testing Techniques; why bother? 197
    12.1 What are techniques? 197
    12.2 An overview 199
    12.3 Are techniques for testers only? 200
    12.4 Why use techniques at all? 201
    12.5 When should techniques be used? 202
    12.6 Software and system modeling 204
    12.7 New technology and techniques 205
    12.8 Advantages / disadvantages 206

13  Exploratory testing 209
    13.1 Introduction 209
    13.2 Practicing exploratory testing 214
    13.3 Where exploratory testing fits 217
    13.4 Exploratory testing in action 218
    13.5 Productivity 220

14  Usability testing 223
    14.1 What is usability? 223
    14.2 Contexts of use 227
    14.3 Designing for usability 229
    14.4 Planning and managing usability testing 230
    14.5 Why is usability testing different? 233
    14.6 Designing usability tests 237
    14.7 Early life cycle techniques 238
    14.8 Late life cycle techniques 240
    14.9 Additional sources of information 245

15  Performance Testing 249
    15.1 Performance testing 249
    15.2 Motivation 251
    15.3 Conducting a performance test 252
    15.4 Risk management 257
    15.5 Experiences from real life 259

Part 5: Incident management

16  The Bug Reporting Processes 263
    16.1 Definitions 263
    16.2 IEEE 1044 bug process 264
    16.3 Bug reporting process 265
    16.4 A hypothetical case study 268
    16.5 Classification: beyond the failure description 272
    16.6 Process quality indicators 276
    16.7 Handling challenges 280
    16.8 Implementing changes 284

Part 6: Test Process Improvement

17  Testing Maturity Model 289
    17.1 History and background 289
    17.2 TMM maturity levels 290
    17.3 TMM structure 294
    17.4 TMM Level 2: Definition 295
    17.5 From detection to prevention 299

18  Test Process Improvement 301
    18.1 How good is your test process? 301
    18.2 The model 302
    18.3 The improvement process 311
    18.4 Conclusions and remarks 313

Part 7: Test tools

19  Test tool overview 317
    19.1 A closer look 317
    19.2 Advantages 319
    19.3 Considerations 320
    19.4 Test tool overview 321

20  Tool evaluation and selection 327
    20.1 Do you really need a tool? 327
    20.2 The need for a formal evaluation 328
    20.3 Identify and document requirements 329
    20.4 Conduct market research 329
    20.5 Organise supplier presentations 330
    20.6 Formally evaluating the test tool 331
    20.7 Post evaluation activities 332

21  Test execution automation 335
    21.1 Advantages 335
    21.2 Record and playback 336
    21.3 Test programming 338
    21.4 Data driven tests 339
    21.5 Testing with action words 341
    21.6 Consequences 345

Part 8: People

22  Team Building 351
    22.1 Steaming along or just plodding? 351
    22.2 What makes a successful team? 352
    22.3 People in a team 355
    22.4 How to build a more effective team? 356
    22.5 Management 356
    22.6 Retaining test expertise 359
    22.7 Your career belongs to you! 362
    22.8 How do we spot a good tester? 362
    22.9 Communications 364
    22.10  The end game 369

23  Test career paths 371
    23.1 Three dimensions 371
    23.2 The test career cube 373

Annexes

Annex A. Testing standards assessment 377
Annex B. ISO 9126 Quality Characteristics 381
Annex C. Review and inspection forms 385
Annex D. SUMI Case studies 391
Annex E. Tool Evaluation Criteria 401

Literature 467
Authors 423
Improve Quality Services 485
Glossary 487
Index 521