End-to-end testing with TMap Next^Ž - Contents

Management summary 9
Preface from the chairman 13
Preface from the authors 15

1 Introduction 17
1.1 Why an approach to end-to-end testing? 17
1.2 Evolution of the approach 18
1.3 Reading guide 18

2 Terms and starting organisation 21
2.1 Terms relating to chains 21
2.2 The term end-to-end test 23
2.3 Terms relating to the end-to-end test: the SIT test types 24
     2.3.1 Interface evaluation 25
     2.3.2 Interface test 26
     2.3.3 SIT test types and the V-model 27
2.4 Starting organisation 28

3 Overview of the end-to-end test process 29
3.1 Overview of end-to-end testing 29
3.2 Adaptive 30

4 Integral end-to-end test plan 31
4.1 Inventorying processes and systems 33
4.2 Analysing chain risks and determining feasibility 36
     4.2.1 Analysing chain risks 36
     4.2.2 Determining feasibility 39
4.3 Determining the scope, sub-chains, strategy and
test environment 43
4.4 Planning and estimating 48
4.5 Creating the IETP 50
     4.5.1 Defining the infrastructure 50
     4.5.2 Defining the organisation and setting up the management 53
     4.5.3 Determining the project risks of the end-to-end test as well as appropriate measures 55
     4.5.4 Finalising the integral end-to-end test plan 56
4.6 Go/No Go integral end-to-end test plan 58
4.7 Management 59

5 Sub end-to-end test plan 61
5.1 Inventorying the dataflows 63
     5.1.1 Detailing the process and system landscape 64
     5.1.2 Establishing dataflows 66
5.2 Determining limitations and measures 68
     5.2.1 Determining limitations 68
     5.2.2 Determining the measures for the limitations 69
5.3 Determining the strategy, planning and estimate 70
     5.3.1 Determining the sub end-to-end test strategy 70
     5.3.2 Determining the planning 72
     5.3.3 Creating the estimate 74
5.4 Creating the SETP 75
     5.4.1 Establishing the assignment 75
     5.4.2 Defining the test products 76
     5.4.3 Defining the organisation 77
     5.4.4 Defining the infrastructure 80
     5.4.5 Organising the management 82
     5.4.6 Determining the test project risks and measures 83
     5.4.7 Finalising the sub end-to-end test plan 83
5.5 Go/No Go sub end-to-end test plan 84
5.6 Management 85

6 Setting up the test infrastructure 87
6.1 Specifying test infrastructure build 89
     6.1.1 Establishing the process and system landscape 90
     6.1.2 Collecting, inspecting and validating the requirements 91
     6.1.3 Collecting the acceptance criteria 93
     6.1.4 Determining bottlenecks and measures 94
     6.1.5 Creating the building planning and estimate 95
     6.1.6 Creating order forms 97
     6.1.7 Creating the building assignment 98
     6.1.8 Compiling the management section and management agreements 98
     6.1.9 Describing the intake procedure for the end-to-end test environment 100
     6.1.10 Recording preconditions and assumptions 101
     6.1.11 Determining critical success factors & critical paths during building activities 102
6.2 Specifying the setup 103
     6.2.1 Specifying the setup assignment 103
     6.2.2 Creating a step-by-step plan 104
     6.2.3 Creating the planning and estimate 105
     6.2.4 Describing the acceptance procedure for the end-to-end test environment 106
6.3 Creating the infrastructure design document 107
6.4 Go/No Go infrastructure design document 108
6.5 Realising the end-to-end test infrastructure 108
6.6 End-to-end test infrastructure intake 110
     6.6.1 Completeness check 111
     6.6.2 Executing a pretest 112
     6.6.3 Reporting defects 112
     6.6.4 Retesting after fixing 113
     6.6.5 Creating the intake report 113

7 End-to-end test execution 115
7.1 Preparation 117
     7.1.1 Inventorying the test basis 118
     7.1.2 Examining test basis coherence 119
     7.1.3 Completing the test basis 120
     7.1.4 Assessing the testability of the test basis 120
     7.1.5 Finalising the test basis 121
7.2 Specifying logical end-to-end test cases 121
     7.2.1 Selecting existing logical test cases 128
     7.2.2 Limiting the number of logical test cases 129
     7.2.3 Specifying additional logical test cases 130
     7.2.4 Interlinking logical test cases into a logical end-to-end test case 131
     7.2.5 Evaluating logical end-to-end test cases and processes 132
7.3 Filling basic files and creating physical end-to-end test cases from logical ones 134
     7.3.1 Filling master files 135
     7.3.2 Setting up environmental data 135
     7.3.3 Realising the initial situation 136
     7.3.4 Creating physical end-to-end test cases from logical ones 138
7.4 Creating the end-to-end test scenario 141
7.5 Go/No Go end-to-end test scenario 144
7.6 Executing the end-to-end test 144
7.7 Completing the end-to-end test 145

8 Roles 147
8.1 End-to-end test roles 147
8.2 Other test roles 149
8.3 Other roles, non test-specific 150

9 Techniques 151
9.1 Chain Role Play 151
9.2 Chain Risk Analysis (CRA) 153

10 Aspects of the test infrastructure 157
10.1 Aspects of the test environment 157
     10.1.1 Depersonalisation carries obligations 157
     10.1.2 Testing in the production environment 159
     10.1.3 Middleware, enterprise service bus and interfaces 160
10.2 Tools in an end-to-end test environment 163

Annex A: The ideal chain organisation 165
Annex B: Chain quality characteristics 169
Annex C: Chain risk overview 171
Annex D: Sources 173

Sogeti Group 175