Acceptance test A test executed by the user(s) and manager(s) in an environment simulating the operational environment to the greatest possible extent, that should demonstrate that the developed system meets the functional and quality requirements. Base Cluster arrangement See: Base Clusters. Base Clusters The base Clusters provide an improvement path (of 13 Clusters) from the Initial level up to fully Optimizing in which no single business driver is relevant or leading: the improvement of the test process is regarded as a general improvement process. BDTM Business Driven Test Management is aimed at enabling the client to manage the test process on rational and economic grounds. Important BDTM aspects are: result, risk, time and cost. BDTPI Business Driven Test Process Improvement is a frame of reference for determining the maturity of an organization’s test process and for setting up and implementing specific and realistic measures, based on the business-drivers, for the improvement of this test process. Business case Provides the economic justification for the project and answers the questions: why do we do this project, which investments are needed, what does the client wish to achieve with the result? Business-driven Clusters The business-driven Clusters are an adapted version of the base Clusters to suit a specific business situation or particular business driver. By first acknowledging which Key areas are more or less relevant to the required bias (e.g. cost, time, quality as a business driver), and secondly by determining how this influences the distribution of the Checkpoints over the Clusters, new Clusters are specified attuned to a specific business driver. In this situation a Cluster may exceed the borders of a Maturity level and contain Checkpoints of multiple Maturity levels. Business-driven Clusters do not change the position of individual Checkpoints in the Test maturity matrix as the position of Checkpoints is fixed. Business driver A management directive, usually a direct derivative of the organization’s vision and/or business strategy, which desires specific outcomes of the organization at an operational level. It is a reason, motivator or challenge for test process improvement, commonly indicated as (a combination of) result, risk, time and cost. Checkpoint A Checkpoint is the measuring unit of the (TPI) model. A Checkpoint is phrased as a statement that can be confirmed with a ‘yes’ or a ‘no’. Fulfilling a Checkpoint means that the answer for a specific test process is ‘yes’, with sufficient proof available to substantiate it. A Checkpoint always relates to one Key area and one Maturity level. Cluster A Cluster is a group of Checkpoints from multiple Key areas that function as one improvement step. Clusters are used for the purpose of increasing the maturity of the test process. Each Cluster is identified by an alphabetic letter that identifies its position in the improvement path, where Cluster ‘A’ is the first improvement step. CMMI Capability Maturity Model Integration (CMMI) in software engineering and organizational development is a process improvement approach that provides organizations with the essential elements for effective process improvement. It can be used to guide process improvement. Controlled level The second level of maturity (after Initial) in the TPI model. It can be expressed as “doing the right things”. Coverage The ratio between that which can be tested and that which is tested with the test set. Defect Any kind of difference between the actual behavior of the test object and the expected behavior. Defect management The process of recognizing, investigating, taking action and disposing of defects. It involves recording defects, classifying them and identifying the impact. Dynamic testing Testing by execution of the test object and/or the running of software. Effectiveness The degree to which the information system meets the demands of the organization and the profile of the end users for whom it is intended, as well as the degree to which the information system contributes to the achievement of business goals. Efficiency The relationship between the performance level of the system (expressed in the transaction volume and overall speed) and the amount of resources (CPU cycles, I/O time, memory and network capacity, etc.) that are used. Efficient level The third level of maturity (after Initial and Controlled) in the TPI model. It can be expressed as “Doing things the right way”. Enabler Enablers in the Business Driven TPI model connect Key areas with aspects of the Software Development Life Cycle (SDLC) in order to keep test process improvements aligned with other processes from the SDLC. End-to-end test The dynamic test intended to demonstrate that the consecutive series of systems supports the (business) process according to specifications. Error Human mistake; this action takes place prior to any defects/faults and/or failures. Evaluation Evaluation is assessing the intermediary products in the system development process. Exploratory Testing The simultaneous learning, designing and executing of tests, in other words every form of testing in which the tester designs his tests during test execution and the information obtained is reused to design new and improved test cases. Failure The result or manifestation of one or more defects/faults. When the system is performing differently from the required behavior, from a viewpoint outside the system. Users will see the failure. Fault The result of an error residing in the code or document. Fault is the view from inside the system. Fault is the state where mistake or error exists. Developers will see the fault. Function point analysis (FPA) A method aiming to measure the size of the functionality of an automated system. The measurement is independent of the technology. This measurement may be used as a base for the measurement of productivity, the estimation of the needed resources, and project control. Initial level The first level of maturity in the TPI model. It can be described as “ad hoc activities”. Improvement suggestion A practice-based and adaptable description of how Checkpoints can be met. Also contains useful hints and tips related to the Key area. Inspection A formal evaluation technique, with products being read thoroughly by a group of experts. In addition to determining whether the solution is adequately processed, an inspection focuses primarily on achieving consensus on the quality of a product. The aim of the inspection is to help the author find as many deviations as possible in the available time. This quality improvement process for written material consists of two dominant components; product (document itself) improvement and process improvement (of both document production and Inspection). Integration test A test, aiming at the proper functioning of two or more identifiable end products. Key area Part of the test process that is considered as a combination of coherent aspects. In order to measure and improve the test process in a more detailed way and step by step, the Business Driven TPI model consists of a set of 16 Key areas. For each Key area, the maturity can be measured separately. The Key areas together cover all aspects of the test process. Known errors Defects that have been found but have not been solved (yet). Logical test case Describes, in logical terms, the circumstances in which the system behavior is examined by indicating which test situations are covered by the test case. Master test plan Test plan by which the various test levels are geared to one another. Maturity A software engineering term indicating to which extent it is planned how to do things when testing software. Maturity category The extent of maturity indicated by Initial, Controlled, Efficient and Optimizing. Maturity level For a specific Key area, the maturity category that has been determined by an assesment or that serves as a goal for future improvement actions. The maturity level of the combination of Key areas defines the maturity of the test process as a whole. Maturity Matrix See: Test maturity matrix. Maturity stage See: Tool-specific maturity stage. Metrics Quantified observations of the characteristics of a product or process used to estimate and manage the test process, to give justification for the test process, to substantiate test advice and to compare systems or processes. Metrics are also important for improving the test process. Optimizing level The fourth (and highest) level of maturity in the TPI Model, after Initial, Controlled and Efficient. It can be expressed as; “Continuously adapting to ever-changing circumstances”. Outsourcing Subcontracting a process, such as product design or manufacturing, to a third-party company. More specific for testing: An organization’s test process can be (partly) outsourced to a supplier. Physical test case The concrete elaboration of a logical test case, with choices having been made for the values of all required inputs and settings of the environmental factors. Pre-test Testing the delivered product in such a way that it is determined whether or not the product is of sufficient quality to execute a complete test of this product. Principal stakeholder The person responsible for the test assignment and the first in line for test reports. Product risk The chance that the product fails in relation to the expected damage if this occurs: Product risk = Chance of failure * Damage where Chance of failure = Chance of defects * Frequency of use A product risk can obstruct the proper functioning of a product (especially an information system). Usually product risks are part of the Test strategy. Product risk analysis Analyzing the product to be tested with the aim of achieving a joint view, for the test manager and other stakeholders, of the more or less risky characteristics and parts of the product to be tested so that the thoroughness of testing can be related to this view. Project risk A risk that can obstruct the execution of a project in accordance with the plan. Project risks can apply to the entire project or more specific to the test project. Quality The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs. Quality assurance All the planned and systematic activities necessary to provide adequate confidence that a product or service meets the requirements for quality. Quality characteristic A property of an information system. Regression Regression is the phenomenon that the quality of a system deteriorates as a whole as a result of individual amendments. Regression test A regression test is aimed at verifying that all the unchanged parts of a system still function correctly after the implementation of a change. Return On Investment (ROI) The ratio of money gained or lost (whether realized or unrealized) on an investment relative to the amount of money invested. Reliability The degree to which the information system remains free from interruptions. Reusability The degree to which parts of the information system, or the design, can be reused for the development of different applications. Review An evaluation technique where a product (60-80% complete) is submitted to a number of reviewers with the question to assess it from a certain perspective (depending on the review type). A review focuses primarily on finding courses for a solution on the basis of the knowledge and competencies of the reviewers, and on finding and correcting defects. There are various review types, such as: technical review (e.g. selecting solution direction/alternative), management review (e.g. determining project status), peer review (review by colleagues), and expert review (review by experts). Risk reporting A description of the extent to which the system meets the specified quality requirements and the risks associated, as defined in the product risk, with bringing a particular version into production, including any available alternatives. Role Describes one or more tasks and the knowledge and skills required to carry them out. SDLC See: Software Development Life Cycle. Software Development Life Cycle The Software Development Life Cycle (SDLC), or Systems Development Life Cycle in systems engineering and software engineering, is the process of creating or altering systems, and the models and methodologies that people use to develop these systems. The concept generally refers to computer or information systems. In software engineering the SDLC concept underpins many kinds of software development methodologies. These methodologies form the framework for planning and controlling the creation of an information system. SPI Software Process Improvement, a generic term for models that aim at improving the quality of software and the related processes. Examples are CMMI® and SPICE®. SPICE® Software Process Improvement and Capability Determination is based on the international standard ISO/IEC 15504 and is a specific framework for a software process improvement. Stakeholder A person who affects and/or can be affected by and/or has an interest in the test process and/or the improvement of the test process. Static testing Testing by examining end-products (such as manuals or source code) without any programs being executed. System integration test A test carried out by the supplier, with the aim of demonstrating that (sub)system interface agreements have been met, correctly interpreted and correctly implemented. System test A test carried out by the supplier in a (manageable) laboratory environment, with the aim of demonstrating that the developed system, or parts of it, meet with the functional and non-functional specifications and the technical design. Test basis The test basis is the information that defines the required system behavior. Test case Used to examine whether the system displays the desired behavior under specific circumstances. Test design technique A standardized method of deriving test cases from a particular test basis that will achieve a certain coverage. Test environment A composition of parts, such as hardware and software, connections, environment data, maintenance tools and management processes in which a test is carried out. Test goal A test goal is a success criterion for the test assignment formulated in the language of the client or stakeholder. Test infrastructure Consists of the facilities and resources necessary to facilitate the satisfactory execution of the test. A distinction is made between test environments, test tools and workplaces. Test level A group of test activities that are managed and executed collectively. Test line The operational organization to provide test services to one or more clients. A test line has a fixed team of testers, infrastructure, test tools and standardized work procedures. Test maturity matrix A matrix that visualizes the combination of Key areas, Checkpoints and Maturity levels. Test object The information system (or part thereof) to be tested. Test organization The whole of the test functions, facilities, procedures and activities including their relationships. Test plan In a test plan the general structure and the strategic choices with respect to the test to be executed are formulated. The test plan forms the scope of reference during the execution of the test and also serves as an instrument to communicate with the client of the test. The test plan is a description of the test project, including a description of the activities and the planning; therefore it is not a description of the tests themselves. Test point Unit of measurement for the size of the high-level test to be executed. Test point analysis (TPA) A method with the possibility to perform a technology-independent measurement of the test depth level of an information system, on the basis of a function point analysis, and to use this measurement as a basis for a productivity measurement, an estimate of the required resources, and project management. Test policy Describes how an organization deals with the people, resources and methods involved with the test process in the various situations. Test process The collection of working methods, techniques and tools used to perform a test. Test Process Improvement Optimizing quality, costs and lead time of the test process, in relation to the total information services. Test script Combines multiple physical test cases to be able to execute them in an efficient and simple manner. Test situation An isolated condition under which the test object displays a specific behavior that needs to be tested. Test strategy The distribution of the test effort and coverage over the parts to be tested or aspects of the test object aimed at finding the most important defects as early and cheaply as possible. Test team A group of people who, led by a test manager, undertake test activities. Test technique A set of actions aimed at creating a test deliverable by a universal method. Test tool An automated instrument that supports one or more test activities, such as planning, control, specification and execution. Test tool policy Describes how an organization handles the acquisition, implementation and use of test tools in the various situations. Test type A group of test activities with the intention of checking the information system in respect of a number of correlated (part aspects of) quality characteristics. Test unit A collection of processes, transactions and/or functions that are tested collectively. Testability The ease and speed with which characteristics of the system can be tested (following each adjustment). Testing A process that provides insight into, and advice on, quality and the related risks. Testware All artifacts that are produced in test activities, such as a test plan, test cases and test report. In TPI NEXT artifacts that are used as input to the test process, like the test basis and the test object, are considered as testware as well. The authorship and ownership of these artifacts lies outside the test team, but as these artifacts are essential to the test process, they are within scope of testware management. Tool-specific maturity stage These maturity stages allow for a more specific assesment of the maturity in the use of test tools and provide particular guidance for improving the way a tool is used. Tool-specific maturity stages take into account both the objectives of a tool and how the use of a tool should be incorporated in the testing process. Tool-specific maturity stages have no one-on-one relationship with the Maturity levels. TPI NEXT A frame of reference for determining the maturity of an organization’s test process and for setting up and implementing specific and realistic measures, based on the business-drivers, for the improvement of this test process. Unit integration test A test carried out by the developer in the development environment, with the aim of demonstrating that a logical group of units meets the requirements defined in the technical specifications. Unit test A test carried out in the development environment by the developer, with the aim of demonstrating that a unit meets the requirements defined in the technical specifications. Users acceptance test A test carried out by the future user(s) in an optimally simulated production environment, with the aim of demonstrating that the developed system meets the requirements of the users. Walkthrough An evaluation technique by which the author explains the contents of a product during a meeting. Several different objectives are possible: bringing all participants to the same starting point, transfer of information, asking the participants for additional information or letting the participants choose from the alternatives proposed by the author.