Reference Library Testing Tactics
	This page provides access to a variety of downloadable papers that address software testing techniques. The following topics are considered: Agile Testing Object - Oriented Testing (OOT) Testing Methods Unified Modeling Language (UML) - Related Agile Testing A Framework for Good Enough Testing [PDF] James Bach This article discusses a Good Enough quality model. The Good Enough model is first defined, then the parts of the definition are assessed. The author asks how good the testing is and is it worth improving Agile Testing What is it? Can it work? [PDF] Bret Pettichord The topics for this slide presentation are: what agile testing is, agile development methodologies, the role of testing, test-first programming, refactoring: improve the design of existing code, acceptance testing, a practice for agile testing (conversation test creation, coaching tests, providing test interfaces, exploratory learning). Extreme Rules of the Road How a Tester Can Steer an eXtreme Programming Project Toward Success [PDF] Lisa Crispin This article on XP compares XP with driving a car; first gear is planning, second is writing acceptance tests and third is performing and automating tests. Common problems like issues that come up that look like defects but are outside the scope of the iteration are also discussed. Heuristic Risk-Based Testing [PDF] James Bach This article explains what risk-based testing is and then explains why you would want to do it and how to do it well. Heuristic analysis and two approaches to analysis (inside-out and outside-in) are then presented. Three ways to organize risk - based testing and making it all work are then discussed. New Models for Test Development [PDF] Brian Marick In this paper the author discusses why the "V model" doesn't work. The author then describes a model he thinks is better. Important requirements for test models are then presented. Quality Assurance and Testing in Agile Projects [PDF] Pete McBreen This slide presentation covers the following topics: the manifesto for agile software development, QA in agile projects - what does it mean, agile testing?? - not just another buzzword, what is so special about Test Driven Development, Test Driven Development in non-XP projects, helping users with acceptance testing, back to basics - what are the real requirements, safeguarding the user experience - the role of QA in usability, ensuring quality is designed into the application, assisting in the creation of working software. Back to the top Object - Oriented Testing (OOT) Automated Testing of Object-Oriented Components using Intelligent Test Artifacts [PPT] Michael Silverstein This PowerPoint presentation covers the following topics: component test frameworks, test design questions, pattern responses to design questions, test component, test artifact - properties, test artifact - consequences, test sequence, configured instance, test stimulus, test stimulus - strategy, nested state, aggregate state, state validation, state validation - strategy, object variation, and an example artifice hierarchy. Binder Object - Oriented Testing Metrics [PPT] Author Unknown This PowerPoint presentation outlines what the software engineer should do to: calculate the lack of cohesion in methods, find the number of public attributes, calculate the number of friend classes, and determine the number of root classes. Developing Test Adequacy Criteria for OO Software [PDF] Brigid Haworth This report is on the development of criteria based on structural coverage requirements for object oriented software. The focus on this report is on the development of criteria for objects. Efficient Strategies for Integration and Regression Testing of OO Systems [PDF] *FEE* Thierry Jéron, Jean-Marc Jézéquel, Yves Le Traon and Pierre Morel This paper presents a model, a strategy and a methodology for planning integration and regression testing from an OO model. In the paper a model of structural system test dependencies is produced which serves as a basis for ordering classes and methods to be tested for regression and integration purposes. An algorithm which computes a strategy for integration testing with a quadratic complexity is detailed and also a comparison is given of various integration strategies with the proposed optimized algorithm. Integrating Object - Oriented Testing and Development Processes [PDF] *FEE* John D. McGregor and Timothy D. Korson This paper presents an integrated development and testing process model that will apply object-oriented methods to software system development. Problems with testing are discussed as well as solutions to the problems. Following that is an introduction to the process model, a comprehensive testing process, techniques for testing specific work products, issues in the integration and possible approaches for organizing the testing effort. Investigating the Effectiveness of Object - Oriented Testing Strategies with the Mutation Method [PDF] Sunwoo Kim, John A. Clark and John A. McDermid The mutation method assesses test quality by examining the ability of a test set to distinguish syntactic deviations representing specific types of faults from the program under test. This paper describes an empirical study performed to evaluate the effectiveness of object-oriented (OO) test strategies using the mutation method. The test sets for the experimental system are generated according to three selected OO test methods and their effectiveness is compared by determining how well the developed test sets kill injected mutants derived from an established mutation system Mothra, and the authors own OO-specific mutation technique which is termed Class Mutation. Object - Oriented Program Testing In Black and White: An Integrated Approach to Class - Level Testing of Object - Oriented Programs [PPT] Huo Yan Chen, T.H. Tse, F.T. Chan and T.Y. Chen This slide presentation outlines object - oriented program testing using algebraic specification to formally specify an object - oriented program. Terms and normal forms are presented. Other topics include: detecting errors, fundamental pairs, selecting a test set, path-based domain partition for particular method m, the algorithm GFT, an example of GFT, points about GFT, object equivalence, variable invariant, black - box undecidability, relevant observable context, data member relevance graph, algorithm DOE (Determining Observational Equivalence), limitations, and experimental results. Object Oriented Testing Special Requirements [PDF] Author Unknown This paper discusses the test phases in traditional systems development, path coverage criteria, and special requirements for testing OO software (class/object testing and object integration testing). On Built - in Test Reuse in Object - Oriented Framework Design [PDF] Yingxu Wang, Graham King, Mohamed Fayad, Dilip Patel, Ian Court, Geoff Staples and Margaret Ross Object-oriented frameworks have extended reusability of software from code modules to architectural and domain information. This paper further extends software reusability from code and architecture to built-in tests (BITs) in object-oriented framework development. Methods for embedding BITs at object and object-oriented framework levels are addressed. Behaviors of objects and object-oriented frameworks with BITs in the normal and test modes are analyzed. Systematic reuse methods of BITs in object-oriented framework development are provided. On Testing Object - Oriented Programs [PDF] Yvan Labiche This article discusses the modifications introduced by the object-oriented paradigm in the testing process. This article presents shortly the problems raised by this new approach and some work on these topics. And finally, describes directions for further investigations. Risk - Based Object Oriented Testing [PDF] Linda H. Rosenberg, Ruth Stapko and Albert Gallo This paper discusses risk-based testing, which takes into account the probability of failure of a portion of code as determined by its complexity. This paper details the tasks involved in risk-based testing. Test Case Generation for Class - Level Object - Oriented Testing [PDF] T.H. Tse and Zhinong Xu This paper discusses a new testing process to generate test cases for object-oriented programs, with the focus on classes with mutable objects. The test case construction process is guided by formal object-oriented specifications. In this approach, testers first analyze the formal specification of a class to partition the state space of the class and identify a test model that is based on finite-state machines, then analyze the class specification and the test model to select a set of test data for each method of the class, and finally prepare the test cases of the class from the test model by following various well-developed testing criteria. This paper also extends the subtype relationship to allow testing information on superclasses to be inherited by subclasses. V&V Lifecycle Methodologies [PDF] David F. Rico This paper introduces a new notion called Verification & Validation (V&V) Lifecycle Methodologies, examining what V&V is, and expanding its scope to the entire software lifecycle (much beyond traditional methods of software testing). V&V Lifecycle Methodologies are streamlined and efficient, yet highly holistic and concisely measured step-by-step threaded and sequenced approaches to predicatively and deterministically building-in software quality as it is developed across the entire software lifecycle. V&V is recast in a much more holistic definition and approach, based on a rapidly maturing V&V discipline, solid empirical data, and recent innovations in metrics-based software quality and reliability management and engineering. Back to the top Testing Methods A Graphical Class Representation for Integrated Black - and White - Box Testing [PDF] Sami Beydeda , Volker Gruhn and Michael Stachorski This paper proposes a new graphical representation of classes, which can be used for integrated class-level black and white-box testing. Its distinguishing feature from existing representations is that each method of a class is shown from two perspectives, namely the specification and implementation view. Both the specification of a method and its implementation are represented as control flow graphs, which allows black- and white-box testing by structural techniques. Moreover, a test suite reduction technique has been developed for adjusting white-box test cases to black box testing. Automated Software Testing to Cope with Market Demands [PDF] Magic Software Enterprises Ltd. This paper discusses the MAGICIAN project that Magic Software Enterprises Ltd. Conducted. The MAGICIAN project was part of an overall process to raise the level of development technology from level one on the SEI CMM scale to level two. This article discusses how the project was performed, the result analysis, the lessons learned, and future actions. Criteria for Generating Specification - based Tests [PDF] A. Jefferson Offutt, Yiwei Xiong and Shaoying Liu This paper presents general criteria for generating test inputs from state-based specifications. These techniques provide coverage criteria that are based on the specifications, and are made up of several parts, including test prefixes that contain inputs necessary to put the software into the appropriate state for the test values. The test generation process includes several steps for transforming specifications to tests. Empirical results from a comparative case study application of these criteria are presented. Designing Unit Test Cases [PDF] IPL Information Processing Ltd. This paper discusses test design and presents a general process to develop unit test specifications. This paper also explains specific design techniques to design unit test cases with. Distributed Systems Testing [PDF] *FEE* Anna Liu and Paddy Nixon This paper contains five papers that have been selected as part of a minitrack and present many challenges and new advances in the field of distributed systems testing. Generating Goal - Oriented Test Cases [PDF] *FEE* Anneliese von Mayrhauser, Michael Scheetz and Eric Dahlman This paper describes an AI planner assisted approach to generate test cases for system testing based on high level test objectives. Test generation is based on an extended UML model of the system under test and a mapping of high-level test objectives into initial and goal conditions of the planner. This paper illustrates the use of this approach on a series of high level test objectives one might apply to a robot controlled tape silo. Generating Software Test Data by Evolution [PDF] *FEE* Christoph C. Michael, Gary McGraw and Michael A. Schatz This paper discusses the use of generic algorithms (Gas) for automatic software test data generation extending previous work on dynamic test data generation where the problem of test data generation is reduced to one of minimizing a function. The function is minimized using one of two genetic algorithms in place of the local minimization techniques used in earlier research. The implementation of the system is described and the effectiveness of the approach and the effect of program complexity are examined. Incremental Regression Testing [PDF] Hiralal Agrawal, Joseph R. Horgan, Edward W. Krauser and Saul A. London The purpose of regression testing is to ensure that bug fixes and new functionality introduced in a new version of a software do not adversely affect the correct functionality inherited from the previous version. This paper explores efficient methods of selecting small subsets of regression test sets that may be used to establish the same. Measurement Driven Testing Process: Further Empirical Studies [PDF] Giovanni Denaro, Sandro Morasca and Mauro Pezzè This paper exploits logistic regression as a technique for correlating module fault-proneness to software metrics. In particular, it proposes and evaluates a new method for computing the quality of prediction of the models, discusses test strategies that can take advantage of the produced models for tuning the testing processes, and provides additional experimental data for supporting the validity of logistic regression for this purpose. On Fault Coverage of Tests for Finite State Specifications [PDF] A. Petrenko and G. v. Bochmann Testing is a trade-off between increased confidence in the correctness of the implementation under test and constraints on the amount of time and effort that can be spent in testing. This paper analyzes basic ideas underlying the techniques for fault coverage analysis and assurance mainly developed in the context of protocol conformance testing based on finite state models. Special attention is paid to parameters which determine the testability of a given specification and influence the length of a test suite which guarantees complete fault coverage. Protocol Testing: Review of Methods and Relevance for Software Testing [PDF] *FEE* Gregor v. Bochmann and Alexandre Petrenko Since communication protocols are implemented in software and/or hardware, this paper explains in which way testing protocol implementations is different from the usual software testing. This paper also reviews major results in the area of protocol testing and also discusses how the methods could also be relevant in the general context of software testing. Software Testing by Statistical Methods Preliminary Success Estimates for Approaches based on Binomial Models, Coverage Designs, Mutation Testing, and Usage Models [PDF] David Banks, William Dashiell, Leonard Gallagher, Charles Hagwood, Raghu Kacker and Lynne Rosenthal Exhaustive conformance testing of software is not practical because variable input values and variable sequencing of inputs result in too many possible combinations to test. This paper addresses alternatives for exhaustive testing based on statistical methods, including experimental designs of tests, statistical selection of representative test cases, quantification of test results, and provision of statistical levels of confidence or probability that a program implements its functional specification correctly. The goal of this work is to ensure software quality and to develop methods for software conformance testing based on known statistical techniques, including multivariable analysis, design of experiments, coverage designs, usage models, and optimization techniques, as well as to provide quantitative measures of quality, reliability, and conformance to specifications, including statistical measures and confidence levels. Structured Testing Analysis and Extensions [PDF] Arthur Henry Watson This paper analyzes the theoretical properties of structured testing which is a methodology for software module testing based on the cyclomatic complexity measure of McCabe. This paper also describes the implementation of a system to support structured testing, evaluates error detection performance, and extends structured testing to cover integration testing. Many more topics are covered including Weyuker's axioms for testing criteria and showing a variant of structured testing in which only executable paths are considered satisfies those axioms. Using Domain Models for System Testing [PDF] A. von Mayrhauser and R. Mraz Domain Models have long been used as a basis for software development and reuse. This paper presents a specialized, simplified domain model that has been used for system testing in industry as the framework for a system testing approach called Application Domain Based Testing. Sleuth, a test suite generation tool, is based on this concept. This paper reports on the domain analysis, the domain model components, and industrial experience of reusing domain models and tests generated with Sleuth. White Box Testing [PDF] AppLabs This report on AppLabs white box testing process begins with an introduction that includes goals, scope, staff and deliverables. The requirements analysis is then discussed along with practices, parameters, test areas, security, testing tools, infrastructure, and documentation. Back to the top Unified Modeling Language (UML) - Related Adaptive Scenario - Based Testing Using UML [PDF] W.T. Tsai, R. Paul, Xhibin Cao, Bingnan Xiao and Lian Yu This slide presentation begins with an introduction to testing then covers testing with UML. Scenario specification, analysis, and patterns (formalized scenarios, requirement patterns and verification patters, case studies and evaluation) are then presented. A Testing Profile for UML [PDF] Ina Schieferdecker This slide presentation covers the following: motivation, why a test specification, the testing process, the UML family, test development, test case specification, test sequence charts, and a test system. A UML - Based Approach to System Testing [PDF] Lionel Briand and Yvan Labiche System testing is concerned with testing an entire system based on its specifications. In the context of object-oriented, UML development, this means that system test requirements are derived from UML analysis artifacts such as use cases, their corresponding sequence and collaboration diagrams, class diagrams, and possibly Object Constraint Language (OCL) expressions across all these artifacts. This paper describes a methodology in a practical way and is illustrated with an example. Focus on UML Testing Strategies [PDF] Isabel Evans and Richard Warden This paper shows how to set about designing an UML test strategy. Topics include: setting goals for the test strategy, doesn't UML provide a strategy, identify UML usage, carry out a risk analysis on UML and application knowledge, include requirements reviews in the strategy, and common issues to address in your strategy. Generating Test Cases from UML Specifications [PDF] Aynur Abdurazik and Jeff Offutt This paper presents a technique that uses Offutt's state-based specification test data generation model to generate test cases from UML statecharts. A tool TCGen has been developed to demonstrate this technique with specifications written in Software Cost Reduction (SCR) method and Unified Modeling Language (UML). Experimental results from using this tool are presented. Object - Oriented Specification - Based Testing Using UML Statechart Diagrams [PDF] Marlon E. Vieira, Marcio S. Dias and Debra J. Richardson This paper summarizes an approach to specification-based testing that employs UML statechart diagrams as the underlying specification, and automatically generates test drivers and a test script to execute the component under test. The test drivers cover the specification according to a selected test criterion and verify the behavior of the component under test. This approach has been implemented in a prototype called DAS-BOOT (Design And Specification-Based Object-Oriented Testing). Software Testing and the UML [PDF] Clay E. Williams The software testing community has had much less awareness and debate about UML then software design and development communities, and has largely been absent as the modeling standard was developed. This is an important issue, because in many software development organizations, the cost of testing can account for more than 40% of the total development cost for a software system. This abstract seeks to explore the possibility of using the UML for software testing. Test Cases Generation from UML State Diagrams [PDF] Y.G. Kim, H.S. Hong , S.M. Cho, D.H. Bae and S.D. Cha The paper discusses the application of state diagrams in UML to class testing. A set of coverage criteria is proposed based on control and data flow in UML state diagrams and it is shown how to generate test cases satisfying these criteria from UML state diagrams. Testing the Consistency of Dynamic UML Diagrams [PDF] Gregor Engels, Jan Hendrik Hausmann, Reiko Heckel and Stefan Sauer This slide presentation covers the following topics: consistency in multi-view software models (consistency of UML behavior specifications), Dynamic Meta Modeling, DMM environment for automated model testing, and DMM rules for consistency specification. The UML and Testing: A Perfect Fit? [PDF] Clay Williams This slide presentation covers UML and testing with Enterprise Application Integration (EAI) and Dynamic e-business (Web services). Topics include: EAI scenario, Web services application, how can UML help, UML and EAI testing, UML and Web services testing, and some implications for UML. The UML Testing Profile An Overview [PDF] Alessandra Cavarra This paper introduces the UML testing profile draft and compares it to the AGEDIS Modeling Language (AML), emphasizing, when possible, communalities and divergences. UML and Testing [PDF] Simon Pickin This slide presentation outlines the following: UML, UML and testing, describing software tests in UML, test description language TeLa, testing software designed in UML, UML and component testing, area of application, test objectives & test cases, test synthesis in the UML context, test synthesis with UMLaut/TGV, assumptions about the UML spec, generation of simulation API from UML specification, LTS built via simulation API, ATC example, and test synthesis method. UML - Based Integration Testing for Component - Based Software [PDF] Ye Wu, Mei-Hwa Chen and Jeff Offutt The Unified Modeling Language (UML) has been widely adopted in component-based software development processes. Many of its useful tools, such as interaction diagrams, statechart diagrams, and component diagrams, characterize the behavior of components in various aspects, and thus can be used to help test component-based systems. This paper first analyzes different test elements that are critical to test component-based software, then a group of UML-based test adequacy criteria is proposed that can be used to test component-based software. Back to the top