Practical Model-Based Testing: A Tools Approach |
|
|
|
|
| Mark Utting (Author), Bruno Legeard (Author) |
| November 2006, Morgan Kaufmann, Hardcover, 456 pages, ISBN 0123725011
|
|
|
|
 |
|
This book gives a practical introduction to model-based testing, showing how to write models for testing purposes and how to use model-based testing tools to generate test suites. It is aimed at testers and software developers who wish to use model-based testing, rather than at tool-developers or academics.
The book focuses on the mainstream practice of functional black-box testing and covers different styles of models, especially transition-based models (Unified Modeling Language (UML) state machines) and pre/post models (Unified Modeling Language (UML)/OCL specifications and B notation). The steps of applying model-based testing are demonstrated on examples and case studies from a variety of software domains, including embedded software and information systems.
From this book you will learn:
* The basic principles and terminology of model-based testing
* How model-based testing differs from other testing processes
* How model-based testing fits into typical software lifecycles such as agile methods and the Unified Process
* The benefits and limitations of model-based testing, its cost effectiveness and how it can reduce time-to-market
* A step-by-step process for applying model-based testing
* How to write good models for model-based testing
* How to use a variety of test selection criteria to control the tests that are generated from your models
* How model-based testing can connect to existing automated test execution platforms such as Mercury Test Director, Java JUnit, and proprietary test execution environments
* Presents the basic principles and terminology of model-based testing
* Shows how model-based testing fits into the software lifecycle, its cost-effectiveness, and how it can reduce time to market
* Offers guidance on how to use different kinds of modeling techniques, useful test generation strategies, how to apply model-based testing techniques to real applications using case studies |
 |
|
Preface XIII
About the Authors XIX
The Challenge 1
What Do We Mean by Testing? 3
What Is Model-Based Testing? 6
A Smart Card Example 10
Summary 17
Further Reading 17
The Pain and the Gain 19
Classic Testing Processes 19
A Manual Testing Process 20
A Capture/Replay Testing Process 22
A Script-Based Testing Process 23
A Keyword-Driven Automated Testing Process 24
Solved and Remaining Problems 25
The Model-Based Testing Process 26
Models: Build or Borrow? 31
Your Maturity Level 33
Your Testing Maturity Level 33
Your Modeling Maturity Level 34
Hypothetical Case: Total Testing Hours 35
Assumptions 36
Conclusions 38
Model-Based Testing Experience Reports 40
Model-Based Testing at IBM 40
Model-Based Testing at Microsoft 41
Model-Based Testing in the Smart Card Industry 43
Model-Based Testing in the Automotive Industry 47
Benefits of Model-Based Testing 48
SUT Fault Detection 48
Reduced Testing Cost and Time 49
Improved Test Quality 50
Requirements Defect Detection 51
Traceability 52
Requirements Evolution 54
Limitations of Model-Based Testing 54
Summary 56
Further Reading 56
A Model of Your System 59
How to Model Your System 60
Notations for Modeling 62
Choosing a Notation 64
A Case Study 66
DVM Requirements 66
DVM High-level Design 66
Transition-Based Models 69
Finite State Machines 69
Overview and History of Statecharts 70
Unified Modeling Language (UML) State Machines 72
The Unified Modeling Language (UML) Object Constraint Language 75
Pre/Post Models in B 78
The Complete B Method 78
A Simple Drink Vending Machine 79
Overview of B Machines 82
The B Toolkit 85
A Richer DVM Model 91
Other Pre/Post Notations 98
Summary 103
Further Reading 104
Selecting Your Tests 107
Structural Model Coverage 110
Control-Flow-Oriented Coverage Criteria 111
Data-Flow-Oriented Coverage Criteria 114
Transition-Based Coverage Criteria 115
Unified Modeling Language (UML)-Based Coverage Criteria 120
Data Coverage Criteria 122
Boundary Value Testing 123
Statistical Data Coverage 127
Pairwise Testing 128
Fault-Based Criteria 130
Requirements-Based Criteria 131
Explicit Test Case Specifications 132
Statistical Test Generation Methods 133
Combining Test Selection Criteria 133
Summary 136
Further Reading 136
Testing from Finite State Machines 139
Testing Qui-Donc with a Simple FSM 140
Informal Requirements 140
Modeling Qui-Donc with an FSM 141
Generating Tests 148
Complete Testing Methods 155
EFSMs and the ModelJUnit Library 157
Extended Finite State Machines 157
The ModelJUnit Library 160
An EFSM Model of Qui-Donc 162
Unit Testing ZLive with EFSMs 167
The System under Test: ZLive FlatPred 168
A Family of Models 169
Encoding FlatPredModel in Java 175
Test Results 179
Labeled Transition Systems Models 182
Summary 183
Further Reading 184
Testing from Pre/Post Models 187
How to Write Pre/Post Models for Testing 188
Adding Requirements Tags 191
The System Process Scheduler Example 192
Functional Requirements 192
Modeling the Scheduler 194
An Introduction to Test Targets 200
Test Selection for the Scheduler 203
Understanding the Test Targets 204
Test Case Generation Using LTG/B 205
Traceability between Requirements and Test Cases 207
Generating More Tests 208
The Triangle Example 214
Informal Specification and Formal Models 214
The High-Level Triangle Model 215
Test Generation for Compound Decisions 217
Evaluation of the Generated Test Suites 227
Robustness Testing from a Pre/Post Model 233
Syntactic Modeling and Format Testing 234
Testing a Chat System with Spec Explorer 237
An Overview of Spec Explorer 240
The Chat System 242
The Spec# Model 242
Test Generation with Spec Explorer 245
Summary 248
Further Reading 250
Testing from Unified Modeling Language (UML) Transition-Based Models 251
Unified Modeling Language (UML) Modeling Notations 252
Testing an eTheater with LTG/Unified Modeling Language (UML) 253
Requirements 253
Assumptions for the Test Model 257
A Unified Modeling Language (UML) Model of eTheater 257
Generating Tests Using LEIRIOS LTG/Unified Modeling Language (UML) 261
Generating Better Test Suites 263
Testing a Protocol with Qtronic 266
Protocol 267
Model 267
Importing the Model into Qtronic 277
Connecting Qtronic to the SUT 277
Generating and Running Tests 279
Summary 280
Further Reading 281
Making Tests Executable 283
Principles of Test Adaptation 284
The Adaptation Approach 286
The Transformation Approach 288
Which Approach Is Better? 290
Example: The eTheater System 291
Transforming Tests into Ruby 295
Writing a Ruby Adapter for eTheater 297
Executing the eTheater Tests 300
Summary 303
Further Reading 303
The GSM 11.11 Case Study 305
Overview of the GSM 11.11 Standard 306
Selected Files 307
Security Aspects 308
Selected Commands 309
Modeling GSM 11.11 in B 311
Abstracting the Behavior 311
The Data Model 312
The Operational Model 316
Validation and Verification of the B Model 321
Validation by Animation 321
Model Verification 322
Generating Tests with LTG/B 323
Model Coverage Criteria and Test Generation Parameters 324
Computing Test Targets 324
Generating Test Cases 327
Generating Executable Scripts 327
LTG Test Script Generator 328
GSM 11.11 Adaptation Layer 332
Test Execution 334
Summary 337
Further Reading 338
The ATM Case Study 339
Overview of the ATM System 340
Modeling the ATM System in Unified Modeling Language (UML) 344
Class Diagrams 344
Modeling Behavior with State Machines and OCL 349
Comparing the Design and Test Models 355
Generating Test Cases 357
Initial State for Test Generation 357
Generation of Test Cases 359
Discussion 359
Generating Executable Test Scripts 364
Executing the Tests 365
Summary 370
Further Reading 370
Putting It into Practice 371
Prerequisites for Model-Based Testing 371
Selecting a Model-Based Testing Approach 373
People, Roles, and Training 377
Model-Based Testing and Agile Methods 380
Test-Driven Development 380
Acceptance Tests and the Iterative Process 381
Agile Modeling 382
Model-Based Testing and the Unified Process 382
Introducing the Unified Process 383
Extending the Unified Process with Model-Based Testing 384
Epilogue 387
Model-Based Testing 387
Future Trends 388
Summary of B Abstract Machine Notation 391
Machine Structure 391
Logical Predicates (P, Q) 392
Numbers (n, m) 392
Sets (S, T) 392
Relations (r) 393
Functions (f) 393
Commands and Substitutions (G H, I, J) 393
Summary of Common OCL Constructs 397
Basic OCL Types 397
OCL Collections 398
Commercial Tools 401
Glossary 405
Bibliography 409
Index 419 |
|
|
