University of Calgary Logo
Department of Electrical and Computer Engineering
 

Dependability, Reliability, and Testing of Software Systems
(SENG 637)
Course Outline
()
Course Brochure
This course is also available as an industrial course of the Lecture Series on Software Systems for The Future. Please click on the above image for the 2 pages course brochure (PDF format) and contact the series editor for reservation. You can also purchase the course CDROM online. The CDROM includes the most recent version of all the slides, handouts and other deliverables.

Instructor: Dr. Behrouz Far (Associate Professor, Schulich School of Engineering, UofC)
   
Course/Year: Software Engineering Graduate Program (SENG)
   
Department: Electrical and Computer Engineering
   
School Year: Winter 2010
   
Timetable: Lecture: Tuesday (17:30-20:20) starting January 12th, 2010
   
Room: Lectures: ICT 516 (located on the 5th Floor, ICT Building)
   
Background Courses: SENG 301: Analysis and Design of Large Scale Software I (Recommended)
SENG 421: Software Metrics (Recommended)
   
Lecture Format: Lecture (1x3 hours sessions per week), total 36 hours
   
Contact: Office: ICT 543
Tel./Fax. (403) 210-5411
   
Instructor's email: far@ucalgary.ca
   
Course Outline: Course Outline (PDF format)

Announcements
  1. Sample reliability measures (for project).

1. Course Description and Outline

This course introduces software reliability process, reliability growth models and shows techniques to improve and predict software reliability. Concepts such as defining necessary reliability, developing operational profiles, techniques to improve and predict software reliability, preparing and executing test, black box testing, white box testing, unit testing, system testing, and integration testing will be explained.

2. Course Web Site

The SENG 637 course home page contains links to up-to-date course information, problem assignments, announcements, as well as laboratory and examination scheduling. The SENG 637 course home page is available through the B.H. Far's home page at the URL:
(http://www.enel.ucalgary.ca/People/far/Lectures/SENG637/)

3. Allocation of Marks

Criteria Total mark Comments
Final Project Presentation (individual or group) 20% About 15 minutes presentation and 5 minutes discussion. will be held on Tuesday 6/April/2010.
Tutorial Report (individual) 30% Assignments for tutorials
Deadline: 13/April/2010 (Tuesday)
Project (individual or group) 50% Assignments for projects
Suggested projects list
Sample project document
Deadline: 13/April/2010 (Tuesday)

4. Detail Contents

Regular Sessions
1st week 12/Jan/2010 (Tue):
Chapter 1:
Overview of software reliability engineering
  • Introducing the course.
  • What is software reliability?
  • What factors affect software quality?
  • What is software reliability engineering?
  • Introducing software reliability engineering process.
2nd week 19/Jan/2010 (Tue):
Chapter 2:
Review of software Reliability Models
  • Basic Features of the Software Reliability Models.
  • Single Failure Model.
  • Reliability Growth Model.
  • Weibull and Gamma Failure Class Models.
  • Bayesian Models.
  • Early Life-Cycle Prediction Models.
3rd week 26/Jan/2010 (Tue):
Chapter 3:
System reliability
  • Serial and parallel system reliability.
  • Active redundancy.
  • Reliability Block Diagram (RBD).
  • Hazard Analysis.
  • Failure Modes and Effect Analysis (FMEA).
  • Fault Tree Analysis (FTA).
4th week 2/Feb/2010 (Tue):
Chapter 4:
Necessary reliability
  • Introducing failure severity class and failure intensity objective concepts.
  • Steps in defining necessary reliability.
  • Computing failure intensity objective for developed software.
  • Strategies to meet failure intensity objective.
5th week 9/Feb/2010 (Tue):
Chapter 5:
Operational profiles
  • Defining function, operation, run, run type, operational mode, operational profile.
  • Representation of operational profile.
  • Procedure to define operational profile.
  • Create functions/operations list.
  • Determine occurrence rate of individual operations.
  • Determine occurrence probabilities.
6th week 23/Feb/2010 (Tue):
Chapter 6:
Preparing for test
  • Direct and indirect input variables.
  • Operation, load and regression test.
  • What is a test case?
  • How to manage test cases?
  • Test procedure.
  • Equivalence classes and boundary conditions.
  • How to develop test cases?
7th week 2/Mar/2010 (Tue):
Chapter 7:
Executing test
  • How to allocate test time among system components based on test type (feature test, regression test, load test) and operation modes?
  • In what order the test should be carried on?
  • How to write a test suite?
  • How many test cases and runs should be invoked?
  • What deviation could be discovered?
  • How to document the execution results?
8th week 9/Mar/2010 (Tue):
Chapter 8:
Software reliability tools
  • Introducing SRE simulation and reliability growth tools such as: SMERFS, SRMP, SoftRel, CASRE.
  • Case study using CASRE.
9th week 16/Mar/2010 (Tue):
Chapter 9:
Applying failure data to guide decisions
  • Guiding decisions for certification test.
  • Guiding decisions for reliability growth test.
  • Zero failure testing.
  • Handling program evolution, ureported failures and variations of operational profiles.
10th week 23/Mar/2010 (Tue):
Chapter 10:
Deploying software reliability engineering
  • Software Quality System (SQS); Software Quality Assurance (SQA) and Software Reliability Engineering (SRE).
  • Quality, test and data plans.
  • Roles and responsibilities.
  • Sample quality and test plan.
  • Best current practice of SRE.
  • Practice variations.
11th week 30/Mar/2010 (Tue):
Chapter 11:
Cleanroom software development
  • Cleanroom software development.
  • Comparing craft-based and cleanroom software engineering.
  • Cleanroom development process.
  • cleanroom design and verification.
  • cleanroom testing.
12th week 6/Apr/2010 (Tue):
Students presentation

7. Textbooks and Suggested References

Reference Materials: The course handouts are the major source of information. The handouts, slides and extra reading materials will be distributed on each session. All the handouts and other materials are also downloadable from the course web page. For those with limited access to the internet, a CDROM containing all the documents will be distributed on demand.
   
Main Textbook:
This book is used as the regular textbook for this course.
   
Additional Recommended Text and Reference Books:
   
 
  • Software Reliability: Measurement, Prediction and Application, J.D. Musa, A. Iannini, K. Okumoto, (621 p.), McGraw-Hill (1987).
    ISBN 0-07-044093-X.
   
 
   
 
   
 
  • Testing Computer Software, C. Kaner, J. Falk, H.Q. Nguyen, 2nd edition, (480 p.), John Wiley and Sons, (1999).
    ISBN: 0-471-35846-0.
   
 
   
Here is a list of other textbooks that may be found useful:
  1. Addison-Wesley - A Practical Guide to Testing Object Oriented Software (2001).
  2. Addison-Wesley - Critical Testing Processes Plan Prepare Perform Perfect (2003).
  3. Addison-Wesley - Effective Software Testing (2002).
  4. Addison-Wesley - Software Test Automation (1999).
  5. Addison-Wesley - Test Driven Development By Example (2002).
  6. Addison-Wesley - Testing Embedded Software (2003).
  7. Addison-Wesley - Testing Extreme Programming (2002).
  8. Apress - Software Exorcism A Handbook for Debugging and Optimizing Legacy Code (2003).
  9. Apress - Test Driven Development A J2EE Example (2005).
  10. Artech House - Building Reliable Component based Software Systems (2002).
  11. Artech House - ISO 9001 2000 Quality Management System Design (2003).
  12. Artech House - Practical Guide To Software Quality Management 2Ed (2003).
  13. Artech House - Software Fault Tolerance Techniques and Implementation (2001).
  14. Artech House - Testing and Quality Assurance for Component Based Software (2003).
  15. CMU TR022-96 - Cleanroom Software Engineering Reference Model.
  16. CRC Press - Design for Reliability (2001).
  17. Elsevier Science - Reliability Maintainability and Risk (2001).
  18. IEEE - Preventive Risk Management For Software Projects.
  19. IEEE - SWEBOK Software Engineering.
  20. IEEE 1008-1987 - Standard for Software Unit Testing (1986).
  21. Kluwer - Computing System Reliability Models and Analysis (2004).
  22. Manning - Junit in Action (2004)
  23. Manning - JUnit Recipes Practical Methods for Programmer Testing (2005).
  24. Microsoft Press - Testing Software Patterns (2003).
  25. Morgan Kaufmann - Unit Testing in Java How Tests Drive the Code (2003).
  26. NIST - The Economic Impacts of Inadequate Infrastructure for Software Testing (2002).
  27. OReilly - Perl Testing A Developers Notebook (2005).
  28. Pragmatic Programmers - Pragmatic Unit Testing in C# with Nunit (2004).
  29. Pragmatic Programmers - Pragmatic Unit Testing in Java With Junit (2003).
  30. Pragmatic Programmers - Pragmatic Unit Testing in Java with JUnit (2003).
  31. Prentice-Hall - Java Testing And Design From Unit Testing To Automated Web Tests (2004).
  32. Prentice-Hall - Java Testing Design and Automation (2004).
  33. Prentice-Hall - Software Development Building Reliable Systems (1999).
  34. Springer Verlog - Practical Software Testing (2003).
  35. Springer Verlog - Reliability Engineering Theory and Practice 3Ed (1999).
  36. Thomson - Game Testing All in One (2005).
  37. Wiley - Automated Web Testing Toolkit - Expert Methods For Testing and Managing Web Applications (2001).
  38. Wiley - Case Studies in Reliability and Maintenance (2003).
  39. Wiley - Optimal Reliability Modeling Principles and Applications (2003).
  40. Wiley - Reliability Survivability and Quality of Large Scale Telecommunication Systems (2003).
  41. Wiley - Reliability of Computer Systems and Networks - Fault Tolerance Analysis and Design (2002).
  42. Wiley - Software Quality Engineering - Testing, Quality Assurance and Quantifiable Improvement (2005).
  43. Wiley - Software Testing Fundamentals Methods and Metrics (2003).
  44. Wiley - Statistical Design and Analysis of Experiments 2Ed (2003).
  45. Wiley - Statistical Methods for Survival Data Analysis 3Ed (2003).
  46. Wiley - The Art of Software Testing 2Ed (2004).
  47. Wiley - The Web Testing Companion (2003).
  48. Wiley - Trustworthy Systems Through Quantitative Software Engineering (2005).

8. Other Information

Related Links:

All the slides and notes can be viewed on-line using Netscape Navigator or MSIE (version 3.x or later) browsers. Copies of the slides in Portable Document Format (PDF) is available for on-line download. Please note that the course downloadable materials are provided solely for the internal use for the registered students of this course. External and industrial participants may contact the author for availability of the materials.
All Unix, PC and MAC users can download, view and print the PDF version of the documents using Adobe's Acrobat Reader.

This page was created by Dr. B.H. Far.If you may find omissions, glitches, have suggestions for improvement of the material presented here, please contact me.
  Copyright Terms. THIS DOCUMENT AND ITS ENTIRE CONTENTS ARE COPYRIGHT 2003 BY DR. B.H. FAR. COPYING, REPUBLISHING AND DISTRIBUTING THIS DOCUMENT IN WHOLE OR IN PART IS PROHIBITED BY LAW. IF YOU DESIRE TO REPUBLISH PARTS OF THIS DOCUMENT IN ELECTRONIC FORM, PLEASE CONTACT THE AUTHOR.