Bachelor and Master Theses

Title: A Study on Timed Base Choice Criteria for Testing Embedded Software
Subject: Computer Science
Level: Basic
Description: Background
Combinatorial test design is a test generation approach, popular in part due to its ability to efficiently create tests. Numerous case studies have appeared presenting practical applications of different combinatorial techniques. The Base Choice (BC) combination strategy was proposed by professors Paul Ammann and Jeff Offutt. It varies the values of one parameter at a time while keeping the values of the other parameters fixed until all of the combinations have been used [1].

Goal (s)/ Problem(s)
Test cases are created based on BC strategy by varying the values of one parameter at a time while keeping the values of the other parameters fixed on the values in the base choice. However, this strategy might not be as effective when used on embedded software where parameters need to be set for a certain amount of time in order to trigger a certain interesting behaviour. By incorporating time as another parameter when generating the tests, the goal is to create a new strategy that will increase not only code coverage but also fault detection compared to base choice strategy.

Method
Our research started with finding the problem of improving BC criteria in industrial software development and testing, and ends with performing an empirical evaluation. The thesis contains two parts: the development of a new coverage criteria and a case study [2]. In order to evaluate the efficiency of TBC algorithm we will also be comparing tests generated using a random algorithm as well as regular Base Choice.

We perform an experimental evaluation using software programs developed by industrial professionals from Bombardier Transportation AB by comparing TBC with BC and Random tests. Measurements will include code coverage in terms of branches taken and decision coverage, cost based on generation time and execution time for each test and fault detection in percentage form.

Expected Outcome(s)
The outcome of this thesis will be the proposal of a new coverage criteria named Timed Base Choice (TBC) that improves upon the already defined BC criteria with the notion of time. By incorporating time as a factor when executing tests while changing the value of each given parameter, we assume that the program will be exercised in a different way than when compared to BC. To test if TBC improves the achieved code coverage as well as the fault detection, in this thesis, we present the results from a comparative evaluation of TBC with BC and random tests. The expected outcome of this thesis will also be an evaluation using real-world programs developed in industry used for comparing three test design techniques in terms of achieved code coverage, fault detection and cost.
Initial Timeplan
Initial time plan consists of:
Background : March 29, 2015 - April 11, 2015
Related work
Research/Implementation: March 29, 2015 - April 24, 2015
Experimentation: April 25, 2015 - May 15, 2015
Report writing: March 29, 2015 - June 4, 2015


Limitations
By only testing TBC comparatively with BC and random on Bombardier’s software, the results might not provide sufficient evidence that TBC is a superior algorithm in terms of code coverage and fault detection. In order to achieve a more general result we would need to perform more studies with systems from other companies.

References:
[1] Grindal, Mats; Offutt, Jeff; Andler, Sten F.. (2005). Combination testing strategies: a survey. Wiley InterScience. (15), 167-199

[2] Runeson, Per, and Martin Höst. "Guidelines for conducting and reporting case study research in software engineering." Empirical software engineering 14.2 (2009): 131-164.
Prel. end date: 2016-06-5
Presentation date: 2016-06-3
Student: Henning Bergström hbm13001@student.mdh.se
IDT supervisor: Eduard Paul Enoiu
eduard.paul.enoiu@mdh.se, +46-21-101624
Examinator: Daniel Sundmark
Daniel Sundmark
daniel.sundmark@mdh.se, +46-21-103145

Rapport och bilagor

Size

Senaste uppdatering

TR1912.pdf

1412039

2016-08-03, 11:22


  • Mälardalen University |
  • Box 883 |
  • 721 23 Västerås/Eskilstuna |
  • 021-101300, 016-153600 |
  • webmaster |
  • Latest update: 2017.10.14