Bachelor and Master Theses

Title: Pairwise Testing for PLC Embedded Software
Subject: Computer Science
Level: Basic
Description: Background
Pairwise testing is a combinatorial approach used to reduce the number of test case inputs to a system in situations where exhaustive testing with all possible inputs is not possible or very expensive. Given a set of input parameters where each parameter can take one of a discrete set of values, a test consists of a collection of inputs which captures all possible combinations of pairs of input values [1].

Goal (s)/ Problem(s)
As far as we know there are no pairwise testing approaches that work with PLC software. In addition, earlier work suggests that pairwise testing can be effective at detecting faults. Complementing earlier work, this thesis proposes an efficient pairwise test data generation strategy for PLC embedded software and an empirical comparison between pairwise testing and manual tests written by industrial engineers in order to mainly compare the fault detection capability.

A research methodology is a structured and systematic process that fulfills a certain research objective. Our research started with finding a problem or opportunity of using pairwise testing in industrial software development, and ends with performing an empirical evaluation [2]. The case study design started with the formulation of the research objective that was broken down into four research questions:
How to implement pairwise for PLC software?
How does pair-wise and manual tests compare in terms of achieved code coverage?
Are pairwise tests less costly to use than manual tests?
Are pairwise tests more effective in fault detection than manual tests?

Overview of the methodology. The TCMS (Train Control Management System) programs are tested using tests suites already created by industrial engineers, and tests suites generated by the use of the pairwise algorithm. All these tests suites will then be executed on the TCMS programs to collect data related to cost and code coverage. In addition these test suites will be analyzed in terms of fault detection using mutation analysis. In this way test suites are executed on both the original and the mutated programs for calculating the mutation score.

First, pairwise testing will be implemented in a tool ready to be used for testing PLC software. An evaluation with regard to its success in meeting code coverage in terms of achieved decision coverage will be performed using a case study. In addition, in the same case study, we will compare pair-wise with manual testing in terms of cost and fault detection using mutation analysis. The resulted data will be analyzed using descriptive and inferential statistics.

Expected Outcome(s)
The results will show the implementation of an automatic pairwise test generator for PLC software and whether this approach may be practical and useful in certain testing scenarios. In this thesis we present also the results from a comparative evaluation between pairwise and manual tests produced by real industrial engineers. We perform this evaluation using real-world programs developed in industry and we compare these two test design techniques in terms of achieved code coverage, fault detection and cost. The outcome of this thesis is a pairwise testing approach for PLC software and an empirical comparison between pairwise generated tests and those created by industrial engineers. We are interested in revealing how good pairwise testing is in terms of code coverage and fault detection when compared to manual tests.

Initial Timeplan
Initial time plan consists of:
Background : March 28, 2016 - April 10, 2016
Related work
Research/Implementation: April 10, 2016 - May 1, 2016
Experimentation: May 2, 2016 - May 15, 2016
Report writing: April 1, 2016 - May 24, 2016

The results are mainly based on programs provided by one company (i.e., Bombardier Transportation AB in Sweden). More studies are needed to generalize the results of this thesis to other systems and domains.


[1] Grindal, M., Offutt, J., & Andler, S. F. (2005). Combination testing strategies: a survey. Software Testing, Verification and Reliability, 15(3), 167-199.

[2] Runeson, Per, and Martin Höst. "Guidelines for conducting and reporting case study research in software engineering." Empirical software engineering 14.2 (2009): 131-164.

Prel. end date: 2016-06-15
Presentation date: 2016-06-05
Student: Linus Eklund
Student: Peter Charbachi
IDT supervisor: Eduard Paul Enoiu, +46-21-101624
Examinator: Daniel Sundmark
Daniel Sundmark, +46-21-103145

Rapport och bilagor


Senaste uppdatering



2016-08-03, 11:26

  • Mälardalen University |
  • Box 883 |
  • 721 23 Västerås/Eskilstuna |
  • 021-101300, 016-153600 |
  • webmaster |
  • Latest update: 2017.08.23