Welcome and Opening MessageA-MOST at Maracaípe
Chair(s): Raluca Lefticaru University of Bradford

Welcome and Opening MessageITEQS at Paiva

Welcome and Opening MessageIWCT at Serrambi

Welcome and OpeningMutation at Tamandaré

Keynote 1Mutation at Tamandaré
Chair(s): Jie M. Zhang University College London, UK

Keynote TalkA-MOST at Maracaípe
Chair(s): Raluca Lefticaru University of Bradford

Keynote TalkITEQS at Paiva

IWCT IIWCT at Serrambi

Welcome and Opening MessageAIST at Muro Alto
Chair(s): Alexandra Mendes HASLab/INESC TEC & Universidade da Beira Interior

Keynote TalkAIST at Muro Alto
Chair(s): Alexandra Mendes HASLab/INESC TEC & Universidade da Beira Interior

Session 1Mutation at Tamandaré

ITEQS IITEQS at Paiva

A-MOST IA-MOST at Maracaípe
Chair(s): Uraz Cengiz Türker University of Leicester, UK

IWCT IIIWCT at Serrambi

AIST Papers Session IAIST at Muro Alto

Keynote 2Mutation at Tamandaré

A-MOST IIA-MOST at Maracaípe
Chair(s): Uraz Cengiz Türker University of Leicester, UK

ITEQS IIITEQS at Paiva

Award SessionMutation at Tamandaré

Session 2Mutation at Tamandaré

ClosingMutation at Tamandaré

Best Paper Award and ClosingITEQS at Paiva

IWCT IIIIWCT at Serrambi

A-MOST IIIA-MOST at Maracaípe
Chair(s): Uraz Cengiz Türker University of Leicester, UK

Keynote TalkA-MOST at Maracaípe
Chair(s): Uraz Cengiz Türker University of Leicester, UK

AIST Papers Session IIAIST at Muro Alto
Chair(s): Nuno Pombo Universidade da Beira Interior

IWCT Organizing and Steering Committee MeetingIWCT at Serrambi

ClosingAIST at Muro Alto
Chair(s): Alexandra Mendes HASLab/INESC TEC & Universidade da Beira Interior

Invited TutorialA-MOST at Maracaípe
Chair(s): Florian Lorber Aalborg University

A-MOST IVA-MOST at Maracaípe
Chair(s): Florian Lorber Aalborg University

ClosingA-MOST at Maracaípe
Chair(s): Florian Lorber Aalborg University

Intro & WelcomeResearch Papers at Porto de Galinhas

Testing Tools ITesting Tools Track at Carneiros

Testing Concurrent and Quantum SystemsResearch Papers at Porto de Galinhas
Chair(s): Marcos Lordello Chaim

Keynote 1Research Papers at Porto de Galinhas
Chair(s): Ana Paiva Faculty of Engineering of the University of Porto

Tools Demo ITool Demos Track at Boa Viagem

Testing Tools IITesting Tools Track at Carneiros

Testing and LearningResearch Papers at Porto de Galinhas
Chair(s): Andrea Stocco Università della Svizzera italiana (USI)

Journal First IJournal-First Papers at Boa Viagem
Chair(s): Heike Wehrheim Paderborn University

Testing Tools IIITesting Tools Track at Carneiros

Models, Testing and VerificationResearch Papers at Porto de Galinhas
Chair(s): Mike Papadakis University of Luxembourg, Luxembourg

Industry Best Practices for CI/CDIndustry Track at Carneiros

Industry IIndustry Track at Carneiros

Industry IIIndustry Track at Carneiros

Slicing and Static AnalysisResearch Papers at Porto de Galinhas
Chair(s): Leopoldo Teixeira Federal University of Pernambuco

Keynote 2Research Papers at Porto de Galinhas
Chair(s): Fabiano Ferrari Federal University of São Carlos

Award SessionResearch Papers at Porto de Galinhas

Posters IPosters Track at Boa Viagem
Chair(s): Anna Rita Fasolino Federico II University of Naples, Sreedevi Sampath University of Maryland Baltimore County

Industry IIIIndustry Track at Carneiros

Test ReuseResearch Papers at Porto de Galinhas
Chair(s): Paolo Tonella USI Lugano, Switzerland

Faults and Fault InjectionResearch Papers at Porto de Galinhas
Chair(s): André T. Endo Federal University of Technology - Paraná (UTFPR)

Industry IVIndustry Track at Carneiros

Autonomous and Cyber-Physical SystemsResearch Papers at Porto de Galinhas
Chair(s): Paolo Arcaini National Institute of Informatics

Keynote 3Research Papers at Porto de Galinhas
Chair(s): Robert Hierons University of Sheffield

ICST Steering Committee meetingResearch Papers at Porto de Galinhas

Program RepairResearch Papers at Porto de Galinhas
Chair(s): Angelo Gargantini University of Bergamo

Journal First IIJournal-First Papers at Boa Viagem
Chair(s): Augusto Sampaio Universidade Federal de Pernambuco

Empirical and User StudiesResearch Papers at Porto de Galinhas
Chair(s): Michael Felderer University of Innsbruck

Welcome and IntroductionDevOps4CPS-Testing at Muro Alto

Starting INTUITESTBEDSINTUITESTBEDS at Paiva

InSTA OpeningInSTA at Serrambi

Welcome to NEXTANEXTA at Tamandaré

InSTA Research PaperInSTA at Serrambi

Challenges of testing cyber-physical systems along the DevOps processDevOps4CPS-Testing at Muro Alto

Active Machine Learning to Test Autonomous DrivingNEXTA at Tamandaré

Keynote Speaker: Karl Meinke Session Chair: Michael Felderer

KeynoteINTUITESTBEDS at Paiva

Invited Keynote Dr. Mariano Ceccato University of Verona

Security Testing of Android Apps

Android facilitates apps interoperation and integration through inter-process communication mechanism, by allowing an app to request a task from another app that is installed on the same device. However, this interoperation mechanism poses security risks if an app does not implement it properly, such as permission re-delegation vulnerabilities, i.e., a form of privilege escalation where unprivileged malicious apps exploit vulnerable privileged apps to take privileged actions on the attacker behalf. Static analysis techniques as well as run-time protections have been proposed to detect permission re-delegation vulnerabilities. However, as acknowledged by their authors, most of these approaches are affected by many false positives because they do not discriminate between benign task requests and actual permission re-delegation vulnerabilities. In this keynote, we will present a recent approach aiming at filling this gap and at bridging static and dynamic analysis with security testing for precise detection of permission re-delegation vulnerabilities. Our approach first groups a large set of benign and non-vulnerable apps into different clusters, based on their similarities in terms of functional descriptions. It then generates permission re-delegation model for each cluster, which characterizes common permission re-delegation behaviors of the apps in the cluster. Given an app under test, our approach checks whether it has permission re-delegation behaviors that deviate from the model of the cluster it belongs to. If that is the case, it generates test cases to detect the vulnerabilities, that show how the vulnerabilities can be exploited. Empirical validation suggests that this security testing approach outperforms state-of-the-art in terms of vulnerability detection precision.

InSTA Research PaperInSTA at Serrambi

AI-based Test Automation: A Grey Literature AnalysisNEXTA at Tamandaré

Authors: Filippo Ricca, DIBRIS, Università di Genova, Italy; Alessandro Marchetto, Independent Researcher, Italy and Andrea Stocco, Università della Svizzera italiana (USI), Switzerland Abstract: This paper provides the results of a survey of the grey literature concerning the use of artificial intelligence to improve test automation practices. We surveyed more than 1,200 sources of grey literature (e.g., blogs, white-papers, user manuals, StackOverflow posts) looking for highlights by professionals on how AI is adopted to aid the development and evolution of test code. Ultimately, we filtered 136 relevant documents from which we extracted a taxonomy of problems that AI aims to tackle, along with a taxonomy of AI-enabled solutions to such problems. Manual code development and automated test generation are the most cited problem and solution, respectively. The paper concludes by distilling the six most prevalent tools on the market, along with think-aloud reflections about the current and future status of artificial intelligence for test automation. Session Chair: Michael Felderer

InSTA Emerging IdeaInSTA at Serrambi

Keynote TBADevOps4CPS-Testing at Muro Alto

openCAESAR: a DevOps Platform for Systems Engineering by Maged Elaasar

Flaky Mutants; Another Concern for MutationTestingNEXTA at Tamandaré

Authors: Sten Vercammen, Serge Demeyer, Markus Borg and Robbe Claessens Abstract: Mutation testing is the state-of-the-art technique for assessing the fault detection capability of a test suite. An underlying assumption, rarely mentioned, is that the system under test behaves completely deterministically. This is rarely the case, as each mutant changes the code, it is highly likely that some introduce non-determinism. We call these flaky mutants. As they are only detected intermittently, they cause unreliable mutation testing scores, waste developer time, possibly unfruitful tests, and potential loss in confidence in the mutation testing technique. We want to raise awareness of this issue as we found that these flaky mutants are easy to create and occur in real projects. We also share some thoughts on how to tackle this issue. Mutation testing is the state-of-the-art technique for assessing the fault detection capability of a test suite. An underlying assumption, rarely mentioned, is that the system under test behaves completely deterministically. This is rarely the case, as each mutant changes the code, it is highly likely that some introduce non-determinism. We call these flaky mutants. As they are only detected intermittently, they cause unreliable mutation testing scores, waste developer time, possibly unfruitful tests, and potential loss in confidence in the mutation testing technique. We want to raise awareness of this issue as we found that these flaky mutants are easy to create and occur in real projects. We also share some thoughts on how to tackle this issue. Session Chair: Kristian Sandahl

InSTA Industry Experience ReportInSTA at Serrambi

InSTA ClosingInSTA at Serrambi

Using Advanced Code Analysis for Boosting Unit Test CreationNEXTA at Tamandaré

Authors: Miroslaw Zielinski, Parasoft Corporation, Poland and Rix Groenboom, Parasoft Corporation, Netherlands. Abstract: Unit testing is a popular testing technique, widespread in enterprise IT and embedded/safety-critical. For enterprise IT, unit testing is considered to be good practice and is frequently followed as an element of test-driven development. In the safety-critical world, there are many standards, such as ISO 26262, IEC 61508, and others, that either directly or indirectly mandate unit testing. Regardless the area of the application, unit testing is very time-consuming and teams are looking for strategies to optimize their efforts. This is especially true in the safety-critical space, where demonstration of test coverage is required for the certification. In this presentation, we share the results of our research regarding the use of advanced code analysis algorithms for augmenting the process of unit test creation. The discussion includes automatic discovery of inputs and responses from mocked components that maximize the code coverage and automated generation of the test cases. Session Chair: Kristian Sandahl

INTUITESTBEDS Full research paper INTUITESTBEDS at Paiva

Test Automation with Grad-CAM Heatmaps - A Future Standard Block in MLOps for Vision AI?DevOps4CPS-Testing at Muro Alto

INTUITESTBEDS Full research paper INTUITESTBEDS at Paiva

QRTest: Automatic Query Reformulation for Information Retrieval Based Regression Test Case PrioritizationNEXTA at Tamandaré

Author: Maral Azizi, East Carolina University, US. Abstract: The most effective regression testing algorithms have long running times and often require dynamic or static code analysis, making them unsuitable for the modern software development environment where the rate of software delivery could be less than a minute. More recently, some researchers have developed information retrieval-based (IR-based) techniques for prioritizing tests such that the higher similar tests to the code changes have a higher likelihood of finding bugs. A vast majority of these techniques are based on standard term similarity calculation, which can be imprecise. One reason for the low accuracy of these techniques is that the original query often is short, therefore, it does not return the relevant test cases. In such cases, the query needs reformulation. The current state of research lacks methods to increase the quality of the query in the regression testing domain. Our research aims at addressing this problem and we conjecture that enhancing the quality of the queries can improve the performance of IR-based regression test case prioritization (RTP). Our empirical evaluation with six open source programs shows that our approach improves the accuracy of IR-based RTP and increases regression fault detection rate, compared to the common prioritization techniques. Session Chair: Sahar Tahvili

An Empirical Study of Parallelizing Test Execution Using CUDA Unified Memory and OpenMP GPU OffloadingNEXTA at Tamandaré

Authors: Taghreed Bagies and Ali Jannesari, Lowa State University, US. Abstract: The execution of software testing is costly and time-consuming. To accelerate the test execution, researchers have applied several methods to run the testing in parallel. One method of parallelizing the test execution is by using a GPU to distribute test case inputs among several threads running in parallel. In this paper, we investigate three programming models CUDA Unified Memory, CUDA Non-Unified Memory, and OpenMP GPU offloading to parallelize the test execution and discuss the challenges using these programing models. We use eleven benchmarks and parallelize their test suites by using these models. We evaluate their performance in terms of execution time, analyze the results, and report the limitations of using these programming models. Session Chair: Vahid Garousi

Panel and Open DiscussionDevOps4CPS-Testing at Muro Alto

INTUITESTBEDS Position paper INTUITESTBEDS at Paiva

Closing INTUITESTBEDSINTUITESTBEDS at Paiva

Advancing Test Automation Using Artificial Intelligence (AI)NEXTA at Tamandaré

Keynote speaker: Jeremy S. Bradbury, PhD, Associate Professor, Computer Science, Associate Dean, School of Graduate and Postdoctoral Studies Ontario Tech University. Abstract: In recent years, software testing automation has been enhanced through the use of Artificial Intelligence (AI) techniques including genetic algorithms, machine learning and deep learning. The use cases for AI in test automation range from providing recommendations to the complete automation of software testing activities. To demonstrate the breadth of application, I will present several recent examples of how AI can be leveraged to support automated testing in rapid release cycles. Furthermore, I will discuss my own successes and failures in using AI to advance test automation as well as share the lesson I have learned. Session Chair: Sahar Tahvili

Session ICCIW at Maracaípe

ClosingNEXTA at Tamandaré

ADEPTNESS and COSMOS sessionDevOps4CPS-Testing at Muro Alto

Session IICCIW at Maracaípe

ClosingDevOps4CPS-Testing at Muro Alto

Session IIICCIW at Maracaípe