ICST 2021
Mon 12 - Fri 16 April 2021

Conference Dates
Conference Dates are in time zone (GMT-03:00) Brasilia, Distrito Federal, Brazil, and may differ from the viewed time zone.
Rooms
Tracks
Badges
Your Program
You're viewing the program in a time zone which is different from your device's time zone - change time zone

Conference Day
Mon 12 Apr

Displayed time zone: Brasilia, Distrito Federal, Brazil change

09:00 - 09:15
Welcome and Opening MessageA-MOST at Maracaípe
Chair(s): Raluca LefticaruUniversity of Bradford
09:00
15m
Day opening
Welcome and Opening Message
A-MOST

09:00 - 09:15
Welcome and Opening MessageITEQS at Paiva
09:00
15m
Day opening
Welcome and Opening Message
ITEQS
Mehrdad SaadatmandRISE Research Institutes of Sweden, Dragos TruscanÅbo Akademi University, Eduard Paul EnoiuMälardalen University
09:00 - 09:15
Welcome and Opening MessageIWCT at Serrambi
09:00
15m
Day opening
Welcome and Opening Message
IWCT
Dimitris SimosSBA Research
09:00 - 09:10
Welcome and OpeningMutation at Tamandaré
09:00
10m
Day opening
Welcome
Mutation
Amin AlipourUniversity of Houston, Jie M. ZhangUniversity College London, UK
09:10 - 10:00
Keynote 1Mutation at Tamandaré
Chair(s): Jie M. ZhangUniversity College London, UK
09:10
50m
Keynote
Mutation for Compiler Testing
Mutation
Dan HaoPeking University, China
09:15 - 10:15
Keynote TalkA-MOST at Maracaípe
Chair(s): Raluca LefticaruUniversity of Bradford
09:15
60m
Keynote
Nobody cares about models, but everybody wants automation
A-MOST
Jeff OffuttGeorge Mason University
09:15 - 10:15
Keynote TalkITEQS at Paiva
09:15
60m
Keynote
Keynote: Security Testing and Resilience
ITEQS
09:15 - 10:15
IWCT IIWCT at Serrambi
09:15
30m
Paper
A Combinatorial Approach to Explaining Image Classifiers
IWCT
Jaganmohan ChandrasekaranUniversity of Texas at Arlington, USA, Yu Lei, Raghu KackerNational Institute of Standards and Technology, Rick KuhnNational Institute of Standards and Technology
09:45
15m
Short-paper
Combinatorial Testing Metrics for Machine Learning
IWCT
Erin LanusVirginia Tech, Laura J. Freeman, Rick KuhnNational Institute of Standards and Technology, Raghu KackerNational Institute of Standards and Technology
10:00
15m
Short-paper
Computing Sequence Covering Arrays using Unified Combinatorial Interaction Testing
IWCT
Hanefi Mercan, Cemal YilmazSabancı University
09:30 - 09:40
Welcome and Opening MessageAIST at Muro Alto
Chair(s): Alexandra MendesHASLab/INESC TEC & Universidade da Beira Interior
09:30
10m
Day opening
Welcome and Opening Message
AIST

09:40 - 10:40
Keynote TalkAIST at Muro Alto
Chair(s): Alexandra MendesHASLab/INESC TEC & Universidade da Beira Interior
09:40
60m
Keynote
Learning How to Test - Generating Context-Infused Test Cases
AIST
K: Gregory GayChalmers and the University of Gothenburg
10:00 - 10:50
Session 1Mutation at Tamandaré
10:00
16m
Short-paper
Efficiently Sampling Higher Order Mutants Using Causal Effect
Mutation
Saeyoon OhKorea Advanced Institute of Science and Technology (KAIST), Seongmin LeeKorea Advanced Institute of Science and Technology, Shin YooKorea Advanced Institute of Science and Technology
10:16
16m
Full-paper
Inducing Subtle Mutations with Program RepairBest Paper Award
Mutation
Florian Schwander, Rahul GopinathCISPA, Germany, Andreas ZellerCISPA Helmholtz Center for Information Security
10:33
16m
Short-paper
Automatic Equivalent Mutants Classification Using Abstract Syntax Tree Neural Networks
Mutation
Samuel PeacockTowson University, Lin DengTowson University, Josh DehlingerTowson University, Suranjan ChakrabortyTowson University
10:30 - 11:30
ITEQS IITEQS at Paiva
10:30
30m
Full-paper
Software Defects Rules Discovery
ITEQS
A: Andreea VescanBabes-Bolyai University, A: Camelia SerbanDepartment of Computer Science, Babes-Bolyai University, A: Gloria Cerasela Crisan"Vasile Alecsandri" University of Bacau
11:00
30m
Full-paper
Online GANs for Automatic Performance Testing
ITEQS
A: Ivan PorresÅbo Akademi University, Hergys RexhaÅbo Akademi University, Sebastien LafondÅbo Akademi University
10:45 - 11:45
IWCT IIIWCT at Serrambi
10:45
30m
Paper
A Combinatorial Approach to Testing Deep Neural Network-based Autonomous Driving Systems
IWCT
Jaganmohan ChandrasekaranUniversity of Texas at Arlington, USA, Yu Lei, Raghu KackerNational Institute of Standards and Technology, Rick KuhnNational Institute of Standards and Technology
11:15
30m
Paper
SYSMODIS: A Systematic Model Discovery Approach
IWCT
Omer KorkmazSabancı University, Cemal YilmazSabancı University
11:00 - 11:50
Keynote 2Mutation at Tamandaré
11:00
50m
Keynote
What It Would Take to Use Mutation Testing in Industry?
Mutation
Moritz BellerFacebook, USA
11:15 - 11:45
A-MOST IIA-MOST at Maracaípe
Chair(s): Uraz Cengiz TürkerUniversity of Leicester, UK
11:15
30m
Paper
Process mining on a robotic mechanism
A-MOST
11:45 - 12:45
ITEQS IIITEQS at Paiva
11:45
30m
Full-paper
Assuring Fairness of Algorithmic Decision Making
ITEQS
A: Marc HauerTU Kaiserslautern - Algorithm Accountability Lab , A: Rasmus AdlerFraunhofer IESE, A: Katharina ZweigTU Kaiserslautern - Algorithm Accountability Lab
12:15
30m
Full-paper
Enabling Fast Exploration and Validation of Thermal Dissipation Requirements for Heterogeneous CPU Platforms
ITEQS
A: Joel ÖhrlingAbo Akademi University, A: Dragos TruscanÅbo Akademi University, A: Sebastien LafondÅbo Akademi University
11:50 - 12:00
Award SessionMutation at Tamandaré
11:50
10m
Awards
Award Announcement
Mutation
Jie M. ZhangUniversity College London, UK
12:30 - 12:45
12:30
15m
Day closing
Concluding Remarks
Mutation

12:45 - 13:00
Best Paper Award and ClosingITEQS at Paiva
12:45
15m
Day closing
Best paper award and closing
ITEQS
Mehrdad SaadatmandRISE Research Institutes of Sweden, Dragos TruscanÅbo Akademi University, Eduard Paul EnoiuMälardalen University
12:45 - 14:00
IWCT IIIIWCT at Serrambi
12:45
15m
Short-paper
A Practical Method for API Testing in the Context of Continuous Delivery and Behavior Driven Development
IWCT
13:00
30m
Paper
An environment for benchmarking combinatorial test suite generators
IWCT
Andrea BombardaUniversity of Bergamo, Edoardo CrippaUniversity of Bergamo, Angelo GargantiniUniversity of Bergamo
Pre-print
13:30
30m
Paper
Combinatorially XSSing Web Application Firewalls
IWCT
Bernhard Garn, Daniel Sebastian Lang, Manuel Leithner, Rick KuhnNational Institute of Standards and Technology, Raghu KackerNational Institute of Standards and Technology, Dimitris SimosSBA Research
13:00 - 13:30
A-MOST IIIA-MOST at Maracaípe
Chair(s): Uraz Cengiz TürkerUniversity of Leicester, UK
13:00
30m
Paper
Test Sequence Generation with Cayley Graphs
A-MOST
Sylvain HalléUniversité du Québec à Chicoutimi, Raphael KhouryUniversité du Québec à Chicoutimi, Canada
13:30 - 14:30
Keynote TalkA-MOST at Maracaípe
Chair(s): Uraz Cengiz TürkerUniversity of Leicester, UK
13:30
60m
Keynote
Model-Based Testing and Model-Learning for Variability-Intensive Systems
A-MOST
Mohammad MousaviUniversity of Leicester, UK
13:30 - 14:50
AIST Papers Session IIAIST at Muro Alto
Chair(s): Nuno PomboUniversidade da Beira Interior
13:30
30m
Paper
Prioritized Test Generation Guided by SoftwareFault Prediction
AIST
Eran H, Roni Stern, Rui AbreuFaculty of Engineering, University of Porto, Portugal
14:00
30m
Paper
Supervised Learning for Test Suit Selection in Continuous Integration
AIST
Ricardo Miguel Martins, Rui AbreuFaculty of Engineering, University of Porto, Portugal, João Pedro Nunes Nadkarni
14:30
20m
Paper
AutoKG - An Automotive Domain Knowledge Graph for Software Testing: A position paper
AIST
14:30 - 15:00
IWCT Organizing and Steering Committee MeetingIWCT at Serrambi
14:50 - 15:00
ClosingAIST at Muro Alto
Chair(s): Alexandra MendesHASLab/INESC TEC & Universidade da Beira Interior
14:50
10m
Day closing
Closing
AIST

15:00 - 15:45
Invited TutorialA-MOST at Maracaípe
Chair(s): Florian LorberAalborg University
15:00
45m
Tutorial
UPPAAL for Test Case Generation
A-MOST
Marius MikucionisAalborg University
15:45 - 16:15
A-MOST IVA-MOST at Maracaípe
Chair(s): Florian LorberAalborg University
15:45
30m
Paper
A Model-Based Test Script Generation Framework for Embedded Software
A-MOST
Wasif AfzalMälardalen University, Eduard Paul EnoiuMälardalen University
16:15 - 16:30
ClosingA-MOST at Maracaípe
Chair(s): Florian LorberAalborg University
16:15
15m
Day closing
Closing
A-MOST

Conference Day
Tue 13 Apr

Displayed time zone: Brasilia, Distrito Federal, Brazil change

09:00 - 09:15
09:00
15m
Day opening
Intro & Welcome
Research Papers

09:15 - 10:15
Testing Tools ITesting Tools Track at Carneiros
09:15
30m
Paper
COSMO: Code Coverage Made Easier for Android
Testing Tools Track
Andrea Romdhana, Mariano CeccatoUniversity of Verona, Gabriel Georgiu, Alessio Merlo, Paolo TonellaUSI Lugano, Switzerland
09:45
30m
Paper
Differential coverage: automating coverage analysis
Testing Tools Track
Henry CoxMediatek
09:15 - 10:15
Testing Concurrent and Quantum SystemsResearch Papers at Porto de Galinhas
Chair(s): Marcos Lordello Chaim
09:15
30m
Paper
Synthesizing Multi-threaded Tests from Sequential Traces to Detect Communication Deadlocks
Research Papers
09:45
30m
Paper
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
Research Papers
Shaukat AliSimula Research Laboratory, Norway, Paolo ArcainiNational Institute of Informatics , Xinyi Wang, Tao YueNanjing University of Aeronautics and Astronautics
10:45 - 12:00
Keynote 1Research Papers at Porto de Galinhas
Chair(s): Ana PaivaFaculty of Engineering of the University of Porto
10:45
75m
Keynote
Keynote Talk - Fuzzing, symbolic execution, and synthesis for testing
Research Papers
I: Corina S. PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center
13:00 - 14:30
13:00
30m
Demonstration
Tool Demonstration: iv4XR Agent-based Testing Framework
Tool Demos Track
DOI Media Attached
13:30
30m
Demonstration
Gamekins: Gamifying Software Testing in Jenkins
Tool Demos Track
Philipp StraubingerUniversity of Passau, Gordon FraserUniversity of Passau
14:00
30m
Demonstration
StackFul: Inter-Process Concolic Testing for Full-stack Web Applications
Tool Demos Track
Maarten VandercammenVrije Universiteit Brussel, Coen De RooverVrije Universiteit Brussel
13:00 - 14:30
Testing Tools IITesting Tools Track at Carneiros
13:00
30m
Paper
RiverFuzzRL - an open-source tool to experiment with reinforcement learning for fuzzing
Testing Tools Track
Ciprian PaduraruUniversity of Bucharest, Miruna Gabriela PaduraruUniversity of Bucharest , Alin StefanescuUniversity of Bucharest
13:30
30m
Paper
ROBY: a Tool for Robustness Analysis of Neural Network Classifiers
Testing Tools Track
Paolo ArcainiNational Institute of Informatics , Andrea BombardaUniversity of Bergamo, Silvia BonfantiUniversity of Bergamo, Angelo GargantiniUniversity of Bergamo
Pre-print
14:00
30m
Paper
Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantification
Testing Tools Track
Michael WeissUniversità della Svizzera Italiana (USI), Paolo TonellaUSI Lugano, Switzerland
Pre-print
13:00 - 14:30
Testing and LearningResearch Papers at Porto de Galinhas
Chair(s): Andrea StoccoUniversità della Svizzera italiana (USI)
13:00
30m
Paper
Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring
Research Papers
Michael WeissUniversità della Svizzera Italiana (USI), Paolo TonellaUSI Lugano, Switzerland
Pre-print
13:30
30m
Paper
A Search-Based Testing Framework for Deep Neural Networks of Source Code Embedding
Research Papers
Maryam Vahdat Pour, Zhuo Li, Lei MaUniversity of Alberta, Hadi HemmatiUniversity of Calgary
Pre-print
14:00
30m
Paper
Learning-Based Fuzzing of IoT Message Brokers
Research Papers
Bernhard AichernigGraz University of Technology, Edi Muskardin, Andrea PferscherInstitute of Software Technology, Graz University of Technology
Pre-print Media Attached File Attached
15:00 - 16:30
Journal First IJournal-First Papers at Boa Viagem
Chair(s): Heike WehrheimPaderborn University
15:00
30m
Paper
Adaptive metamorphic testing with contextual bandits
Journal-First Papers
Helge SpiekerSimula Research Laboratory, Norway, Arnaud GotliebSimula Research Laboratory
Link to publication DOI Pre-print
15:30
30m
Paper
Classifying generated white-box tests: an exploratory study
Journal-First Papers
Dávid Honfi, Zoltán MicskeiBudapest University of Technology and Economics
Link to publication DOI File Attached
16:00
30m
Paper
Hansie: Hybrid and Consensus Regression Test Prioritization
Journal-First Papers
Shouvick MondalFederal University of Pernambuco, Brazil, Rupesh NasreIIT Madras, India
Link to publication DOI Media Attached File Attached
15:00 - 16:30
Testing Tools IIITesting Tools Track at Carneiros
15:00
30m
Paper
Prut4j: Protocol Unit Testing fo(u)r Java
Testing Tools Track
Florian Joost SlobOpen University of the Netherlands, Sung-Shik JongmansOpen University of the Netherlands
15:30
30m
Paper
PatrIoT: IoT Automated Interoperability and Integration Testing Framework
Testing Tools Track
Miroslav BuresCzech Technical University, Czechia, Bestoun S. AhmedKarlstad University Sweden, Vaclav Rechtberger, Matej Klima, Michal Trnka, Miroslav Jaros, Xavier Bellekens, Dani Almog, Pavel HeroutUniversity of West Bohemia, Pilsen, CZ
Link to publication Pre-print
16:00
30m
Paper
STILE: a Tool for Parallel Execution of E2E WebTest Scripts
Testing Tools Track
Dario Olianas, Maurizio LeottaDIBRIS, University of Genova, Italy, Filippo Ricca, Matteo Biagiola, Paolo TonellaUSI Lugano, Switzerland
15:00 - 16:30
Models, Testing and VerificationResearch Papers at Porto de Galinhas
Chair(s): Mike PapadakisUniversity of Luxembourg, Luxembourg
15:00
30m
Paper
Modeling with Mocking
Research Papers
Jouke StoelCWI, Jurgen VinjuCWI, Netherlands, Tijs van der StormCWI & University of Groningen, Netherlands
Pre-print
15:30
30m
Paper
Uncertainty-aware Exploration in Model-based Testing
Research Papers
Matteo CamilliFree University of Bozen-Bolzano, Angelo GargantiniUniversity of Bergamo, Patrizia ScandurraUniversity of Bergamo, Italy, Catia TrubianiGran Sasso Science Institute
Pre-print
16:00
30m
Paper
Demystifying the Challenges of Formally Specifying API Properties for Runtime Verification
Research Papers
Leopoldo TeixeiraFederal University of Pernambuco, Breno MirandaFederal University of Pernambuco, Henrique RebeloUniversidade Federal de Pernambuco, Marcelo d'AmorimFederal University of Pernambuco
Pre-print
21:30 - 22:30
Industry Best Practices for CI/CDIndustry Track at Carneiros
21:30
60m
Keynote
Keynote Talk - Industry Best Practices for CI/CD
Industry Track
File Attached
22:30 - 00:00
22:30
30m
Paper
Assisting Bug Report Assignment Using Automated Fault Localisation: An Industrial Case Study
Industry Track
Jeongju SohnKAIST, South Korea, Gabin AnKAIST, Jingun HongSAP Labs, Dongwon HwangSAP Labs, Shin YooKorea Advanced Institute of Science and Technology
23:00
30m
Paper
Targeting Patterns of Driving Characteristics in Testing Autonomous Driving Systems
Industry Track
Paolo ArcainiNational Institute of Informatics , Xiaoyi ZhangNational Institute of Informatics, Japan, Fuyuki IshikawaNational Institute of Informatics
23:30
30m
Paper
How Does Fuzzing Techniques Perform in Real Industrial Settings?
Industry Track
Mingzhe WangTsinghua University, Jie Liang, Chijin Zhou, Yuanliang Chen, Zhiyong WuTsinghua University, China, Yu JiangTsinghua University

Conference Day
Wed 14 Apr

Displayed time zone: Brasilia, Distrito Federal, Brazil change

09:15 - 10:45
09:15
30m
Paper
Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types.
Industry Track
Daniil Stepanov, Marat AkhinSaint Petersburg Polytechnic University / JetBrains Research, Mikhail BelyaevSaint Petersburg Polytechnic University / JetBrains Research
09:45
30m
Paper
Automatically Checking Semantic Equivalence between Versions of Large-Scale C Projects.
Industry Track
Viktor Malík, Tomáš VojnarBrno University of Technology
10:15
30m
Paper
A Failed attempt at creating Guidelines for Visual GUI Testingtesting: An industrial case study.
Industry Track
Emil AlégrothBlekinge Institute of Technology, Elin Petersén, John Tinnerholm
09:15 - 10:45
Slicing and Static AnalysisResearch Papers at Porto de Galinhas
Chair(s): Leopoldo TeixeiraFederal University of Pernambuco
09:15
30m
Paper
Efficiently Finding Data Flow SubsumptionsDistinguished Paper Award
Research Papers
09:45
30m
Paper
MANDOLINE: Dynamic Slicing of Android Applications with Trace-Based Alias AnalysisDistinguished Paper Award
Research Papers
Khaled Ahmed, Mieszko Lis, Julia RubinUniversity of British Columbia, Canada
Pre-print
10:15
30m
Paper
Address-Aware Query Caching for Symbolic Execution
Research Papers
David TrabishTel Aviv University, Israel, Shachar ItzhakyTechnion, Noam Rinetzky
Pre-print
11:15 - 12:30
Keynote 2Research Papers at Porto de Galinhas
Chair(s): Fabiano FerrariFederal University of São Carlos
11:15
75m
Keynote
Keynote Talk - Some challenges and pitfalls in engineering contemporary software systems under the plague of defects
Research Papers
I: Guilherme Horta TravassosFederal University of Rio de Janeiro
12:30 - 13:00
12:30
10m
Awards
Award Session
Research Papers

12:40
10m
Awards
Assessing Oracle Quality with Checked CoverageMost Influential Paper Award
Research Papers
David Schuler, Andreas ZellerCISPA Helmholtz Center for Information Security
Link to publication DOI
12:50
10m
Awards
Experiences of System-Level Model-Based GUI Testing of an Android ApplicationMost Influential Paper Award
Research Papers
Tommi Takala, Mika Katara, Julian HartyCommercetest Limited
Link to publication DOI
14:00 - 15:45
Posters IPosters Track at Boa Viagem
Chair(s): Anna Rita FasolinoFederico II University of Naples, Sreedevi SampathUniversity of Maryland Baltimore County
14:00
35m
Poster
Fuzz Testing of Quantum Program
Posters Track
Jiyuan Wang, Fuchen MaTsinghua University, Yu JiangTsinghua University
14:35
35m
Poster
Repair Cross Browser Layout Issues by Combining Learning and Search-based Technique
Posters Track
Zhenyue Long, Guoquan WuInstitute of Software at Chinese Academy of Sciences, China, Yifei Zhang, Wei ChenInstitute of Software at Chinese Academy of Sciences, China, Jun WeiState Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences & University of Chinese Academy of Sciences
15:10
35m
Poster
Testing Complex Event Patterns with Narrowing and Constraint Solving
Posters Track
14:00 - 16:00
Industry IIIIndustry Track at Carneiros
14:00
30m
Paper
Industrial-Scale Passive Testing with T-EARS.
Industry Track
Daniel Flemström, Henrik Jonsson, Eduard Paul EnoiuMälardalen University, Wasif AfzalMälardalen University
14:30
30m
Paper
Boosting Exploratory Testing of Industrial Automation Systems with AI.
Industry Track
15:00
30m
Paper
Experiences from Large-Scale Model Checking: Verification of a Vehicle Control System.
Industry Track
Jonas FritzschUniversity of Stuttgart, Institute of Software Engineering, Tobias Schmid, Stefan WagnerUniversity of Stuttgart
Pre-print
15:30
30m
Paper
Digital Twins Are Not Monozygotic - Comparing ADAS Testing in Two Industry-Grade Automotive Simulators.
Industry Track
Markus BorgRISE Research Institutes of Sweden, Raja Ben Abdessalem, Shiva NejatiUniversity of Ottawa, Canada / University of Luxembourg, Luxembourg, Francois-Xavier Jegeden, Donghwan ShinUniversity of Luxembourg
14:00 - 15:00
Test ReuseResearch Papers at Porto de Galinhas
Chair(s): Paolo TonellaUSI Lugano, Switzerland
14:00
30m
Paper
Self determination: A comprehensive strategy for making automated tests more effective and efficient
Research Papers
Kesina Baral, Jeff OffuttGeorge Mason University, Fiza Mulla
14:30
30m
Paper
Artefact Relation Graphs for Unit Test Reuse Recommendation
Research Papers
Robert WhiteUniversity College London, UK, Jens KrinkeUniversity College London, Earl T. BarrUniversity College London, UK, Federica SarroUniversity College London, Chaiyong RakhitwetsagulMahidol University, Thailand
15:30 - 17:00
Faults and Fault InjectionResearch Papers at Porto de Galinhas
Chair(s): André T. EndoFederal University of Technology - Paraná (UTFPR)
15:30
30m
Paper
An Empirical Study of Flaky Tests in Python
Research Papers
Martin GruberBMW Group, Stephan LukasczykUniversity of Passau, Florian Kroiß, Gordon FraserUniversity of Passau
Pre-print
16:00
30m
Paper
Fast Kernel Error Propagation Analysis in Virtualized Environments
Research Papers
Nicolas CoppikTU Darmstadt, Oliver SchwahnTU Darmstadt, Neeraj Suri
16:30
30m
Paper
Dissecting Strongly Subsuming Second-Order Mutants
Research Papers
João Paulo DinizFederal University of Minas Gerais, Brazil, Chu-Pan WongCarnegie Mellon University, USA, Christian KaestnerCarnegie Mellon University, Eduardo FigueiredoFederal University of Minas Gerais, Brazil

Conference Day
Thu 15 Apr

Displayed time zone: Brasilia, Distrito Federal, Brazil change

09:15 - 10:45
Autonomous and Cyber-Physical SystemsResearch Papers at Porto de Galinhas
Chair(s): Paolo ArcainiNational Institute of Informatics
09:15
30m
Paper
IoTBox: Sandbox Mining to Prevent Interaction Threats in IoT Systems
Research Papers
Hong Jin Kang, David LoSingapore Management University, Sheng Qin Sim
09:45
30m
Paper
Quality Metrics and Oracles for Autonomous Vehicles Testing
Research Papers
Gunel JahangirovaUSI Lugano, Switzerland, Andrea StoccoUniversità della Svizzera italiana (USI), Paolo TonellaUSI Lugano, Switzerland
Pre-print
10:15
30m
Paper
Anomaly Detection with Digital Twin in Cyber-Physical Systems
Research Papers
Xu Qinghua, Shaukat AliSimula Research Laboratory, Norway, Tao YueNanjing University of Aeronautics and Astronautics
11:15 - 12:30
Keynote 3Research Papers at Porto de Galinhas
Chair(s): Robert HieronsUniversity of Sheffield
11:15
75m
Talk
Keynote Talk - Testing Machine Learning-Enabled Systems
Research Papers
I: Lionel BriandUniversity of Luxembourg and University of Ottawa
13:00 - 14:00
ICST Steering Committee meetingResearch Papers at Porto de Galinhas
13:00
60m
Meeting
ICST Steering Committee meeting
Research Papers

14:00 - 15:00
Program RepairResearch Papers at Porto de Galinhas
Chair(s): Angelo GargantiniUniversity of Bergamo
14:00
30m
Paper
Automatic Program Repair as Semantic Suggestions: An Empirical Study
Research Papers
Diogo Campos, André RestivoLIACC, Universidade do Porto, Porto, Portugal, Hugo Sereno FerreiraFaculty of Engineering, University of Porto, Portugal, Afonso RamosFaculty of Engineering of the University of Porto
14:30
30m
Paper
Exploring True Test Overfitting in Dynamic Automated Program Repair using Formal Methods
Research Papers
Amirfarhad NilizadehUniversity of Central Florida, Gary LeavensUniversity of Central Florida, Xuan-Bach D. LeSingapore Management University, Singapore, Corina S. PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center, David CokCEA, LIST, Software Safety and Security Laboratory
15:30 - 16:30
Journal First IIJournal-First Papers at Boa Viagem
Chair(s): Augusto SampaioUniversidade Federal de Pernambuco
15:30
30m
Paper
Search‐based crash reproduction using behavioural model seeding
Journal-First Papers
Pouria DerakhshanfarDelft University of Technology, Xavier DevroeyDelft University of Technology, Gilles PerrouinUniversité de Namur, Andy ZaidmanDelft University of Technology, Arie van DeursenDelft University of Technology, Netherlands
Link to publication DOI Media Attached
16:00
30m
Paper
A Federated Society of Bots for Smart Contract Testing
Journal-First Papers
Emanuele ViglianisiFBK, Mariano CeccatoUniversity of Verona, Paolo TonellaUSI Lugano, Switzerland
15:30 - 17:00
Empirical and User StudiesResearch Papers at Porto de Galinhas
Chair(s): Michael FeldererUniversity of Innsbruck
15:30
30m
Paper
A Large-scale Study on API Misuses in the Wild
Research Papers
Xia LiKennesaw State University, Jiajun JiangTianjin University, China, Samuel BentonThe University of Texas at Dallas, Yingfei XiongPeking University, Lingming ZhangUIUC
16:00
30m
Paper
System and Software Testing in Automotive: an Empirical Study on Process Improvement Areas
Research Papers
16:30
30m
Paper
Simulation for Robotics Test Automation: Developer Perspectives
Research Papers
Afsoon AfzalCarnegie Mellon University, Deborah S. KatzCarnegie Mellon University, Claire Le GouesCarnegie Mellon University, Christopher Steven TimperleyCarnegie Mellon University

Conference Day
Fri 16 Apr

Displayed time zone: Brasilia, Distrito Federal, Brazil change

09:00 - 09:10
Welcome and IntroductionDevOps4CPS-Testing at Muro Alto
09:00
10m
Talk
Welcome and Introduction
DevOps4CPS-Testing
Sebastiano PanichellaZurich University of Applied Sciences, Aitor ArrietaUniversity of Mondragon, Shaukat AliSimula Research Laboratory, Norway
09:00 - 09:15
Starting INTUITESTBEDSINTUITESTBEDS at Paiva
09:00
15m
Day opening
Welcome and Opening Message
INTUITESTBEDS

09:00 - 09:05
InSTA OpeningInSTA at Serrambi
09:00
5m
Talk
InSTA Opening
InSTA

09:00 - 09:10
Welcome to NEXTANEXTA at Tamandaré
09:00
10m
Talk
Welcome to NEXTA
NEXTA
Sigrid EldhEricsson, Sweden, Sahar TahviliEricsson AB, Vahid GarousiQueen's University Belfast, Michael FeldererUniversity of Innsbruck, Kristian SandahlLinköping University
09:10 - 09:50
Challenges of testing cyber-physical systems along the DevOps processDevOps4CPS-Testing at Muro Alto
09:10
40m
Keynote
Challenges of testing cyber-physical systems along the DevOps process
DevOps4CPS-Testing
Andreas UlrichSiemens AG
09:10 - 09:35
Active Machine Learning to Test Autonomous DrivingNEXTA at Tamandaré

Keynote Speaker: Karl Meinke Session Chair: Michael Felderer

09:10
25m
Keynote
Active Machine Learning to Test Autonomous Driving
NEXTA
Karl MeinkeThe Royal Institute of Technology
09:15 - 10:15
KeynoteINTUITESTBEDS at Paiva

Invited Keynote Dr. Mariano Ceccato University of Verona

Security Testing of Android Apps

Android facilitates apps interoperation and integration through inter-process communication mechanism, by allowing an app to request a task from another app that is installed on the same device. However, this interoperation mechanism poses security risks if an app does not implement it properly, such as permission re-delegation vulnerabilities, i.e., a form of privilege escalation where unprivileged malicious apps exploit vulnerable privileged apps to take privileged actions on the attacker behalf. Static analysis techniques as well as run-time protections have been proposed to detect permission re-delegation vulnerabilities. However, as acknowledged by their authors, most of these approaches are affected by many false positives because they do not discriminate between benign task requests and actual permission re-delegation vulnerabilities. In this keynote, we will present a recent approach aiming at filling this gap and at bridging static and dynamic analysis with security testing for precise detection of permission re-delegation vulnerabilities. Our approach first groups a large set of benign and non-vulnerable apps into different clusters, based on their similarities in terms of functional descriptions. It then generates permission re-delegation model for each cluster, which characterizes common permission re-delegation behaviors of the apps in the cluster. Given an app under test, our approach checks whether it has permission re-delegation behaviors that deviate from the model of the cluster it belongs to. If that is the case, it generates test cases to detect the vulnerabilities, that show how the vulnerabilities can be exploited. Empirical validation suggests that this security testing approach outperforms state-of-the-art in terms of vulnerability detection precision.

09:15
60m
Keynote
Security Testing of Android Apps
INTUITESTBEDS
Mariano CeccatoUniversity of Verona
09:25 - 09:45
InSTA Research PaperInSTA at Serrambi
09:25
20m
Paper
Syntax-Tree Similarity for Test-Case Derivability in Software Requirements
InSTA
Satoshi MasudaIBM Research, Tohru Matsuodani, Kazuhiko TsudaUniversity of Tsukuba Tokyo
09:35 - 10:00
AI-based Test Automation: A Grey Literature AnalysisNEXTA at Tamandaré

Authors: Filippo Ricca, DIBRIS, Università di Genova, Italy; Alessandro Marchetto, Independent Researcher, Italy and Andrea Stocco, Università della Svizzera italiana (USI), Switzerland Abstract: This paper provides the results of a survey of the grey literature concerning the use of artificial intelligence to improve test automation practices. We surveyed more than 1,200 sources of grey literature (e.g., blogs, white-papers, user manuals, StackOverflow posts) looking for highlights by professionals on how AI is adopted to aid the development and evolution of test code. Ultimately, we filtered 136 relevant documents from which we extracted a taxonomy of problems that AI aims to tackle, along with a taxonomy of AI-enabled solutions to such problems. Manual code development and automated test generation are the most cited problem and solution, respectively. The paper concludes by distilling the six most prevalent tools on the market, along with think-aloud reflections about the current and future status of artificial intelligence for test automation. Session Chair: Michael Felderer

09:35
25m
Paper
AI-based Test Automation: A Grey Literature Analysis
NEXTA
Filippo RiccaUniversità di Genova
09:45 - 10:05
InSTA Emerging IdeaInSTA at Serrambi
09:45
20m
Paper
Boosted Exploratory Test Architecture: Coaching Test Engineers with Word Similarity
InSTA
Yasuharu NishiUniversity of Electro-Communications, Tokyo, Yusuke ShibasakiUniversity of Electro-Communications
09:50 - 10:30
Keynote TBADevOps4CPS-Testing at Muro Alto

openCAESAR: a DevOps Platform for Systems Engineering by Maged Elaasar

10:00 - 10:25
Flaky Mutants; Another Concern for MutationTestingNEXTA at Tamandaré

Authors: Sten Vercammen, Serge Demeyer, Markus Borg and Robbe Claessens Abstract: Mutation testing is the state-of-the-art technique for assessing the fault detection capability of a test suite. An underlying assumption, rarely mentioned, is that the system under test behaves completely deterministically. This is rarely the case, as each mutant changes the code, it is highly likely that some introduce non-determinism. We call these flaky mutants. As they are only detected intermittently, they cause unreliable mutation testing scores, waste developer time, possibly unfruitful tests, and potential loss in confidence in the mutation testing technique. We want to raise awareness of this issue as we found that these flaky mutants are easy to create and occur in real projects. We also share some thoughts on how to tackle this issue. Mutation testing is the state-of-the-art technique for assessing the fault detection capability of a test suite. An underlying assumption, rarely mentioned, is that the system under test behaves completely deterministically. This is rarely the case, as each mutant changes the code, it is highly likely that some introduce non-determinism. We call these flaky mutants. As they are only detected intermittently, they cause unreliable mutation testing scores, waste developer time, possibly unfruitful tests, and potential loss in confidence in the mutation testing technique. We want to raise awareness of this issue as we found that these flaky mutants are easy to create and occur in real projects. We also share some thoughts on how to tackle this issue. Session Chair: Kristian Sandahl

10:00
25m
Keynote
Flaky Mutants; Another Concern for MutationTesting
NEXTA
Sten VercammenUniversity of Antwerp, Belgium, Serge DemeyerUniversity of Antwerp, Belgium, Markus BorgRISE Research Institutes of Sweden
10:05 - 10:25
InSTA Industry Experience ReportInSTA at Serrambi
10:05
20m
Paper
Multi-company Consumer Product Software Test Architecture Industry Experience Report
InSTA
Jon D. HagarGrand Software Testing, LLC, Hot Sulphur Springs, CO
10:25 - 10:30
InSTA ClosingInSTA at Serrambi
10:25
5m
Talk
InSTA Closing
InSTA

10:25 - 10:50
Using Advanced Code Analysis for Boosting Unit Test CreationNEXTA at Tamandaré

Authors: Miroslaw Zielinski, Parasoft Corporation, Poland and Rix Groenboom, Parasoft Corporation, Netherlands. Abstract: Unit testing is a popular testing technique, widespread in enterprise IT and embedded/safety-critical. For enterprise IT, unit testing is considered to be good practice and is frequently followed as an element of test-driven development. In the safety-critical world, there are many standards, such as ISO 26262, IEC 61508, and others, that either directly or indirectly mandate unit testing. Regardless the area of the application, unit testing is very time-consuming and teams are looking for strategies to optimize their efforts. This is especially true in the safety-critical space, where demonstration of test coverage is required for the certification. In this presentation, we share the results of our research regarding the use of advanced code analysis algorithms for augmenting the process of unit test creation. The discussion includes automatic discovery of inputs and responses from mocked components that maximize the code coverage and automated generation of the test cases. Session Chair: Kristian Sandahl

10:25
25m
Paper
Using Advanced Code Analysis for Boosting Unit Test Creation
NEXTA
10:30 - 11:00
INTUITESTBEDS Full research paper INTUITESTBEDS at Paiva
10:30
30m
Paper
Model-based Automated Testing of Mobile Applications: An Industrial Case Study
INTUITESTBEDS
Stefan KarlssonABB AB, Mälardalen University
Pre-print
11:00 - 11:30
Test Automation with Grad-CAM Heatmaps - A Future Standard Block in MLOps for Vision AI?DevOps4CPS-Testing at Muro Alto
11:00 - 11:30
INTUITESTBEDS Full research paper INTUITESTBEDS at Paiva
11:00
30m
Paper
Improving Mobile User Interface Testing with Model Driven Monkey Search
INTUITESTBEDS
11:00 - 11:25
QRTest: Automatic Query Reformulation for Information Retrieval Based Regression Test Case PrioritizationNEXTA at Tamandaré

Author: Maral Azizi, East Carolina University, US. Abstract: The most effective regression testing algorithms have long running times and often require dynamic or static code analysis, making them unsuitable for the modern software development environment where the rate of software delivery could be less than a minute. More recently, some researchers have developed information retrieval-based (IR-based) techniques for prioritizing tests such that the higher similar tests to the code changes have a higher likelihood of finding bugs. A vast majority of these techniques are based on standard term similarity calculation, which can be imprecise. One reason for the low accuracy of these techniques is that the original query often is short, therefore, it does not return the relevant test cases. In such cases, the query needs reformulation. The current state of research lacks methods to increase the quality of the query in the regression testing domain. Our research aims at addressing this problem and we conjecture that enhancing the quality of the queries can improve the performance of IR-based regression test case prioritization (RTP). Our empirical evaluation with six open source programs shows that our approach improves the accuracy of IR-based RTP and increases regression fault detection rate, compared to the common prioritization techniques. Session Chair: Sahar Tahvili

11:00
25m
Talk
QRTest: Automatic Query Reformulation for Information Retrieval Based Regression Test Case Prioritization
NEXTA
11:25 - 11:50
An Empirical Study of Parallelizing Test Execution Using CUDA Unified Memory and OpenMP GPU OffloadingNEXTA at Tamandaré

Authors: Taghreed Bagies and Ali Jannesari, Lowa State University, US. Abstract: The execution of software testing is costly and time-consuming. To accelerate the test execution, researchers have applied several methods to run the testing in parallel. One method of parallelizing the test execution is by using a GPU to distribute test case inputs among several threads running in parallel. In this paper, we investigate three programming models CUDA Unified Memory, CUDA Non-Unified Memory, and OpenMP GPU offloading to parallelize the test execution and discuss the challenges using these programing models. We use eleven benchmarks and parallelize their test suites by using these models. We evaluate their performance in terms of execution time, analyze the results, and report the limitations of using these programming models. Session Chair: Vahid Garousi

11:25
25m
Paper
An Empirical Study of Parallelizing Test Execution Using CUDA Unified Memory and OpenMP GPU Offloading
NEXTA
Taghreed BagiesKing Abdulaziz University
11:30 - 12:30
Panel and Open DiscussionDevOps4CPS-Testing at Muro Alto
11:30 - 11:50
INTUITESTBEDS Position paper INTUITESTBEDS at Paiva
11:30
20m
Paper
A Metric Framework for the Gamification of Web and Mobile GUI Testing
INTUITESTBEDS
Riccardo CoppolaPolitecnico di Torino
11:50 - 12:15
Closing INTUITESTBEDSINTUITESTBEDS at Paiva
11:50
25m
Day closing
Open Discussion and Closing
INTUITESTBEDS

11:50 - 12:20
Advancing Test Automation Using Artificial Intelligence (AI)NEXTA at Tamandaré

Keynote speaker: Jeremy S. Bradbury, PhD, Associate Professor, Computer Science, Associate Dean, School of Graduate and Postdoctoral Studies Ontario Tech University. Abstract: In recent years, software testing automation has been enhanced through the use of Artificial Intelligence (AI) techniques including genetic algorithms, machine learning and deep learning. The use cases for AI in test automation range from providing recommendations to the complete automation of software testing activities. To demonstrate the breadth of application, I will present several recent examples of how AI can be leveraged to support automated testing in rapid release cycles. Furthermore, I will discuss my own successes and failures in using AI to advance test automation as well as share the lesson I have learned. Session Chair: Sahar Tahvili

11:50
30m
Keynote
Advancing Test Automation Using Artificial Intelligence (AI)
NEXTA
Jeremy BradburyOntario Tech University
12:00 - 13:30
Session ICCIW at Maracaípe
12:00
30m
Talk
Welcome to CCIW
CCIW
12:30
30m
Talk
Code Quality Management System Through CI
CCIW
Raja KumarSamsung R&D Institute India , Sasikumar SathasivamSamsung R&D Institute India
File Attached
13:00
30m
Talk
Continuous Build Monitoring System
CCIW
File Attached
12:20 - 12:30
ClosingNEXTA at Tamandaré
12:20
10m
Talk
Closing
NEXTA

13:30 - 14:50
ADEPTNESS and COSMOS sessionDevOps4CPS-Testing at Muro Alto
14:50 - 15:00
15:30 - 16:00
Session IIICCIW at Maracaípe
15:30
30m
Talk
Open Discussion
CCIW