ICST 2021
Mon 12 - Fri 16 April 2021

Events

Title
A Failed attempt at creating Guidelines for Visual GUI Testingtesting: An industrial case study.
Industry Track
Applicable Micropatches and Where to Find Them: Finding and Applying New Security Hot Fixes to Old Software.
Industry Track
Assisting Bug Report Assignment Using Automated Fault Localisation: An Industrial Case Study
Industry Track
Automatically Checking Semantic Equivalence between Versions of Large-Scale C Projects.
Industry Track
Boosting Exploratory Testing of Industrial Automation Systems with AI.
Industry Track
Digital Twins Are Not Monozygotic - Comparing ADAS Testing in Two Industry-Grade Automotive Simulators.
Industry Track
Experiences from Large-Scale Model Checking: Verification of a Vehicle Control System.
Industry Track
Pre-print
Exploring the Triggering Modes of Spectrum-Based Fault Localization: An Industrial Case.
Industry Track
How Does Fuzzing Techniques Perform in Real Industrial Settings?
Industry Track
Industrial-Scale Passive Testing with T-EARS.
Industry Track
Keynote Talk - Industry Best Practices for CI/CD
Industry Track
File Attached
Targeting Patterns of Driving Characteristics in Testing Autonomous Driving Systems
Industry Track
Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types.
Industry Track

Call for Papers

ICST 2021 invites high quality submissions in all areas of software testing, verification, and validation. The industry track has the same topics and requires the same rigorousness as the research track. However, the industry track papers should prioritize impact and realism over novelty. We expect that industry track publications report useful testing tools, successful stories and experiences when applying software testing theories and methodologies into practice, or promising prototypes that have the potential to impact practice in the next few years. Ideally they should be evaluated on real-world, industrial data, and discuss non-functional aspects such as scalability and reliability, and also compare to pre-existing industrial best practices. The industry track papers will be published in the ICST 2021 proceedings and listed by IEEE databases.

Topics of interest include, but is not limited to, the following:

  • Domain specific testing, such as: Autonomous vehicles, advanced control systems, mobile or web app testing, embedded systems testing, concurrent systems, GUI testing, etc.
  • Formal verification
  • Model based testing
  • Model checking
  • Search based software testing
  • Security testing
  • Software reliability
  • Test automation
  • Testability and design
  • Testing and development processes
  • Testing education
  • Testing / debugging frameworks and tools
  • Applications of software testing theories in practice
  • Empirical studies
  • Experience reports
  • Mobile testing
  • Testing AI products
  • Applying machine learning techniques to testing

Each submission will be reviewed by at least three members of the ICST Program Committee.

Format

Industry papers must conform to the two-column IEEE conference publication format. Templates for LaTeX and Microsoft Word are available from http://www.ieee.org/conferences_events/conferences/publishing/templates.html: please use the letter format template and conference option.

Papers should be submitted in the PDF format: they must not exceed ten pages, plus up to two pages of references. Submissions will be handled via EasyChair https://easychair.org/conferences/?conf=icst2021.

Papers must neither have been previously accepted for publication nor be under submission in another conference or journal.

For your paper to be published in the ICST 2021 conference proceedings, at least one of the authors of the paper must register for the conference and confirm that she/he will present the paper in person.

Important Dates (all dates are AoE)

Submission of Abstract: October 12, 2020

Submission of Industry Papers: October 18, 2020

Author Notification: December 11, 2020

Camera Ready: January 31, 2021

Double Blind

Industry track papers do not need to be double-blinded. Since industry papers typically rely heavily on the industrial/practical context in which the work was carried out it would be too much to ask to require this context to be anonymized. Please contact the industry track chairs if you have more questions.

FAQ

Q: Does the industry track accept abstracts or short papers?

A: No. The industry track has the same rigorousness as the research papers, therefore, the industry track papers need to present the motivation, methodology, design of experiments / case studies, results, discussions, and related work in a clear and comprehensive way. Abstracts or short papers don't have enough room to incorporate these required and important contents.

Q: My paper has less than 10 pages, can I submit it to the industry track?

A: Yes, as long as your paper clearly presents the motivation, methodology, design of experiments / case studies, results, discussions, and related work.

Dates
Plenary
You're viewing the program in a time zone which is different from your device's time zone change time zone

Tue 13 Apr

Displayed time zone: Brasilia, Distrito Federal, Brazil change

09:00 - 09:15
09:00
15m
Day opening
Intro & Welcome
Research Papers

21:30 - 22:30
Industry Best Practices for CI/CDIndustry Track at Carneiros
21:30
60m
Keynote
Keynote Talk - Industry Best Practices for CI/CD
Industry Track
John Micco VMware
File Attached
22:30 - 00:00
22:30
30m
Paper
Assisting Bug Report Assignment Using Automated Fault Localisation: An Industrial Case Study
Industry Track
Jeongju Sohn KAIST, South Korea, Gabin An KAIST, Jingun Hong SAP Labs, Dongwon Hwang SAP Labs, Shin Yoo Korea Advanced Institute of Science and Technology
23:00
30m
Paper
Targeting Patterns of Driving Characteristics in Testing Autonomous Driving Systems
Industry Track
Paolo Arcaini National Institute of Informatics , Xiao-Yi Zhang National Institute of Informatics, Japan, Fuyuki Ishikawa National Institute of Informatics
23:30
30m
Paper
How Does Fuzzing Techniques Perform in Real Industrial Settings?
Industry Track
Mingzhe Wang Tsinghua University, Jie Liang , Chijin Zhou , Yuanliang Chen , Zhiyong Wu Tsinghua University, China, Yu Jiang Tsinghua University

Wed 14 Apr

Displayed time zone: Brasilia, Distrito Federal, Brazil change

09:15 - 10:45
09:15
30m
Paper
Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types.
Industry Track
Daniil Stepanov , Marat Akhin Saint Petersburg Polytechnic University / JetBrains Research, Mikhail Belyaev Saint Petersburg Polytechnic University / JetBrains Research
09:45
30m
Paper
Automatically Checking Semantic Equivalence between Versions of Large-Scale C Projects.
Industry Track
Viktor Malík , Tomáš Vojnar Brno University of Technology
10:15
30m
Paper
A Failed attempt at creating Guidelines for Visual GUI Testingtesting: An industrial case study.
Industry Track
Emil Alégroth Blekinge Institute of Technology, Elin Petersén , John Tinnerholm
14:00 - 16:00
Industry IIIIndustry Track at Carneiros
14:00
30m
Paper
Industrial-Scale Passive Testing with T-EARS.
Industry Track
Daniel Flemström , Henrik Jonsson , Eduard Paul Enoiu Mälardalen University, Wasif Afzal Mälardalen University
14:30
30m
Paper
Boosting Exploratory Testing of Industrial Automation Systems with AI.
Industry Track
15:00
30m
Paper
Experiences from Large-Scale Model Checking: Verification of a Vehicle Control System.
Industry Track
Jonas Fritzsch University of Stuttgart, Institute of Software Engineering, Tobias Schmid , Stefan Wagner University of Stuttgart
Pre-print
15:30
30m
Paper
Digital Twins Are Not Monozygotic - Comparing ADAS Testing in Two Industry-Grade Automotive Simulators.
Industry Track
Markus Borg RISE Research Institutes of Sweden, Raja Ben Abdessalem , Shiva Nejati University of Ottawa, Canada / University of Luxembourg, Luxembourg, Francois-Xavier Jegeden , Donghwan Shin University of Luxembourg
  • A Failed attempt at creating Guidelines for Visual GUI Testingtesting: An industrial case study. Emil Alégroth, Elin Petersén and John Tinnerholm

  • Applicable Micropatches and Where to Find Them: Finding and Applying New Security Hot Fixes to Old Software. Mac Malone, Yicheng Wang, Kevin Snow and Fabian Monrose

  • Assisting Bug Report Assignment Using Automated Fault Localisation: An Industrial Case Study. Jeongju Sohn, Gabin An, Jingun Hong, Dongwon Hwang and Shin Yoo

  • Automatically Checking Semantic Equivalence between Versions of Large-Scale C Projects. Viktor Malík and Tomas Vojnar

  • Boosting Exploratory Testing of Industrial Automation Systems with AI.Raphael Eidenbenz, Carsten Franke, Thanikesavan Sivanthi and Sandro Schoenborn

  • Digital Twins Are Not Monozygotic - Comparing ADAS Testing in Two Industry-Grade Automotive Simulators. Markus Borg, Raja Ben Abdessalem, Shiva Nejati, Francois-Xavier Jegeden and Donghwan Shin

  • Experiences from Large-Scale Model Checking: Verification of a Vehicle Control System. Jonas Fritzsch, Tobias Schmid and Stefan Wagner

  • Exploring the Triggering Modes of Spectrum-Based Fault Localization: An Industrial Case. Tung Dao, Max Wang and Na Meng

  • How Does Fuzzing Techniques Perform in Real Industrial Settings? Mingzhe Wang, Jie Liang, Chijin Zhou, Yuanliang Chen, Zhiyong Wu and Yu Jiang

  • Industrial-Scale Passive Testing with T-EARS. Daniel Flemström, Henrik Jonsson, Eduard Paul Enoiu and Wasif Afzal

  • Targeting Patterns of Driving Characteristics in Testing Autonomous Driving Systems. Paolo Arcaini, Xiao-Yi Zhang and Fuyuki Ishikawa

  • Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types. Daniil Stepanov, Marat Akhin and Mikhail Belyaev

Industry Best Practices for CI/CD

John Micco, VMware

Abstract: Industrial best practices for Continuous Integration (CI) and Continuously Delivery (CD) are constantly evolving, and the state of the art is advancing quickly across the industry. In this talk I will discuss the best practices that have been implemented by many top software companies across the industry including Google, Netflix and VMWare where I have worked. I will discuss how these systems are being optimized to reduce human and machine resources required to qualify software releases and reduce the risk of latent defects.

Bio: John is a Cloud Transformation Architect at VMWare working on Continuous Integration / Delivery for VMWare’s flagship vSphere product line. He has more than 15 years of experience working on CI/CD systems at Mathworks, Google and Netflix. He has given several keynote addresses at industry and academic conferences on Software testing and software engineering. He is one of the organizers of the Google Journal Club - which does monthly reviews of Academic papers on Continuous Integration and software delivery.

ICST 2021 invites high quality submissions in all areas of software testing, verification, and validation. The industry track has the same topics and requires the same rigorousness as the research track. However, the industry track papers should prioritize impact and realism over novelty. We expect that industry track publications report useful testing tools, successful stories and experiences when applying software testing theories and methodologies into practice, or promising prototypes that have the potential to impact practice in the next few years. Ideally they should be evaluated on real-world, industrial data, and discuss non-functional aspects such as scalability and reliability, and also compare to pre-existing industrial best practices. The industry track papers will be published in the ICST 2021 proceedings and listed by IEEE databases.

Topics of interest include, but is not limited to, the following:

  • Domain specific testing, such as: Autonomous vehicles, advanced control systems, mobile or web app testing, embedded systems testing, concurrent systems, GUI testing, etc.
  • Formal verification
  • Model based testing
  • Model checking
  • Search based software testing
  • Security testing
  • Software reliability
  • Test automation
  • Testability and design
  • Testing and development processes
  • Testing education
  • Testing / debugging frameworks and tools
  • Applications of software testing theories in practice
  • Empirical studies
  • Experience reports
  • Mobile testing
  • Testing AI products
  • Applying machine learning techniques to testing

Each submission will be reviewed by at least three members of the ICST Program Committee.

Format

Industry papers must conform to the two-column IEEE conference publication format. Templates for LaTeX and Microsoft Word are available from http://www.ieee.org/conferences_events/conferences/publishing/templates.html: please use the letter format template and conference option.

Papers should be submitted in the PDF format: they must not exceed ten pages, plus up to two pages of references. Submissions will be handled via EasyChair https://easychair.org/conferences/?conf=icst2021.

Papers must neither have been previously accepted for publication nor be under submission in another conference or journal.

For your paper to be published in the ICST 2021 conference proceedings, at least one of the authors of the paper must register for the conference and confirm that she/he will present the paper in person.

Important Dates (all dates are AoE)

Submission of Abstract: October 12, 2020

Submission of Industry Papers: October 18, 2020

Author Notification: December 11, 2020

Camera Ready: January 31, 2021

Double Blind

Industry track papers do not need to be double-blinded. Since industry papers typically rely heavily on the industrial/practical context in which the work was carried out it would be too much to ask to require this context to be anonymized. Please contact the industry track chairs if you have more questions.

FAQ

Q: Does the industry track accept abstracts or short papers?

A: No. The industry track has the same rigorousness as the research papers, therefore, the industry track papers need to present the motivation, methodology, design of experiments / case studies, results, discussions, and related work in a clear and comprehensive way. Abstracts or short papers don't have enough room to incorporate these required and important contents.

Q: My paper has less than 10 pages, can I submit it to the industry track?

A: Yes, as long as your paper clearly presents the motivation, methodology, design of experiments / case studies, results, discussions, and related work.