ICST 2021
Mon 12 - Fri 16 April 2021

All information about this workshop can also be found at the workshop’s official website: http://gist.nju.edu.cn/iwct2021/

Combinatorial Testing (CT) is a widely applicable generic methodology and technology for software verification and validation. In a combinatorial test plan, all interactions between parameters up to a certain level are covered. Studies show that CT can significantly reduce the number of test cases while remaining very effective for fault detection. This workshop aims to bring together researchers, developers, users, and practitioners to discuss and exchange ideas and experiences in the development and application of CT methods, techniques, and tools. We invite submissions of high-quality papers presenting original work on both theoretical and practical aspects of combinatorial testing.

In this workshop, we plan to bring together researchers actively working on combinatorial testing, and create a productive and creative environment for sharing and collaboration. Since this workshop was the first venue dedicated to CT, yet there are many researchers working in the field, we expect, like in previous years, to see high responsiveness to take part in the workshop. Researchers attending the workshop will have an opportunity to publish their work in a dedicated venue, create new collaborations and take active part in the growing community of researchers working in the field.

The workshop will also be a meeting place between academia and industry, thus uniting academic excellence and industrial experience and needs. This will allow participants from academia to learn about the industrial experience in practical application of CT to real-life testing problems, and together with the colleagues from industry identify obstacles for future research. Industrial participants will have an opportunity to meet the leading scientists in the field, and hear about the latest advances and innovations.

You're viewing the program in a time zone which is different from your device's time zone - change time zone

Conference Day
Mon 12 Apr

Displayed time zone: Brasilia, Distrito Federal, Brazil change

09:00 - 09:15
Welcome and Opening MessageIWCT at Serrambi
09:00
15m
Day opening
Welcome and Opening Message
IWCT
Dimitris SimosSBA Research
09:15 - 10:15
IWCT IIWCT at Serrambi
09:15
30m
Paper
A Combinatorial Approach to Explaining Image Classifiers
IWCT
Jaganmohan ChandrasekaranUniversity of Texas at Arlington, USA, Yu Lei, Raghu KackerNational Institute of Standards and Technology, Rick KuhnNational Institute of Standards and Technology
09:45
15m
Short-paper
Combinatorial Testing Metrics for Machine Learning
IWCT
Erin LanusVirginia Tech, Laura J. Freeman, Rick KuhnNational Institute of Standards and Technology, Raghu KackerNational Institute of Standards and Technology
10:00
15m
Short-paper
Computing Sequence Covering Arrays using Unified Combinatorial Interaction Testing
IWCT
Hanefi Mercan, Cemal YilmazSabancı University
10:45 - 11:45
IWCT IIIWCT at Serrambi
10:45
30m
Paper
A Combinatorial Approach to Testing Deep Neural Network-based Autonomous Driving Systems
IWCT
Jaganmohan ChandrasekaranUniversity of Texas at Arlington, USA, Yu Lei, Raghu KackerNational Institute of Standards and Technology, Rick KuhnNational Institute of Standards and Technology
11:15
30m
Paper
SYSMODIS: A Systematic Model Discovery Approach
IWCT
Omer KorkmazSabancı University, Cemal YilmazSabancı University
12:45 - 14:00
IWCT IIIIWCT at Serrambi
12:45
15m
Short-paper
A Practical Method for API Testing in the Context of Continuous Delivery and Behavior Driven Development
IWCT
13:00
30m
Paper
An environment for benchmarking combinatorial test suite generators
IWCT
Andrea BombardaUniversity of Bergamo, Edoardo CrippaUniversity of Bergamo, Angelo GargantiniUniversity of Bergamo
Pre-print
13:30
30m
Paper
Combinatorially XSSing Web Application Firewalls
IWCT
Bernhard Garn, Daniel Sebastian Lang, Manuel Leithner, Rick KuhnNational Institute of Standards and Technology, Raghu KackerNational Institute of Standards and Technology, Dimitris SimosSBA Research
14:30 - 15:00
IWCT Organizing and Steering Committee MeetingIWCT at Serrambi

Not scheduled yet

Not scheduled yet
Meeting
IWCT Organizing and Steering Committee Meeting
IWCT

Call for Papers

The call for papers for IWCT 2021 can be found also at: http://gist.nju.edu.cn/iwct2021/

Submission Guidelines

Full and Short Papers: We invite submissions of high-quality papers presenting original work on both theoretical and practical aspects of combinatorial testing. We accept both full papers (up to 10 pages) and short papers (up to 4 pages).

Poster Session: We will also have a poster session, for authors to present their work in an informal and interactive setting. We encourage submissions presenting research work in progress or important conclusions from practical experience.

In particular, we welcome posters focusing on

  • experience reports from integration of CT by industrial organizations
  • lessons learned from using CT tools in industrial applications

An extended abstract of the poster (up to 2 pages) should be submitted for review by the submission deadline. Accepted poster abstracts will be included in the proceedings.

Submission Server:

https://easychair.org/conferences/?conf=iwct2021

Proceedings and Best Paper Awards

  • Accepted (full and short) papers as well as extended abstracts will be published in the IEEE Digital Library.
  • This edition of IWCT will feature two best paper awards in the categories of “best foundation/modeling paper” and “best CT application paper”.

Topics

Topics of interest for full and short papers include, but are not limited to:

Combinatorial testing workflow

  • Modeling the input space for CT
  • Efficient algorithms to generate t-way test suites, especially involving support of constraints
  • Determination of expected system behavior for each test case
  • Executing CT test suites
  • Combinatorial testing based fault localization
  • Implementation of CT with existing testing infrastructures
  • Handling changes in test requirements

Real-world experience in deployment of combinatorial testing

  • Empirical studies and feedback from practical applications of CT
  • Evaluation and return of investment metrics to assess the degree of usefulness of CT
  • Methodology used for test space modeling and determination of interaction coverage requirements
  • Discussion of challenges and open problems in the application of CT in industrial settings

Applicability of combinatorial testing

  • Comparison and combination of CT with other dynamic verification methods
  • Investigation of historical records of failures to determine the kind of CT which may have detected faults
  • Combinatorial testing for concurrent and real-time systems
  • CT for testing cloud computing systems and use of combinatorial methods in cloud architecture
  • Application of CT in other domains, e.g. information security, study of gene regulation and other biotechnology applications, mechanical engineering, etc.
  • Combinatorial testing of variability models for software product lines

Combinatorial and complementing methods

  • Combinatorial analysis of existing test suites
  • Test plan reduction and completeness
  • CT and coverage metrics – combining the two, and studying the relationship between them

Questions? Use the IWCT contact form.