Overview

Latest updates:

• Sept 30, 2024: The shortlisted teams have been invited for the paper submission. Please check your emails for instructions!
• Sept 30, 2024: The winners based on Phase 2 have been declared. Final list to be announced after paper submission round.
• Sept 15, 2024: The submissions have been closed. Thank you for your participation!
• Sept 7, 2024: Submission links for phase 3 (Report and Source Code) have been enabled!
• Aug 26, 2024: The submission site has been upgraded to support ensemble learning and the upload of multiple models for Phase 2.
• Aug 17, 2024: The deadline for Phase 1 and 2 has been extended to Sept 10th, 2024.
• Aug 3, 2024: The evaluation criterion for Phase 1 and 2 has been updated.
• Aug 2, 2024: Feedback Form Link has been added.
• Aug 2, 2024: The details for Phase 2 has been added. Please check the submission page.
• July 3, 2024: The RDD2022 data article, published in Geoscience Data Journal is now available online.
• June 7, 2024: Please use this form to provide the details of libraries required to run your model for Phase 2!
• June 4, 2024: The submissions for Phase 2 task will be enabled after June. Participants may prepare the list of requirements to run their model at organizers' site in advance - collection form to be released soon
• May 15, 2024: The submissions for phase 1 started.
• May 10, 2024: Web site of the challenge opens, the task is revealed.

Description

Abstract: The challenge addresses the problem of automating the road damage detection (RDD) targeting the optimized inference speed and resource usage. So far, the RDD challenges have prioritized enhancing the performance of RDD algorithms/models, with the F1-score serving as the primary (and sole) metric. However, moving forward, it has become increasingly crucial to address resource optimization concerns, particularly regarding inference speed and memory usage, to enable real-time deployment of these models. Therefore, the current challenge shifts the primary criterion towards optimizing resource usage. The background details are provided as follows.

Details: Considering that roads have a direct and significant impact on people's lives, maintenance and management of roads need to be done exhaustively from time to time. However, a lack of financial resources makes many local governments unable to conduct sufficient inspections on time. Some municipalities automate road damage detection by using high-performance sensors. However, the high cost of these sensors makes it infeasible to use them at the country level owing to the vast area of roads to be inspected.

Therefore, there arises a need for a system that makes it easy to assess the road conditions and identify the damage of the road surface at a low cost. A preliminary version of this system was developed, and a challenge was hosted in 2018 (IEEE BigData Cup) to evaluate the contemporary methods working towards the same goal.

After the road damage detection challenge in 2018, several municipalities in Japan started utilizing our automatic road damage detection systems. Through practical use and feedback from several cities, it was realized that the algorithms need to be more robust, especially in a real-world scenario. The algorithm is required to perform well in a variety of situations, including the presence of shadows or reflections in the images. Further, most existing models were limited to road conditions in a single country. Proposing a method that applies to more than one country may lead to the possibility of designing a stand-alone system for road damage detection all over the world. Recognizing this need, another data cup – Global Road Damage Detection Challenge - GRDDC'2020, was organized in 2020.

Compared to the dataset in 2018, in 2020, the dataset volume was increased three times. The challenge was detecting and identifying the damage in road images captured by a vehicle-mounted smartphone. Since the dataset included complicated negative and positive examples collected in real-world situations from three countries (India, Japan, and Czech Republic), the algorithm was expected to have more robustness to meet the challenge.

Next, in 2022, another data cup– Crowd Sensing-based Road Damage Detection challenge, CRDDC’2022, was organized to further improve the performance and robustness of the RDD algorithms. Unlike previous challenges, which invited participants to contribute the models, CRDDC also called the participants for contributing their proposed dataset in the initial competition track, thereby enhancing the diversity and representativeness of the training data available to the researchers in final track.

This year, another challenge is here with new opportunities. The challenge ORDDC’2024 utilizes the 6-country dataset, (Two Subsets – train annotation: public; test annotation: private), coupled with a renewed focus on resource optimization (primarily inference speed), presenting new avenues for innovation in road damage detection. Successful implementation of this challenge holds the promise of revolutionizing road inspection processes worldwide. With the potential for smartphone and drive camera-based solutions to suffice for road inspections across different countries, we envision a future where road maintenance becomes more accessible and efficient on a global scale.

The Task

The challenge will be organized in three phases with different tasks for the participants. The details are as follows.

Phase 1: Accuracy Maximization: Qualifying round: Teams with an F1-score greater than 70% qualify for the top 3 positions in the competition.

Participants are invited to propose an algorithm that can automatically recognize the road damages present in an image captured from any underlying countries: India, Japan, Czech Republic, Norway, United States, and China (based on RDD’2022). Recognition refers to detecting the damage location in the image and identifying the damage type.

The challenge imposes no restrictions on the type of submission and welcomes all the novel algorithms, techniques that are currently under review, and methods that have already been published. Participants may also utilize the strategies from CRDDC’2022.

Phase 2: Resource Optimization:

Participants need to optimize the inference speed of the models proposed in Phase 2, while maintaining an F1-score of more than 70%. (Expert feedback is sought to ensure that the evaluation criteria effectively capture model performance and to consider any necessary refinements. Feedback Form Link)

Phase 3: Report and Source Code: In phase 3, the participants need to submit a detailed report of their proposed solution along with the source code. The submission will be used to finalize the winners.

Note: After the competition phase is completed, a link for submitting the accompanying academic paper will be provided to the top 10 participants (may be increased based on the quality of submissions) as ranked by the final leaderboard. For more details, please refer to the Submissions section.

Useful Links including the previous Road Damage Detection Challenges (CRDDC’2022 and GRDDC’2020):

1. GitHub Website for latest news and updates
2. IEEE BigData'2024 Conference Website
3. Previous Challenge Summary papers: 2020, 2022
4. Detailed Analysis Papers: 2021, 2024
5. Data Articles: 2021, 2022
2024
6. Data Download links: 2018, 2019, 2020, 2022
7. Related Research Articles: RDD2018, RDD2019, RDD2020, RDD2022

Citations

If you use the resources (data, content,…), please cite the following:

1. Arya, D., Maeda, H., & Sekimoto, Y. (2024). From Global Challenges to Local Solutions: A Review of Cross-country Collaborations and Winning Strategies in Road Damage Detection. Advanced Engineering Informatics, 60, 102388. https://doi.org/10.1016/j.aei.2024.102388
2. Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., Omata, H., Kashiyama, T., & Sekimoto, Y. (2022a). Crowdsensing-based Road Damage Detection Challenge (CRDDC’2022). In 2022 IEEE International Conference on Big Data (Big Data) (pp. 6378-6386). IEEE. https://doi.org/10.1109/BigData55660.2022.10021040
3. Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., & Sekimoto, Y. (2024). RDD2022: A multi-national image dataset for automatic Road Damage Detection. Geoscience Data Journal. https://doi.org/10.1002/gdj3.260
4. Arya, D., Maeda, H., Sekimoto, Y., Omata, H., Ghosh, S. K., Toshniwal, D., ... & Kashiyama, T. (2022c). RDD2022-The multi-national Road Damage Dataset released through CRDDC’2022. figshare, https://doi.org/10.6084/m9.figshare.21431547.v1
5. Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., Mraz, A., Kashiyama, T., & Sekimoto, Y. (2021a). Deep learning-based road damage detection and classification for multiple countries. Automation in Construction, 132, 103935. https://doi.org/10.1016/j.autcon.2021.103935
6. Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., & Sekimoto, Y. (2021b). RDD2020: An annotated image dataset for Automatic Road Damage Detection using Deep Learning. Data in Brief, 107133. https://doi.org/10.1016/j.dib.2021.107133.
7. Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., Omata, H., Kashiyama, T., & Sekimoto, Y. (2020). Global Road Damage Detection: State-of-the-art Solutions. IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 2020, pp. 5533-5539, https://doi.org/10.1109/BigData50022.2020.9377790
8. Maeda, H., Kashiyama, T., Sekimoto, Y., Seto, T., & Omata, H. (2021). Generative adversarial network for road damage detection. Computer‐Aided Civil and Infrastructure Engineering, 36(1), 47-60.

Important Dates

Tentative Timeline:

Important dates: