Gilbert Tanner / a year ago /

SAPIENCE - Sense & Avoid - a cooPeratIvE droNe CompEtition

Through a series of collaborative competitions, the Sapience initiative fosters innovation in search and rescue operations, enabling multiple drones to effectively navigate and map GPS-denied environments, detect and deliver aid to victims, and perform complex cooperative tasks.

Competition 1: Indoor Mapping and Object Detection

Tasks:

Mapping the Environment - Construct a map using two drones, navigating various hazards.
Object Detection - Detect and note positions of objects like hazardous containers and injured mannequins.
Aid Delivery - Deliver aid packages to mannequins at specific locations.

Team Klagenfurt:

Gilbert Tanner
Luca Di Pierno
Georg Steinthaler
Tim Schumann
Ben Wesse
Jonas Spieler
Supervisor: Prof. Stephan Weiss

The competition occurred on 29th-30th August 2024 in London and Team Klagenfurt was able to solve all three tasks as specified above. To navigate the indoor environment we made use of ICP odometry based on RTABMAP fusing PointCloud data from the 3d lidar and depth camera. The environment was explored using 3d frontier exploration.

My responsibilities for this competition included:

Developing 3D LiDAR-based SLAM algorithms for featureless indoor environments.
Setting up drone simulations for testing SLAM algorithms and autonomous exploration in ROS.
Managing the integration of all system components, ensuring seamless communication between hardware, sensors, and ROS frameworks to enable full system functionality.

Competition 2: Outdoor Search and Rescue

The competition occurred on 6th-12th July 2025 in Alabama, USA. The competition aimed at simulating the aftermath of a severe storm where damage assessments of residential communities, search and rescue for victims, and delivery of lifesaving medical supplies needs to be performed. The competition was held on a 3.45 square mile test site near the Huntsville International Airport (KHSV).

Tasks:

Multidrone autonomous mapping of a wide area under GPS-denied conditions and creation of an orthomosaic map.
Finding ArUco markers that encode tasks to be performed including:
- Target verification: Autonomously maneuver around a target to capture multiple visual data points.
- Search and Rescue: Search and identify human like figures in an unknown space and capture a visual data point of each
- Overwatch: Hovering next to a structure and counting the number of people entering and leaving
- Supply Delivery: Autonomously maneuver through a series of obstacles to deliver supplies

Team Klagenfurt:

Gilbert Tanner
Luca Di Pierno
Georg Steinthaler
Tim Schumann
Ben Wesse
Jonas Spieler
Supervisor: Prof. Stephan Weiss

My task for this competition was to extend the SLAM framework from competition 1 to work for larger outdoor areas with different terrains ensuring robustness by using both LiDAR and camera sensors as well as IMU, magnetometer and barometer.

I also extended our Gazebo simulation to support multiagent (compatible with both PX4 and Ardupilot) and additionally create a simulation environment in Isaac Sim to allow for photorealistic simulation.

Additionally I also worked on the system integration including communication between the agents in the field.

Other applications

The technology developed for the Sapience project can be used for a multitude of tasks. Below are other applications that were tested with the platform.

Tunnel Inspection

In September 2024, our team had the opportunity to try our setup in a research tunnel in Eisenerz. Over the course of the day, we managed to map one whole tunnel segment by flying the drone manually and running our SLAM. Overall, the drone traveled for around 742 m in that flight. Additionally, we also managed two autonomous exploration flight with the drone exploring a length of around 100 m for each of the two flights fully autonomously.

Bridge inspection

In March 2025, we had the opportunity to participate in a bridge inspection with our drone stack. Flights were done manually and recorded data was used among other things to generate a 3D mesh of the bridge as shown below.