Sound-based drone fault classification using multitask learning

arxiv : https://arxiv.org/abs/2304.11708

Accepted at 29th International Congress on Sound and Vibration (ICSV29).

The drone has been used for various purposes including military applications, aerial photography, and pesticide spraying. However, the drone is vulnerable to external disturbances, and malfunction in propellers and motors can easily occur. To improve the safety of drone operations, early detection of mechanical faults should be made in real-time. In this paper, we propose a sound-based deep neural network (DNN) fault classifier and drone sound dataset. The dataset was constructed by collecting the operating sounds of drones from microphones mounted on three different drones in an anechoic chamber. The dataset includes various operating conditions of drones, such as flight directions (front, back, right, left, clockwise, counter clockwise) and faults on propellers and motors. The drone sounds were then mixed with noises recorded in five different spots on the university campus, with a signal-to-noise ratio (SNR) varying from 10 dB to 15 dB. Using the acquired dataset, we train a DNN classifier, 1DCNN-ResNet, that classifies the types of mechanical faults and their locations from short-time input waveforms. We employ multitask learning (MTL) and incorporate the direction classification task as an auxiliary task to make the classifier learn more general audio features. The test over unseen data reveals that the proposed multitask model can successfully classify faults in drones and outperforms single-task models even with less training data.

please reorganize the file directory like below

drone

ㄴA

ㄴB

ㄴC

For each drone type A, B, and C have 540002 files. (Here, 2 means stereo channel, you can find mic1 and mic2 in subdirectory) They are divided into train, valid, and test by a 6:2:2 ratio. For each file, recording information is labeled below.

{model_type}{maneuvering_direction}{fault}{drone_file_index}{background}{background_file_index}{SNR}

model_type: A, B, C

maneuvering_direction: F(Front), B(Back), R(Right), L(Left), C(Clockwise), CC(Counter-clockwise)

fault: N (Normal), MF1~4 (Moter Failure), PC1~4 (Propeller Cut) -> 1~4 means each motor/propeller of the quadcopter.

Dataset available under below "Homepage" ↓