To evaluate the presented approaches, we created the Physical Anomalous Trajectory or Motion (PHANTOM) dataset consisting of six classes featuring everyday objects or physical setups, and showing nine different kinds of anomalies. We designed our classes to evaluate detection of various modes of video abnormalities that are generally excluded in video AD settings.

The train and test sets of each class contain approximately 30 videos of varying lengths. The train set contains only normal videos, while the test set is evenly balanced between normal and anomalous videos. The classes were designed to be of varying difficulties and to feature different types of anomalies. For example, the window class was filmed in multiple lighting scenarios to increase variance. The normal videos include motion that follows an expected trajectory (pendulum, keyboard) or an expected movement (window). The sushi class features procedural motion, while candle and magnets feature more subtle motion that only appears locally. The anomalous videos can feature an interference of the regular motion (window, candle, magnets), an added or removed step in the usual procedure (sushi), motion that follows a different trajectory (pendulum, keyboard), or contains a different object (pendulum).