LabPics (LabPics Dataset for computer vision for autonomous chemistry labs and medical labs)

Introduced by Eppel et al. in Computer vision for liquid samples in hospitals and medical labs using hierarchical image segmentation and relations prediction

LabPics Chemistry Dataset

LabPics Medical Dataset Dataset for computer vision for autonomous chemistry labs and medical labs.

Experimental chemistry consists largely of the handling of materials in vessels. Whether it involves moving and mixing liquids, dissolving or precipitating solids, or extraction and distillation, these manipulations almost always consist of handling materials within transparent containers and depend heavily on visual recognition. For chemists in the lab, it is crucial not only to be able to identify the vessel and the fill level of the material inside it but also to be able to accurately identify the region and phase boundaries of each individual material phase as well as its type (liquid, solid, foam, suspension, powder, etc.).


The dataset contains annotated images for both material and vessels in chemistry labs, medical labs, and any area where liquids and solids are handled within vessels. There are two levels of annotation for each image. One annotation set for vessels and the second for the material phases inside these vessels. Vessels are defined as any container that can carry materials such as Jars, Erlenmayers, Tubes, Funnels, syringes, IV bags, and any other labware or glassware that can contain or carry materials. Material phases are any material contained within or on the vessel. For example, for two-phase separating liquids, each liquid phase is annotated as one instance. If there is foam above the liquid or a chunk of solid inside the liquid, the foam, liquid, and solid will be annotated as different phases. In addition, vessel parts like cork, labels, and valves are annotated as instances. For each instance, there is a list of all the classes it belongs to, and a list of its property. For vessels, the instance classes are the vessel type (Cup, jar, Separatory-funnel…) and the vessel properties (Transparent, Opaque…). For materials, the classes are the material types ( Liquid, solid, suspension, foam, powder…) and their properties (Scattered, On vessel surface…), and for parts, the part type (cork/label). In addition, the relations between instances are annotated. This includes which materials instances are inside which vessels, which vessels are linked to each other or are inside each other (for vessels inside other vessels), and which material phase is immersed inside another material phase. In addition to instance segmentation maps, the dataset also includes semantic segmentation maps that give for each pixel in the image all the classes to which it belongs. In other words, for each class (Liquid, Solid, Vessel, Foam), there is a map of all the regions in the image belonging to this class. Note that every pixel and every instance can have several classes. In addition, instances often overlap, like in the case of material inside the vessel, vessel inside the vessel, and material phase immerse inside other material (like solid inside liquid).

File sources and copyright

Creating this dataset was impossible without a community of chemists who take and share high-quality photos of their experiments. Most of the images of this dataset were shared by these sources. Copyright for all images belongs to their contributors. The dataset is shared for academic purposes only. For any other use of the images, inquire with the image source. The source of every image is mention in the source.txt file located next to the image file. Specific copyright for each image also appears in the same file. If no copyright is mentioned, contact the image source for inquiries. See Image sources and copyright file in main folder and source.txt in image folder for detail.

"This research was developed with funding from the Defense Advanced Research Projects Agency (DARPA). The views, opinions and/or findings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government


Paper Code Results Date Stars

Dataset Loaders

No data loaders found. You can submit your data loader here.


Similar Datasets