Computer vision and community studies
Computer vision and community studies
In 2018, I oriented my resarch interest to computer vision to developp a methodology to study animal communities and the role of natural ennemies in the conservational biological control. In collaboration with William Puceh and Régis Graptin (LIRMM), we first adapt the Inception-v3 classifier to the recognition of ants in digital images. Next, using a sliding window method, we use heat map to the detection of the ants.
Next I started to work on with Convolutional Neural Networks (CNN) such as the system detector YOLO (Redmon et al. 2014) to train detection models on ants, weevils, spiders and seeds in controlled conditions. This method allow to know the identity and number of agents that are present on an digital images (Figures 2 & 3).
Figure 2. Detection of Cosmopolites sordidus using YOLOv3.
Figure 3. Detection of Messor barbarus using YOLOv3.
With Paul Tresson, William Puech and Philippe Tixier, we started to combine sentinel prey experiments, in-situ imaging and CNN analysis to detect individuals of different invertebrate species and identify their trophic and non-trophic relationship in agrosystems.
We developped the pipeline Corigan that uses YoloV3 (Redmond & Farhadi) or YoloV4 (Bochkovskiy et al. 2020) as core for model detection (Figure 4).
Figure 4. Description of the developed pipeline: From data processing for model training and testing to uses and ecological outputs.
The pipeline allowed us to detect the individuals of the species present on our images and even to distinguish the castes of ants within the same species (Figure 5). It was thus possible to obtain information on the dynamics of ant recruitment (Figure 6) but also to reconstruct the network of trophic and non-trophic interactions (Figure 7).
Figure 5. Example of automatic detections of individuals of different species during our sentinel prey experiments.
Figure 6. Reconstruction of the recruitement dynamics of the ant Pheidole radoszkowskii from automatical analysis of images.
Figure 7. Reconstruction of the network of throphic and non-trophic interaction automatical analysis of images.
The species rarely encountered on the images causing a problem of imbalance of the classes of objects that we want to detect. The CNN will learn well to detect the most frequently encountered species but will have less good results on the less encountered classes. To overcome this problem, we have developed a hierarchical version of the Corigan pipeline. The pipeline will first seek to detect individuals at super-class levels (e.g. spider class, ant class). Once an individual belonging to a superclass is detected (e.g. ant class), the image of the individual is sent to a classifier specialized in the classification of species belonging to this class (e.g. Solenopsis geminata) (Figure 8). This improves the precision metric by approximately 30%.
Figure 8. Hierarchical classification.