A Hybrid CNN-DSP Algorithm for Package Detection in Distance Maps

Abstract

This paper presents a hybrid algorithm for real-time instance segmentation of packages from scenes represented by 2D distance maps (range images). The paper introduces a novel approach combining deep learning-based methods and digital signal processing methods to enable accurate package recognition, using a small training dataset with high variability and distance measurement errors characteristic of Time-of-Flight-based scanning. Two convolutional neural networks with architecture optimized for training with a limited number of samples perform an initial segmentation of package components (sides and edges). An algorithm based on digital signal processing methods performs refinement of intermediate results, and combines package components into packages. Training and evaluation of the algorithm were performed on a custom dataset containing scenes of packages, shipping bags, and packaging of irregular shapes with various sizes, orientations, and degrees of occlusion, organized either in ordered stacks or arbitrary order. The convolutional neural networks provide a reliable distinction between components of packages and components of other types of packaging and surroundings. Package sides containing a sufficient number of distance points are correctly combined into packages. Thus, the proposed algorithm represents a solid basis for fully automated loading/unloading of packages with arbitrary sizes and materials from transport trailers and storage spaces. The dataset and annotations for box side surfaces are available at: https://dipteam.feit.ukim.edu.mk/results-package-detection.html .