↓Skip to main content

Augmented Reality at logistical Ports

·557 words·3 mins
YOLO Python Deep Learning Machine Learning Object Detection Classification Numpy OpenCV PyTorch OCR Data Scientist Blender Git

In this project, an augmented reality (AR) system was developed for port logistics. The aim of the system is to identify containers through object recognition and to read out relevant information such as container number (BIC), weight and capacity. By using Computer Vision and Deep Learning, an automated solution was developed that significantly improves the process of container identification and classification.

Process: Identification of containers #

Container identification is carried out using a combination of object recognition and optical character recognition (OCR). First, the relevant text areas on the containers are recognized, which contain information such as the container number, weight and capacity. These character areas are then analyzed using OCR to extract the information they contain.

graph LR A[object recognition] -->|text excerpt| B[OCR]

Technological structure #

1. object recognition with YOLO #

The YOLO (You Only Look Once) model was used to recognize the relevant container areas. This model was trained on image data to identify specific regions on the containers that contain information such as container number, weight and capacity.

The model makes it possible to efficiently localize the text areas on the containers and process them in real time, which is ideal for use in an AR environment.

2. synthetic data generation #

In order to train the YOLO model efficiently, synthetic data was generated. For this, Blender was used to create realistic images of containers rendered under different conditions. This synthetic data helped to expand the dataset and prepare the model for different scenarios such as lighting conditions, angles and container states. Various data pre-processing steps were applied to improve the quality and variance of the training data. Image manipulation and augmentation** were used as part of the data preprocessing to artificially alter the images, e.g. by rotation, scaling and color manipulation. This allowed the model to become more robust against different conditions. The subsequent combination of synthetic and real data allowed a very good detection rate to be achieved.

graph TD A[real data] --> E[data set] E -->|Training| C[YOLO model] B[Synthetic data with Blender] --> E C -->|Object recognition| D[BIC on container]

3. OCR for text extraction #

Once the relevant regions on the containers were recognized, Optical Character Recognition (OCR) was used to extract the text from the recognized regions. This step allowed the container number, weight and capacity to be automatically captured and passed on to the AR system. In preparation for the OCR, the image sections also had to be manipulated to improve OCR recognition. In addition to classic OCR algorithms, deep learning models were also evaluated.

Challenges #

One challenge during development was the condition of the containers, especially the often faded or dirty lettering. To improve the OCR recognition performance, it was necessary to develop robust pre-processing algorithms which, for example, greatly improved the contrast value.

Conclusion #

The developed AR system enables automated and efficient container identification in port logistics. By combining computer vision, object recognition, OCR and synthetic data generation, a robust and scalable solution was created that significantly optimizes the logistics process in the port.

Activities #

  • Implementation of an object recognizer with YOLOv3 for the detection of BIC codes and container seals in images of containers
  • Generation of a synthetic data set with various BIC codes and container seals using Blender
  • Conducting various AI experiments and model training to identify good AI models
  • Development of algorithms to optimize OCR recognition