【HALCON】I tried OCR(Optical Character Recognition) with MVtec HALCON


1. Summary

In order to monitor the numbers on the analog counter, I tried OCR (optical character recognition / reader, OSR, optical character recognition) using HALCON, an image processing library from MVtec.
I tried OCR processing construction for the first time, but I managed to build it by referring to the sample program included in HALCON.

The following contents are introduced here.

  1. Control the camera from HALCON
  2. Perform image processing (OCR) on the captured image


2. Development environment

  • OS: Windows10 64bit
  • Image processing library: HALCON version11
  • Camera: Basler Pulse (Cheap USB camera for industrial use)
  • Camera control software: pylon

I will omit the details, but I will connect the camera to a computer and perform operations such as imaging from HALCON via pylon. To do this, in addition to installing HALCON and pylon, you need to install a dll file called the interface for pylon. Now HALCON can control pylon. When trying to build a system that combines a camera and image processing like this one, the environment around here is actually troublesome. There is compatibility between the HALCON version and the pylon version, so be careful when setting up the environment. For more information, it is best to contact Lynx Co., Ltd., the exclusive agent of HALCON.

3. Outline of processing

3.1. Control the camera from HALCON

The frame rate is set to 1 (1fps) because it is not necessary to analyze at a high cycle. Although the frame rate is low, real-time processing is being performed.

3.2 Perform image processing (OCR) on the captured image

OCR in HALCON is not a special processing but almost same process as the Machine Learning.
OCR processing is something like,
first, you teach correct images(images is corresponds to the numbers which you want to do OCR) in advance,
then, compare the captured image with the correct images you teach in advance
and, output the closest one based on Machine Learing.

I just wanted to process the numbers (0-9) of the analog counter, so I thought it would be nice to use a pre-trained classifier (prepared in HALCON), but the accuracy was low. After all, I found that using a classifier that I learned by myself is more accurate. We also found that it is important that the Region can be extracted correctly for the characters in the captured image. If Region can be extracted correctly, it seems to be able to be discriminated almost successfully.

4. Result

You can see how you are monitoring the numbers on the analog counter from YouTube here.
The numbers on the analog counter are changed with my finger, but it looks good OCR result.