Bin-picking challenges - Occlusion

A challenge in pick and place robotics is when we lose data, miss picks, and experience low 3D image quality due to occlusion "blind spots."

emoticon-1
What is occlusion?
emoticon-2
Why does it matter?
emoticon-3
How to solve it with Zivid 3D

What is Occlusion?

Occlusion occurs due to the relative placement of projector/camera and camera, creating "blind spots" (shadow areas) in our target scene. This causes a loss of scene details and hence negatively affecting 3D image quality.

Occlusion is directly proportional to the camera's baseline and becomes very prevalent on smaller objects with relatively large features in the Z-axis (towards the camera). A larger baseline camera reduces the overlapping region that both cameras or camera/projector can detect, while a small baseline camera increases it.

Three types of occlusion 

  • Inter-occlusion: From a certain viewpoint, one part of an object is occluded by another part. 

Point cloud examples: 3D laser scanner with missing data (left) vs. Zivid 3D camera (right)  

Inter occlusion point cloud example

  • Object-occlusion: Occlusion on a single object due to large baseline. Results in less data.

Point cloud examples: 3D laser sensor (left) vs. Zivid 3D camera (right)

Object Occlusion point cloud example

  • Bin-occlusion: Occlusion from a bin wall shadowing objects.

Point cloud examples: Laser 3D camera (left) vs. Zivid 3D camera (right) 

Bin occlusion point cloud example

 

Why does it matter?

Almost all objects in a scene will create some shadow areas. The taller the object is, the longer its shadow will be cast, increasing the shadow area where other objects can be hidden. The shadow's length is also dependent on the distance from camera to camera/projector (baseline), and a larger baseline will lead to larger shadows.

A. Permanent shadow 

Some objects to be forever “hidden” behind a permanent shadow, e.g., created by a bin wall, such that the bin-picking program can never empty the bin without help.

Occlusion example no.1 by Zivid 3D

B. Less objects identified

Fewer addressable objects and preferred pick-points to be identified.

Occlusion example no.2 by Zivid 3D

C. Less details

Loss of details and point cloud coverage of an object, resulting in impaired or failed detection. 

The blind spots can potentially hide objects in the scene, e.g., bin walls hide objects in a bin. It makes it more difficult to empty a bin and increase uncertainty in correctly detecting and picking objects.

How to solve it with Zivid 3D

Zivid One+ and Zivid Two 3D cameras are designed for reducing challenges in bin-picking and pick and place applications. The small baseline gives higher pixel coverage, which is especially relevant for small parts. With Zivid cameras, robots can better see smaller objects, such as shiny cylinders with smaller diameters than other 3D cameras with a larger baseline.

A large baseline - loss of scene details, creating holes, and missing data

Occlusion with a larger baseline camera

Zivid 3D with a smaller baseline - Fewer occlusion outliers 

Occlusion with a Zivid camera - a smaller baseline camera

See more. Do more.

If robots see more, they can do more. Zivid's industrial 3D machine vision enables your system to see details with minimal errors and uncertainties. Capture point clouds at exceptional speed, reduce mispicks, and empty bins fast!

Zivid-Knowledge-Base-QSG
Knowledge Base

Learn

Knowledge Base

Learn more about 3D vision technology and guides for development in the Zivid KB.

3D-vision-free-demo
Free Demo

Evaluate

Demo and Talk

Meet our vision experts for Q&As and free demo.

Zivid 3D developer kit bundle-2-1
3D developer kit bundle

Develop

3D Developer Kit

Order our 3D development kit bundle which includes a Zivid 3D camera and all the accessories you need to kickstart your automation project!