Stereo Vision in AI & Artificial Intelligence

Stereo Vision is based on Epipolar Geometry. When you have two cameras (Left and Right) looking at the same scene, a point in the real world will appear at different pixel coordinates in each image. The line connecting the two camera centers is the Baseline. Because we know the focal length and the baseline, we can use simple trigonometry to calculate the exact distance (z) to that point. This is effectively 'Triangulation' using light.

The hardest part of stereo vision is the Correspondence Problem: how do we know that pixel (100, 200) in the left image is the same physical object as pixel (90, 200) in the right image? We use Matching Algorithms like SSD (Sum of Squared Differences) or SGM (Semi-Global Matching). These algorithms look for similar patterns of light and texture. The difference in their horizontal position is called Disparity. Large disparity = Close object; Small disparity = Far object.

For the math to work, the cameras must be perfectly aligned. We use Camera Calibration (often with a checkerboard pattern) to find the 'Intrinsics' and 'Extrinsics' of the lenses. We then Rectify the images, mathematically warping them so that matching points always lie on the same horizontal row. Stereo vision's biggest weakness is Textureless Surfaces (like a plain white wall) where there are no patterns to match, and Repetitive Patterns which can cause the algorithm to get confused about which 'Brick' it is looking at.

How do robots see in 3D using just cameras? Stereo Vision mimics human eyes, using two cameras at slightly different angles to calculate depth through the power of geometry.

By comparing the same point in two different images, we calculate the 'Disparity'. The larger the shift, the closer the object is to the robot.

We use 'Block Matching' or 'Semi-Global Matching' (SGM) to find matching points across the two images, creating a 'Depth Map'.

Checkpoint: If an object has a LARGE 'Disparity' (shift), is it close or far?

• →Far away
• →Close to the camera

Stereo vision is cheaper than LiDAR but requires more processing power and good lighting to see textures clearly.

By mastering Stereo Vision, you enable robots to perceive the world's volume and structure using affordable, standard camera hardware.

Checkpoint: What is the 'Baseline' in a stereo camera setup?

• →The height of the camera
• →The physical distance between the two camera lenses

Stereo Vision mastered! You've learned to see in depth. Ready to use Neural Networks for robotic perception?

Compute Real Depth from Disparity. Finish computing scene depth from a stereo camera pair's focal length, baseline, and pixel disparity.