GLOBAL SHUTTER, CSI CAMERAS IN STEREO – WHAT THAT MEANS
The vision systems on Spiri drones are based on global shutter, CSI (camera serial interface) in stereo. This essay explains our choices and what they mean for you. We’ll begin with definitions and comparisons, and end with a discussion of how our cameras are used.
GLOBAL SHUTTER VS. ROLLING SHUTTER
A camera sensor with a global shutter scans the entire frame at once. With a rolling shutter it scans one line of pixels at a time, typically from top of the frame to bottom.
The problem with rolling shutter is that when a scene is moving quickly, the recorded images are distorted because the first line of the frame was recorded before the last line of the frame. If the camera drives alongside a building, the building will look slanted. If the camera records a propeller, the propeller will look curved. The effect is made worse the higher the resolution of the rolling shutter camera because it takes longer and longer to scan the frame. Global shutter circuitry solves this problem.
CSI BUS VS. USB INTERFACE
The CSI (camera serial interface) bus is a direct connection to the computer’s image processing systems, while the USB (universal serial bus) interface puts a layer of hardware in between the camera and the computer. The reason for running cameras through USB is that typically they are easier to integrate that way; the driver tools ordinarily work out of the box. But for the convenience you pay severely in performance.
Bandwidth is the first problem when running a camera through USB. Even where the theoretical bandwidth for USB3 should be sufficient for video, there are still processing and memory bottlenecks that throttle the bandwidth. The second problem is graphics processing performance. A USB camera feed will go straight to the CPU and can easily overload it.
The CSI bus, on the other hand, opens up the maximum bandwidth for video, and routes the camera feed into the hardware vision pipeline (instead of using RAM), which reduces CPU load. Not only does this lead to much more efficient handling of the video stream at the hardware level, it opens up a plethora of low level software controls to help process the images.
Spiri uses the CSI bus for its two global shutter cameras.
STEREO VS MONOCULAR VISION
Like our own two eyes, the primary advantage of stereo cameras is depth perception. Spiri synchronizes its cameras by having one trigger the other, similar to our own vision with one eye leading, or "dominant". The difference between the two images is perceived as depth, and tasks such as obstacle avoidance and 3D mapping are more accurate and faster than with monocular (one eye/camera) vision.
BRINGING IT TOGETHER ON SPIRI
Spiri uses its vision system to assist in navigation with a process called SLAM (simultaneous location and mapping). Spiri creates a 3D map of its surroundings, and uses the motion of features in that map as an indicator for its own motion, exactly how animals use vision to navigate. With our global shutter cameras and CSI interface, Spiri can rely on its eyes with bandwidth and processing to spare.