- Kinect Sensor - for tracking the paper in 3D space
- Projector - for displaying images onto the surface
- Laptop - with development tools ready to go
- Tripod - to be able to keep the Kinect and the camera above the working surface
During the lab we split into two sub teams. Me and Andres we decided to focus on figuring how to calibrate the camera and the depth sensor with our projector.
Adam, Derek and Dominique spend the time in the lab to fix the sensor on top of the projector using the tripod. And this was their final work:
Adam, Derek and Dominique spend the time in the lab to fix the sensor on top of the projector using the tripod. And this was their final work:
So what about the projector calibration?
If we just use the Kinect with camera, we wouldn't be able to project accurate images onto the surface which is at the angle. In order to fix any distortions of the projection and to be able to project something in a specified region (the paper we track using Kinect colour camera) we needed to know the correspondence between the images projected and those captured by the camera.
A standard way of doing this is to display a known "chess board" images - known in terms of number of black and white squares and their sizes. Then using the camera we try to detect the corners of the chessboard and compare the result to what we know about the chessboard. This way we can calculate the extrinsic and intrinsic parameters of the camera and apply necessary transformation to the image that goes to the projector, so that the final result is "undistorted".
There are documented ways of calibrating the camera on OpenCV website. However, the Kinect also has got a depth sensor. Lab assistants pointed out that the calibration process needs to take into account the depth values read from the Kinect in order to change the project if we for example lift the paper, as z-coordinates would then change and the we would want to scale the projected image. Florian, on of the lab assistants, pointed us to an open source project that extends standard OpenCV calibration process with making use of the depth values from Kinect. The project can be found here.
If we just use the Kinect with camera, we wouldn't be able to project accurate images onto the surface which is at the angle. In order to fix any distortions of the projection and to be able to project something in a specified region (the paper we track using Kinect colour camera) we needed to know the correspondence between the images projected and those captured by the camera.
A standard way of doing this is to display a known "chess board" images - known in terms of number of black and white squares and their sizes. Then using the camera we try to detect the corners of the chessboard and compare the result to what we know about the chessboard. This way we can calculate the extrinsic and intrinsic parameters of the camera and apply necessary transformation to the image that goes to the projector, so that the final result is "undistorted".
There are documented ways of calibrating the camera on OpenCV website. However, the Kinect also has got a depth sensor. Lab assistants pointed out that the calibration process needs to take into account the depth values read from the Kinect in order to change the project if we for example lift the paper, as z-coordinates would then change and the we would want to scale the projected image. Florian, on of the lab assistants, pointed us to an open source project that extends standard OpenCV calibration process with making use of the depth values from Kinect. The project can be found here.
We set up a project in C# to use Kinect SDK and we wanted to spend the rest of the week translating the C++ code from the website into C#, using Emgu CV - a .NET wrapper for OpenCV.

No comments:
Post a Comment