Machine Learning for Robotics
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
ELEC0144
Machine Learning for Robotics
Assignment 1
ELEC0144 – 2022/2023
1 Assignment 1: Camera and Robot Calibration
1.1 Objective Summary
In this assignment, you are required to write Matlab codes for camera calibration, robot
calibration, image processing, and relating 2D pixel position to 3D position.
1.2 Task 1: Camera Calibration Given Simulated Checkerboard Data
In this task, you will write a code to calibrate a camera given some checkerboard data.
The checkerboard pattern was downloaded from Matlab (“checkerboardPattern.pdf”),
and was then printed on an A4 paper. The size of the checkerboard square was
measured and determined to be 22mm (Figure 1). This is an important information as it
will provide the position of the checkerboard points in World frame – The points would
be automatically [0, 0], [22, 0], [44, 0], …, [0, 22], [22, 22], [44, 22], so on and so forth.
Figure 1: Size of checkerboard square [Matlab2020A]
Six images of the checkerboard are then taken using the camera, with the checkerboard
held at six different positions and orientations. The files are “Image1.png” to
“Image6.png”, and are also shown here for your reference (Figure 2).
ELEC0144 – 2022/2023
Page 4
Figure 2: Examples of images taken by webcam.
Next, a code is written to extract the data points (Code 1). Remember, we need two
different types of data points: 1) the corner points in World frame for a single printout,
and 2) the corner points in pixel frame for 6 images.
clear all
close all
clc
% General Settings
numImages = 6; % depends on the number of images you have
squareSize = 22; % in millimeters
% Read files
files = cell(1, numImages);
for i = 1:numImages
files{i} = fullfile(sprintf('Image%d.png', i));
end
% Display the image at working position
I = imread(files{1}); % assume image1 is the working position
figure; imshow(I);
title('Image at Working Position');
% Detect the checkerboard corners in the images.
[imagePoints, boardSize] = detectCheckerboardPoints(files);
% Generate the world coordinates of the checkerboard corners in the
% pattern-centric coordinate system, with the upper-left corner at
(0,0).
worldPoints = generateCheckerboardPoints(boardSize, squareSize);
ELEC0144 – 2022/2023
Page 5
If you run the code, the imagePoints and worldPoints will be generated in the Matlab
workspace. They will be in the form of [x, y] coordinates.
Your task now: Using the extracted data, write a code to calibrate the camera. You
should provide one set of intrinsic parameters as a K-matrix, as well as 6 sets of extrinsic
parameters which should include position and orientation.
Further task: Next, add the following code (Code 2) to your code. The code performs
camera calibration using Matlab’s toolbox.
Code 2: Matlab’s Camera Calibration Code
% Calibrate the camera using Matlab’s toolbox.
imageSize = [size(I, 1), size(I, 2)];
cameraParams = estimateCameraParameters(imagePoints, worldPoints, ...
'ImageSize', imageSize);
Run the code and see the results of “cameraParams” in Matlab workspace, then
compare and discuss the differences between your own results and the toolbox’s
results. Why are there differences? You might want to refer to Zhengyou Zhang’s paper
(Zhang2000) for some details.
Task 1: Write your own code to obtain the intrinsic and extrinsic parameters of the
camera, given checkerboard dataset. Then compare your results with Matlab
toolbox’s results.
What to write in your report:
• Details of the algorithm, derivation, results, discussions and comparisons.
Additional files to submit:
• Your Matlab code (m file) named “Vision_Task1.m”, which will be test run by
your module leader / tutors. The code should be properly commented.
Additionally: Save your intrinsic parameter matrix as “K.mat”, and the extrinsic
parameters of the first image, [1 2 3
] as “Tcw.mat” (cw stands for
camera-to-world), which you will need to use for Task 4 later.
ELEC0144 – 2022/2023
Page 6
1.3 Task 2: Robot Calibration Given Simulated TCP Data
In this task, you will write a code to calibrate the robot, i.e. to find the position and
orientation of the world frame with respect to the robot.
Six points on the checkerboard have been selected. Their positions according to the
World coordinate system (on the checkerboard itself) are:
Point 1 2 3 4 5 6
-22 0 22 44 66 154
-22 22 66 22 88 -22
The same six points have been “probed” by the robot using a sharp-tipped tool. The
positions according to the Robot coordinate system are:
Point 1 2 3 4 5 6
-121.6 -100.4 -79.16 -56.39 -35.55 54.32
222.4 178 133.6 177.2 110.9 219.3
-20 -20.04 -20.08 -19.96 -20.04 -19.7
Task 2: Write your own code to find the position and orientation of the World frame
with respect to the robot frame, given the positions of the checkerboard corner
points.
What to write in your report:
• Details of the algorithm, derivation, results and discussions.
Additional files to submit:
• Your Matlab code (m file) named “Vision_Task2.m”, which will be test run by
your module leader / tutors. The code should be properly commented.
Additionally: Save your result of position and orientation of world frame with respect to
robot frame as a single homogeneous transformation matrix named “Trw.mat” (rw
stands for robot-to-world), which you will need to use for Task 4 later.
ELEC0144 – 2022/2023
Page 7
1.4 Task 3: Image Processing given Simulated Image
In the “Task3Data” folder, you will find an image of “rectangular” object named
“Object.png”. It is of size 640x480 (Figure 3).
Figure 3: Camera view
Your task is to use image processing methods discussed in the lecture to determine the
position, orientation, circularity, edges and corners of the object.
Task 3: Write your own code to find the position, orientation, circularity, edges and
corners of the object.