Title: CS4277_3CV / Camera Models and Calibration
On every note of this deck, unless a note says otherwise:
Title: CS4277_3CV::Camera Models and Calibration
Logo: tum_logo.png

Front: Consider the plane [$]Z = f[/$] as the image plane, what is the central projection mapping from world coordinates [$](X, Y, Z)^T[/$]?
Back: [$$](X, Y, Z)^T \mapsto (fX/Z, fY/Z)^T, \mathbb{R}^3 \mapsto \mathbb{R}^2[/$$]

Front: How is point C, the plane intersecting with p, and the Z axis called?
Back: - C: Camera Centre
- Image Plane
- Principal Axis

Front: What is p and f?
Back: - p: Principal Point
- f: Focal Length

Front: P is the 3x4 homogeneous _________ matrix
Back: P is the 3x4 homogeneous camera projection matrix

Front: Consider the plane [$]Z = f[/$] as the image plane with [$](p_x, p_y)^T[/$] being the coordinates of the principal point, what is the central projection mapping from world coordinates [$](X, Y, Z)^T[/$]?
Back: [$$](X, Y, Z)^T \mapsto (fX/Z + p_x, fY/Z + p_y)^T, \mathbb{R}^3 \mapsto \mathbb{R}^2[/$$]

Front: What is the Camera Instrinsic Matrix and how many DoF does it have?
Back: [$$]\begin{bmatrix} f_x & s & p_x \\ 0 & f_x & p_y \\ 0 & 0 & 1\end{bmatrix}[/$$]
5 DoF

Front: What is the Camera Coordinate Frame?
Back: Frame anchored at the camera. Camera is at origin and principal axis points in the z-axis.

Front: How are the Camera Coordinate Frame and World Coordinate Frame related?
Back: By a rigid transformation (R, t)

Front: Assuming the translation between Camera Coordinate Frame and World Coordinate Frame is 0, and let [$]R[/$] be the rotation matrix.
How to compute [$]X_\text{cam}[/$] from [$]X[/$]?
Back: [$$]X_\text{cam} = RX[/$$]

Front: Assuming there is no rotation between Camera Coordinate Frame and World Coordinate Frame is 0, and let [$]\tilde{C}[/$] be the translation vector.
How to compute [$]X_\text{cam}[/$] from [$]X[/$]?
Back: [$$]X_\text{cam} = X - \tilde{C}[/$$]

Front: Assuming there is rotation and translation between Camera Coordinate Frame and World Coordinate Frame is 0.
Let [$]R[/$] be the rotation matrix and [$]\tilde{C}[/$] the translation vector.
How to compute [$]X_\text{cam}[/$] from [$]X[/$]?
Back: [$$]X_\text{cam} = R(X - \tilde{C})[/$$]
or as in the lecture
[$$]X_\text{cam} = \begin{bmatrix} R & -R\tilde{C} \\ 0 & 1\end{bmatrix}X[/$$]

Front: Give the general mapping of a pinhole mapping?
Back: [$]x = KR[I \vert -\tilde{C}]X[/$], where X is in a world coordinate frame.

Front: Define the camera projection matrix P of the basic pinhole model. How many DoF does P have?
Back: [$]P = KR[I \vert -\tilde{C}][/$]. P has 9 dof.

Front: How many DOF does a general camera projection matrix have?
Back: 11

Front: What are the internal camera parameters?
Back: Parameters contained in [$]K[/$].

Front: What are the external camera parameters?
Back: Parameters of [$]R[/$] and [$]\tilde{C}[/$]

Front: What is the corresponding matrix for a Yaw rotation?
Back: [$$]\begin{bmatrix} \cos \gamma & -\sin \gamma & 0 \\ \sin \gamma & \cos \gamma & 0 \\ 0 & 0 & 1 \end{bmatrix}[/$$]

Front: What is the corresponding matrix for a Pitch rotation?
Back: [$$]\begin{bmatrix} \cos \beta & 0 & \sin \beta \\ 0 & 1 & 0 \\ -\sin \beta & 0 & \cos \beta \end{bmatrix}[/$$]

Front: What is the corresponding matrix for a Roll rotation?
Back: [$$]\begin{bmatrix} 1 & 0 & 0 \\ 0 & \cos \alpha & -\sin \alpha \\ 0 & \sin \alpha & \cos \alpha \end{bmatrix}[/$$]

Front: How is R decomposed into Roll, Pitch, Yaw?
Back: [$$]R = R_\text{yaw} R_\text{pitch} R_\text{roll}[/$$]

Front: What is a Yaw rotation?
Back: Rotation along Z-axis.

Front: What is a Pitch rotation?
Back: Rotation along Y-axis.

Front: What is a Roll rotation?
Back: Rotation along X-axis.

Front: What are two properties of rotation matrices?
Back: 1. Square (either 2x2 or 3x3)
2. Orthonormal

Front: What are 5 important properties of orthonormal matrices?

Front: What is the General Projective Camera?
Back: The only constraint placed on the projection matrix P is rank(P) = 3.

Front: True or false? The Finite Projective set includes cameras that cannot be realised physically.
Back: True.

Front: The set of camera matrices of finite projective cameras [$]P = KR[I \vert -\tilde{C}][/$] is identical with...
Back: the set of homogeneous 3x4 matrices, i.e. [$]P = M[I \vert M^{-1}p_4] = KR[I \vert -\tilde{C}][/$]
for which M is non-singular.

Front: How to compute the camera centre [$]C[/$] from the camera projection matrix [$]P[/$]?
Back: [$]C[/$] is the right null-space of [$]P[/$], i.e. [$]PC = 0[/$].

Front: What is the undefined image point?
Back: The camera centre is one unique point in the world frame for which an image point is not defined.

Front: Why is the camera centre the right-nullspace fo the camera projection matrix?
Back: The points on a 3D line passing through C and A are expressed as a vector span:
[$$]X(\lambda) = \lambda A + (1 - \lambda)C[/$$]
Under the mapping [$]x = PX[/$], points on this line are projected to
[$$]x = PX(\lambda) = \lambda PA + (1-\lambda)PC = \lambda PA[/$$]
i.e. all points [$]X(\lambda)[/$] are mapped to the same image point [$]PA[/$]. Hence, the line must be a ray through the camera centre.

Front: What are the first three column vectors of the camera projection matrix?
Back: The vanishing points of the world coordinate X,Y, and Z-axis.

Front: What is the fourth column vector of the camera projection matrix?
Back: The **image of the world origin [$]\begin{pmatrix} 0 & 0 & 0 & 1\end{pmatrix}^T[/$]

Front: How can the row vectors of the camera projection matrix be interpreted geometrically?
Back: As particular world planes.

Front: What is the principal plane?
Back: The plane through the camera centre parallel to the image plane.

Front: What is the third row vector of the camera projection matrix?
Back: The principal plane.

Front: What are the axis planes?
Back: Defined by the camera centre C and the line x = 0 and y = 0 respectively.

Front: What is the first and second row vector of the camera projection matrix?
Back: The axis planes.

Front: What is the principal axis vector?
Back: [$]v = \det(M)m^3[/$] is a vector in the direction of the principal axis, directed towards the front of the camera.
Where [$]P = [M \vert p_4][/$], and [$]m^3[/$] the third row of M.

Front: How to compute the principal point from the camera projection matrix [$]P = [M \vert p_4][/$]?
Back: The image point [$]x_0 = Mm^3[/$] is the principal point of the camera, where [$]m^{3T}[/$] is the third row of M.

Front: What is forward projection?
Back: Mapping a point in space to an image point: [$]x = PX[/$].

Front: What is back-projection?
Back: Back-projection of points to rays. The ray is the line: [$]X(\lambda) = P^+ x + \lambda C[/$].

Front: What is the pseudo-inverse of P?
Back: [$$]P^+ = P^T(PP^T)^{-1}[/$$]

Front: Let [$]X = (X, Y, Z, T)^T[/$] be a 3D point and P be a camera matrix for a finite camera.
Suppose [$]P(X,Y,Z,T)^T = w(x,y,1)^T[/$]. How to compute the depth of X in front of the principal plane of the camera.
Back: [$$]\text{depth}(X; P) = \frac{\text{sign}(\det M)w}{T ||m^3||}[/$$]

Front: How to determinate if a point X is in front of the camera?
Back: Compute the depth(X; P).

Front: Explain what is meant by the Decomposition of the Camera Matrix!
Back: - Given: The camera matrix P representing a general projective camera.
- Find: The camera centre, orientation, and the internal parameters.

Front: How to decompose the camera projection matrix P into KR?
Back: From RQ decomposition of M.

Front: What are the parameters in [$$]K = \begin{bmatrix} f_x & s & p_x \\ 0 & f_x & p_y \\ 0 & 0 & 1\end{bmatrix}[/$$]?
Back: - [$]f_x[/$]: Scale factor in the x-coordinate direction
- [$]f_y[/$]: Scale factor in the y-coordinate direction
- [$]s[/$]: skew
- [$](p_x, p_y)^T[/$]: coordinates of the principal point.

Front: What is the aspect ratio of camera [$$]K = \begin{bmatrix} f_x & s & p_x \\ 0 & f_x & p_y \\ 0 & 0 & 1\end{bmatrix}[/$$]?
Back: [$]f_y/f_x[/$]

Front: What is Orthographic Projection?
Back: - No change in scale
- optical centre is located at infinity
- Projection rays are parallel
- Model ignores depth

Front: What is the Camera projection matrix for Orthographic Projection?
Back: [$$]P = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} R & \bold{t} \\ \bold{0}^T & 1\end{bmatrix} = \begin{bmatrix} r^{1T} & t_1 \\ r^{2T} & t_2 \\ 0^T & 1\end{bmatrix}[/$$]

Front: How many dof does an orthographic camera projection matrix have?
Back: 5 (three parameters for R, and two for t)

Front: What is a scaled orthographic projection?
Back: A point in 3D space is:
1. projected to a reference plane using orthographic projection, and then
2. projected to the image plane using a perspective projective

Front: What is the camera projection matrix of a scaled orthographic projection?
Back: [$$]P = \begin{bmatrix} k & 0 & 0 \\ 0 & k & 0 \\ 0 & 0 & 1\end{bmatrix} \begin{bmatrix} r^{1T} & t_1 \\ r^{2T} & t_2 \\ 0^T & 1\end{bmatrix} = \begin{bmatrix} r^{1T} & t_1 \\ r^{2T} & t_2 \\ 0^T & 1/k\end{bmatrix}[/$$]

Front: How many dof does a scaled orthographic camera projection matrix have?
Back: 6

Front: What is a weak perspective projection?
Back: - Similar to scaled orthographic projection.
- Difference: allow two different scalings in the two different axial image directions.

Front: What does the DLT algorithm minimize?
Back: The norm [$]||Ah||[/$], where [$]\epsilon = Ah[/$] is called the residual vector.

Front: How does the Algebraic Distance cost function work?
Back: Each correspondence contributes a partial error vector [$]\epsilon_i[/$], where the norm is called the algebraic distance:
[$$]d_\text{alg}(x_i', Hx_i)^2 = ||\epsilon_i||^2[/$$]

Front: What is one disadvantage of the Algebraic Distance cost function?
Back: The disadvantage is that the quantity that is minimized is not meaningful geometrically nor statistically.

Front: What is one advantage of the Algebraic Distance cost function?
Back: linear solution thus computationally inexpensive.

Front: What is the geometric distance cost function?
Back: Difference between the meassured and estimated image coordinates.
The transfer error is minimized over the estimated homography H: [$$]\sum_i d(x_i', Hx_i)^2[/$$]

Front: What is the symmetric transfer error in the geometric distance cost function?
Back: Considers the forward (H) and the backward (H^-1) tranformation: [$$]\sum_i d(x_i,H^{-1}x_i')^2 + d(x_i', Hx_i)^2[/$$]

Front: What is the reprojection transfer error in the geometric distance cost function?
Back: Seeking of a homography [$]\hat{H}[/$] and pairs of perfectly matched points [$]\hat{x}_i[/$] and [$]\hat{x}_i'[/$] that minimize the total error function:
[$$]\sum_i d(x_i, \hat{x}_i)^2 + d(x_i', \hat{x}_i')^2[/$$] subject to [$$]\hat{x}_i' = \hat{H}\hat{x}_i, \forall i[/$$]

Front: What is one disadvantage of the geometric distance cost function?
Back: Reprojection error is computationally complex

Front: What is one advantage of the sampson error cost function?
Back: - Lies between the algebraic and geometric cost functions in terms of complexity
- close approximation to reprojection error

Front: How does the sampson error work?
Back: Let [$]C_H(X) = 0[/$] denote the cost function Ah = 0. [$]\hat{X}[/$] is the desired point so that [$]C_H(\hat{X}) = 0[/$], where [$]\delta_X = \hat{X} - X[/$]. The cost function is approximanted by a taylor expansion: [$$]\mathcal{C}_H(X + \delta_X) = \mathcal{C}_H(X) + (\partial \mathcal{C}_H / \partial X)\delta_X = 0[/$$]

Front: What is the objective of the RANSAC algorithm?
Back: Robust fit of a model to a data set S which contains outliers.

Front: How does the RANSAC algorithm work?
Back: 1. Randomly select a sample of s data points from S and instantiate the model from this subset.
2. Determine a set of data points [$]S_i[/$] which are within a distance threshold t of the model.
3. After N trials, select the **largest consensus set [$]S_i[/$]. The model is re-estimated using all the points in the subset [$]S_i[/$].

Front: What are the minimum number of points needed to fit the model in RANSAC?
Back: 2 for line and 4 for homography

Front: How is the distance threshold t chosen in RANSAC?
Back: Usually chosen empirically

Front: What is the probability that RANSAC never selects a set of s points which all are inliers?
Back: [$$]1-p = (1-w^s)^N \implies N = \frac{\log(1-p)}{\log(1-w^s)}[/$$]
- p: probability that at least one of the random samples of s points is free from outliers
- w: probability that any selected point is an inlier.

Front: Explain the complete algorithm of Robust 2D Homography Computation?
Back: 1. Interest points: Compute keypoints in each image
2. Putative correspondences: match keypoints using descriptors
3. RANSAC robust estimation: Repeat for N samples, where N is determined adaptively
4. Optimal estimation: re-estimate H from all correspondences classified as inliers.