CN101673399A - Calibration method of coded structured light three-dimensional vision system - Google Patents

A calibration method of a coded structured light three-dimensional vision system comprises the following steps of: 1) camera calibration processes: i.e. a two-dimensional calibration template is adopted, a calibration target consists of standard black and white chess checks, feature points are formed by chess checks points, a camera shoots a plurality of images in different positions, check angular points are selected as the feature points for the plurality of images, and a plane calibration method is adopted for calibrating the camera so as to obtain internal parameter calibration K of the camera as well as a mean value focal distance f; and 2) projector calibration processes: i.e. a light template generated from coding is used for projection onto a coplane calibration target, black and white check angular points are used as positioning points, and image coordinates of the black and white check angular points are simultaneously extracted to generate a world coordinate by utilizing theimages obtained from the camera calibration. The calibration method achieves high camera calibration accuracy and can satisfy the requirements of actual application.

The scaling method of coded structured light three-dimensional vision system

Technical field

The present invention relates to computer vision, data processing, Flame Image Process, especially a kind of scaling method of structured light three-dimensional vision system.

Background technology

Light structural three-dimensional recovers 3D shape from the two dimensional image that video camera obtains, and the structured light system has the high good characteristic of precision, and wherein important effect has been brought into play in the demarcation of video camera, projector machine.Video camera, projector respectively with the position mutual relationship of object, mutual relationship between projector and the video camera, imaging geometry model by video camera and projector is determined, the parameter of this two geometric model is called systematic parameter, these parameters must determine that experiment is called the structured light system calibrating with calculation process by experiment and calculating.The demarcation of system is to obtain relation between each sensor and the parameter of self, the reconstruction error of stated accuracy decision total system, and also the nonlinear characteristic of video camera and projector is left in the basket the modeling out of true usually fully.The scaling method of structured light system mainly contains direct demarcation and distributes demarcates two kinds, and in the prior art, the camera calibration technology is very ripe, directly uses the nonlinear model of classics to come calibrating camera, and stated accuracy satisfies practical application.

In the prior art, to the demarcation of projector according to the mutual relationship between itself and the video camera and decide, exist stated accuracy low, can't satisfy the practical application needs.

Summary of the invention

Low for the projector calibrating precision of the scaling method that overcomes existing coded structured light three-dimensional vision system, as can't to satisfy practical application needs deficiency the invention provides a kind of video camera stated accuracy height, can satisfy the scaling method of the coded structured light visual imaging system of practical application needs.

The technical solution adopted for the present invention to solve the technical problems is:

The scaling method of a kind of coded structured light visual imaging system, described scaling method may further comprise the steps:

1), camera calibration process:

Adopt two-dimentional calibrating template, target is that the black and white chess lattice of standard are formed, unique point is that the point of chess lattice is formed, video camera is taken the multiple image of diverse location, select the grid angle point as unique point to described multiple image, adopt the plane reference method that video camera is demarcated, obtain camera intrinsic parameter and demarcate K, and obtain the average focal length

2), projector calibrating process:

The optical mode plate that produces by coding projects on the common target, and black and white grid angle point utilizes the image that obtains as anchor point in camera calibration, extract black and white grid angle point image coordinate simultaneously, generates world coordinates;

X A - X P X A - X B = v A - v P v A - v B , Y A - Y P Y A - Y B = u A - u P u A - u B - - - ( 1 )

Order

λ x = v A - v P v A - v B , λ y = u A - u P u A - u B - - - ( 2 )

Get by (1), (2) formula:

X p = X A + λ x ( X B - X A ) Y p = Y A + λ y ( Y B - Y A ) Z P = Z A = Z B - - - ( 3 ) .

λ _x, λ _yThe double ratio coefficient of representing X and Y direction respectively, (X _p, Y _P, Z _p) world coordinates of expression striation edge feature point.

As preferred another kind of scheme: in described step 2) in, the process of setting up of projector model is:

By projector coordinates x _dWith world coordinates X _wIt is as follows to set up model:

γ x d 1 = m 1 m 14 m 2 m 24 X w Y w Z w 1 = m 11 m 12 m 13 m 14 m 21 m 22 m 23 m 24 X w Y w Z w 1 - - - ( 4 )

γ is arbitrary scale factor, eliminates γ, and conversion (4) formula obtains:

m 1 X w Y w Z w T + m 14 m 2 X w Y w Z w T + m 24 = x d - - - ( 5 )

(5) formula of utilization structure is with m _IkBe the system of homogeneous linear equations of unknown number, find the solution the projector parameter by svd then;

Set up and simplify perspective double ratio unchangeability geometric model, hypothesis space point A, B, C are positioned on same the straight line L, are reference point with B, and the position ratio is defined as:

PR(A，B，C)＝AB/AC????(6)

In like manner, A ', B ', C ' they are under the O of centre of perspectivity effect, the imaging point of corresponding point A, B, C, and be positioned on the same point straight line L1, the position ratio on the image coordinate is:

PR(A′，B′，C′)＝A′B′/A′C′???(7)

Double ratio unchangeability according to the perspective geometry principle obtains:

PR(A，B，C)＝PR(A′，B′，C′)????(8)

At image mid point A ', B ', C ' also is to be positioned on the same straight line L '.

Extract the image coordinate of unique point in the optical mode plate, obtain the world coordinates of reference point B, set up space three-dimensional and come the labeling projection instrument to the mapping relations between the dimension coordinate according to (8) formula.

Beneficial effect of the present invention mainly shows: video camera stated accuracy height, can satisfy the practical application needs.

Description of drawings

Fig. 1 is a double ratio principle of invariance synoptic diagram.

Fig. 2 is the synoptic diagram of high precision target.

Fig. 3 is the synoptic diagram that reads in original uncalibrated image.

Fig. 4 is the synoptic diagram of shooting labeling projection error.

Fig. 5 is striation projection target face figure.

Fig. 6 is the synoptic diagram of edge extracting and fitting a straight line.

Fig. 7 is the synoptic diagram that unique point is obtained.

Fig. 8 is a projector calibrating error synoptic diagram.

Fig. 9 is the synoptic diagram of calibration point three-dimensionalreconstruction.

Embodiment

Below in conjunction with accompanying drawing the present invention is further described.

With reference to Fig. 1～Fig. 9, the scaling method of a kind of coded structured light visual imaging system, described scaling method may further comprise the steps:

1), camera calibration process:

2), projector calibrating process:

The optical mode plate that produces by coding projects on the common target, and black and white grid angle point utilizes the image that obtains as anchor point in camera calibration, extract black and white grid angle point image coordinate simultaneously, generates world coordinates;

During projection single white optical mode plate, directly extract A ₁, B ₁Image coordinate (the u of two angle points _A, v _A), (u _B, v _B), and world coordinates is (X _A, Y _A, Z _A), (X _B, Y _B, Z _B), shown in Figure 7 as in the accompanying drawing asked straight line L _iIntersection point with L obtains P _iThe image coordinate of point is as P ₁The image coordinate of point is (u _P, v _P), its world coordinates is defined as (X _p, Y _P, Z _p), because A ₁, B ₁, P ₁So 3 coplanes are Z _p=Z _A=Z _B, according to the double ratio unchangeability, obtain,

X A - X P X A - X B = v A - v P v A - v B , Y A - Y P Y A - Y B = u A - u P u A - u B - - - ( 1 )

Order

λ x = v A - v P v A - v B , λ y = u A - u P u A - u B - - - ( 2 )

Get by (1), (2) formula:

X p = X A + λ x ( X B - X A ) Y p = Y A + λ y ( Y B - Y A ) Z P = Z A = Z B - - - ( 3 ) .

λ _x, λ _yThe double ratio coefficient of representing X and Y direction respectively, (X _p, Y _P, Z _p) world coordinates of expression striation edge feature point.

In described step 2) in, the process of setting up of projector model is:

By projector coordinates x _dWith world coordinates X _wIt is as follows to set up model:

γ x d 1 = m 1 m 14 m 2 m 24 X w Y w Z w 1 = m 11 m 12 m 13 m 14 m 21 m 22 m 23 m 24 X w Y w Z w 1 - - - ( 4 )

γ is arbitrary scale factor, eliminates γ, and conversion (4) formula obtains:

m 1 X w Y w Z w T + m 14 m 2 X w Y w Z w T + m 24 = x d - - - ( 5 )

(5) formula of utilization structure is with m _IkBe the system of homogeneous linear equations of unknown number, find the solution the projector parameter by svd then;

Set up to simplify perspective double ratio unchangeability geometric model, shown in Figure 1 as in the annex, hypothesis space point A, B, C are positioned on same the straight line L, are reference point with B, and the position ratio is defined as:

PR(A，B，C)＝AB/AC????(6)

In like manner, A ', B ', C ' they are under the O of centre of perspectivity effect, the imaging point of corresponding point A, B, C, and be positioned on the same point straight line L1, the position ratio on the image coordinate is:

PR(A′，B′，C′)＝A′B′/A′C′????(7)

Double ratio unchangeability according to the perspective geometry principle obtains:

PR(A，B，C)＝PR(A′，B′，C′)????(8)

At image mid point A ', B ', C ' also is to be positioned on the same straight line L '.

Extract the image coordinate of unique point in the optical mode plate, obtain the world coordinates of reference point B, set up space three-dimensional and come the labeling projection instrument to the mapping relations between the dimension coordinate according to (8) formula.

In the present embodiment, the projector model is at the conceptive inversion model that is equal to video camera, because the one dimension of space encoding can be simplified to one-dimensional model here.By projector coordinates x _dWith world coordinates X _wIt is as follows to set up model:

γ x d 1 = m 1 m 14 m 2 m 24 X w Y w Z w 1 = m 11 m 12 m 13 m 14 m 21 m 22 m 23 m 24 X w Y w Z w 1 - - - ( 4 )

γ is arbitrary scale factor, eliminates γ, and conversion (4-13) formula obtains:

m 1 X w Y w Z w T + m 14 m 2 X w Y w Z w T + m 24 = x d - - - ( 5 )

(4-14) formula of utilization structure is with m _IkBe the system of homogeneous linear equations of unknown number, decompose by singular value (SVD) then and find the solution the projector parameter.

In three-dimensional reconstruction, the striation marginal point is as unique point, and is different with tessellated angle point, and its world coordinates can't directly obtain, and need utilize the world coordinates of known point to ask indirectly.Set up and simplify perspective double ratio unchangeability geometric model, as shown in Figure 1, hypothesis space point A, B, C are positioned on same the straight line L, are reference point with B, and the position ratio is defined as:

PR(A，B，C)＝AB/AC????(6)

In like manner, A ', B ', C ' they are under the O of centre of perspectivity effect, the imaging point of corresponding point A, B, C, and be positioned on the same point straight line L1, the position ratio on the image coordinate is:

PR(A′，B′，C′)＝A′B′/A′C′????(7)

Double ratio unchangeability according to the perspective geometry principle can obtain:

PR(A，B，C)＝PR(A′，B′，C′)????(8)

In experiment, can find A, B, C on the space line at 3, so corresponding also is to be positioned on the same straight line L ' at image mid point A ', B ', C '.

Extract the image coordinate of unique point in the optical mode plate, can obtain the world coordinates of reference point B according to (4-17) formula.Set up space three-dimensional and come the labeling projection instrument to the mapping relations between the dimension coordinate.

In the structure light vision system, divide independent two steps to demarcate, camera calibration and projector calibrating, the demarcation of each step is all separate, does not rely on mutually, iteration error can not occur, the system accuracy height.

Camera calibration: the three-dimensional coordinate accuracy requirement height of unique point, adopt two-dimentional calibrating template, operate in order to make systematic unity, the single white light of projector projects during camera calibration, coplane target is as shown in Figure 2 made simple and easy.

Target has 21 * 21mm standard black/white gridiron pattern to form, and gridiron pattern is printed by laser high-precision, is close to then on the aluminium sheet face, and unique point is made up of tessellated point.Video camera is taken 10 width of cloth images of diverse location, as shown in Figure 3.

Select the grid angle point as unique point to 10 width of cloth images, use above-mentioned plane reference method that video camera is demarcated, the camera intrinsic parameter calibration result is:

K = 3872.6 0 692.2 0 3858.1 443.3 0 0 1

With the camera standard focal distance f ₀The relative error of=25mm is:

The target unique point is carried out re-projection, and the resultant error scope is the 0.49708-0.65712 pixel, the re-projection error of each width of cloth image as shown in Figure 5, x, y represent with the pixel to be the image coordinate system of unit respectively.

Projector calibrating: projector is as the light source projects device in the structured light system, and its demarcation relates to the precision of system, and this paper demarcates with one dimension simplified model set forth above.The demarcation of projector need be obtained the world coordinates and the striation coordinate figure of feature point for calibration.To project on the coplane target by the optical mode plate that programming produces, black and white grid angle point as shown in Figure 6, is taken four width of cloth images at depth value Z=990mm, 982mm, 974mm, 966mm place respectively as anchor point.

Marginal point carries out fitting a straight line, and the angle point on the straight line of place is also needed to carry out match.

The image that utilization is obtained in camera calibration extracts black and white grid angle point image coordinate simultaneously, and according to the grid characteristic, world coordinates can generate automatically.

X A - X P X A - X B = v A - v P v A - v B , Y A - Y P Y A - Y B = u A - u P u A - u B - - - ( 1 )

Order

λ x = v A - v P v A - v B , λ y = u A - u P u A - u B - - - ( 2 )

Get by (1), (2) formula

X p = X A + λ x ( X B - X A ) Y p = Y A + λ y ( Y B - Y A ) Z P = Z A = Z B - - - ( 3 )

Utilize the principle of double ratio unchangeability to find the solution the world coordinates of striation edge feature point thus, decode the student number coordinate of striation again, demarcate with the projector model.The calibration result scope is: 0-0.2889, the error synoptic diagram as shown in the figure.

The base length of the structure light vision system of present embodiment is 230mm, and four target plane of depth value Z=990mm, 982mm, 974mm, 966mm are reconstructed, and reconstructed error is as shown in table 1.As can be seen from Table 1, utilize the target of the equidistant translation of four width of cloth to demarcate, the mean absolute error of system is 1.38mm, and average relative error is 0.14%.

Table 1

Utilize the parameter of having demarcated respectively four plane reference points to be carried out back projection, reconstruction result as shown in Figure 9.

The precision of measuring accuracy and hardware device is closely related, and in the system calibrating experiment, the reason that error produces is a lot, mainly reduces the following aspects:

1. high-precision gridiron pattern stencil marking exists certain error.

2. extract in the process that produces at angle point, some coplane during camera calibration on the hypothesis gridiron pattern, this plane that will seek template is definitely smooth, but in actual experiment, stencil plane can not be absolute smooth, the image and the hypothesis of taking have little deviation, and this little deviation is introduced to calculate and will be caused error.

3. in the process of projector calibrating, use the double ratio unchangeability and ask world coordinates, introduced error when unique point is carried out fitting a straight line.

4. the error that causes of calibration system.Secondly at first the resolution of vision system and acquisition precision all can influence the error of total system, and scaling method chooses the error that all can cause system with the combination of whole calibrating system each several part.Mainly based on the plane reference method of Zhang Zhengyou, the result from demarcating based on the method for nonlinear iteration, makes the minimum of error to the camera calibration of this paper, and the decomposition of projector model also is based on error minimize.

More than four aspects are the main causes that cause calibrated error.Because experiment condition is limit, the restriction of the video camera of hardware and projector resolution and acquisition precision, in the grid angle point leaching process, the template face is not definitely smooth, causes that angle point extracts error.In addition, above-mentioned system model in theory with computation process in all can defectiveness, also can cause error.