CN113115017B - 3D imaging module parameter inspection method and 3D imaging device - Google Patents

The application discloses 3D imaging module parameter inspection method, 3D imaging module includes the sensing array, the method includes the following step: assembling a 3D imaging module to be inspected to a reference plane, wherein the surface of the reference plane is flat, and the lens internal reference of the 3D imaging module and the position parameters relative to the reference plane are known; using the 3D imaging module to perform depth detection on the reference plane, and acquiring the depth value of each block of the reference plane; respectively corresponding each row of array elements in the sensing array to each block in the reference plane, and determining the depth value of the block corresponding to each row of array elements; calculating and obtaining calculation parameters of the 3D imaging module according to the depth value of the block and the space geometric relationship between the block and the array element corresponding to the block; and judging whether the lens internal parameters and the position parameters of the 3D imaging module are accurate or not according to the calculation parameters.

3D imaging module parameter inspection method and 3D imaging device

Technical Field

The application relates to the field of 3D imaging, in particular to a 3D imaging module parameter inspection method and a 3D imaging device.

Background

The 3D imaging module is used for carrying out 3D imaging calculation by utilizing the assembly information in the process of carrying out 3D imaging. The assembly information includes the relative position relationship of the 3D imaging module with respect to the scene to be imaged and the like after the 3D imaging module is assembled to the application scene, and the accuracy of these parameters may affect the final output 3D image.

In the prior art, a ToF module is often used for 3D imaging. The ToF module, also known as a Time of Flight (ToF) module, can measure a distance, a three-dimensional structure, or a three-dimensional profile of an object to be measured by a pulse signal transmitted from a sensor and a Time interval from transmission to reception or a phase generated by laser light once traveling back and forth to the object to be measured. The ToF sensor can simultaneously obtain a gray image and a distance image, and is widely applied to the fields of somatosensory control, behavior analysis, monitoring, automatic driving, artificial intelligence, machine vision, automatic 3D modeling and the like.

People usually use a 3D imaging module to capture a depth image or a grayscale image of a region to be measured, so as to analyze the region to be measured, and perform subsequent planning, such as path planning. Therefore, the accuracy of the image captured by the 3D imaging module is very important.

It is desirable to provide a technique that can improve the accuracy of images captured by a 3D imaging module.

Disclosure of Invention

In view of this, the present application provides a 3D imaging module parameter checking method and a 3D imaging device, which can improve the accuracy of an image captured by the 3D imaging module.

The application provides a 3D imaging module parameter inspection method, 3D imaging module includes the sensing array, the method includes the following step:

assembling a 3D imaging module to be inspected to a reference plane, wherein the surface of the reference plane is flat, and the lens internal reference of the 3D imaging module and the position parameters relative to the reference plane are known;

using the 3D imaging module to perform depth detection on the reference plane, and acquiring the depth value of each block of the reference plane;

respectively corresponding each row of array elements in the sensing array to each block in the reference plane, and determining the depth value of the block corresponding to each row of array elements;

calculating and obtaining calculation parameters of the 3D imaging module according to the depth value of the block and the space geometric relationship between the block and the array element corresponding to the block;

and judging whether the lens internal parameters and the position parameters of the 3D imaging module are accurate or not according to the calculation parameters.

Optionally, the 3D imaging module includes an optical device, the optical device includes a lens, a center of the lens is equal to a height of a center line in the sensing array with respect to the reference plane, the lens internal parameter includes a theoretical focal length of the lens, and the position parameter includes a theoretical height of the center line of the sensing array with respect to the reference plane.

Optionally, the calculated height of the row array element in the sensing array corresponding to the block x in the sensing array relative to the reference plane is obtained according to the following equation:

wherein Y is a calculated height of a center row of the sensor array relative to the reference plane, depth1 is a depth value of the tile x, V is a theoretical height of the tile x relative to the center row, V0 is a theoretical height of the center row of the sensor array relative to the reference plane, and fy is a theoretical focal length of the lens.

Optionally, when determining whether the lens internal reference and the position parameter of the 3D imaging module are accurate according to the calculation parameter, the method includes the following steps:

obtaining a difference function of the calculation parameters and the theoretical height of the center row of the sensing array compared with the reference plane;

and judging whether the lens internal parameters and the position parameters are accurate or not according to the difference function.

Optionally, the difference function is obtained according to the following equation:

wherein delta1(V) is the difference function, depth1 is the depth value of the tile x of the reference plane, V is the theoretical height of the tile x relative to the center line, V0 is the theoretical height of the center line of the sensing array relative to the reference plane, fy is the theoretical focal length of the lens, and H is the theoretical height of the center of the lens relative to the reference plane.

Optionally, when determining whether the lens internal reference is accurate according to the difference function, the method includes the following steps:

and acquiring the slope of the difference function, and judging that the theoretical focal length of the lens is accurate if the slope is within a first preset range, or else, judging that the theoretical focal length of the lens is inaccurate.

Optionally, when determining whether the position parameter is accurate according to the difference function, the method includes the following steps:

obtaining the difference value of each row of array elements through the difference function, and obtaining the average number of the difference values of each row of array elements;

and judging whether the theoretical height of the central row of the sensing array compared with the reference plane is accurate or not according to the average of the difference values.

Optionally, if the average value of the difference values is smaller than a second preset threshold, it is determined that the theoretical height of the central row of the sensor array compared with the reference plane is accurate, otherwise, it is not accurate.

Optionally, the plane where the array elements of the sensing array are located is perpendicular to the reference plane, and the 3D imaging module parameter inspection is performed at least twice, where each row of array elements in the sensing array is parallel to the reference plane when there is at least one inspection, and each array element in the sensing array is parallel to the reference plane when there is at least one inspection.

The present application further provides a 3D imaging apparatus, comprising:

the 3D imaging module is used for acquiring depth information of the area to be detected;

the controller is connected to the 3D imaging module and used for executing the 3D imaging module parameter inspection method according to the depth information;

and the memory stores control program codes which can be operated by the controller, and the control program codes are used for realizing the 3D imaging module parameter inspection method.

Optionally, the method further includes:

and the input and output device is connected to the memory and is used for a user to input the lens internal parameters of the 3D imaging module and the position parameters relative to the reference plane into the memory.

The 3D imaging module parameter inspection method and the 3D imaging device solve the problem that the accuracy of the camera lens internal parameters and the position parameters of the 3D imaging module is high, and the 3D imaging module can take out the calculation parameters of the 3D imaging module according to the corresponding relation between each array element in the sensing array of the 3D imaging module and the block on the reference plane, the depth value of the checked block, the camera lens internal parameters of the 3D imaging module and the position relation relative to the reference plane, so that the 3D imaging module can take out more accurate depth images.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flow chart illustrating steps of the method for checking parameters of a 3D imaging module according to an embodiment.

Fig. 2 is a schematic diagram illustrating the 3D imaging module performing parameter verification according to an embodiment.

Fig. 3 is a schematic flow chart illustrating steps of the method for checking parameters of a 3D imaging module according to an embodiment.

Fig. 4 is a schematic connection diagram of the 3D imaging device in an embodiment.

Detailed Description

The inventor researches and discovers that when determining the corresponding blocks of each pixel point of the depth image, the parameters of the 3D imaging module needed to be used include the height of the central row/column of the sensing array of the 3D imaging module relative to the plane to which the sensing array is mounted and the focal length of the lens in the 3D imaging module. Therefore, the accuracy of the depth image can be improved by checking the accuracy of the relevant parameters to determine the correction method.

In order to overcome the above problems, the inventor proposes a method for checking parameters of a 3D imaging module, please refer to fig. 1, wherein the method for checking parameters of a 3D imaging module comprises the following steps:

step S101: and assembling the 3D imaging module to be inspected to a reference plane, wherein the surface of the reference plane is flat, and the lens internal reference of the 3D imaging module and the position parameter relative to the reference plane are known.

The 3D imaging module comprises a ToF module. The ToF module comprises a ToF emitting unit, a sensing array, an optical device and a lens. The ToF emitting unit is used for emitting detection optical signals, the sensing array is used for receiving reflected optical signals reflected by the outside, and the optical device is arranged in the light emitting direction of the ToF emitting unit and used for increasing the detection efficiency of the optical signals. The 3D imaging module can acquire a depth image of the reference plane according to the detection optical signal and the reflected optical signal, and determine the depth value of each block on the reference plane.

And in the sensing array, sensor units are used as array elements to acquire the depth value information, and each sensor unit may image one pixel in the depth image correspondingly.

The optics include a lens having a center on the same axis as a center of the sensing array. In fact, the lens center and the center of the sensing array may not be arranged on the same axis.

The reference plane surface is flat and free of depressions and protrusions. The reference plane is used for carrying out parameter calibration of the 3D imaging module, and the influence of the recess and the protrusion on a calibration result can be eliminated.

When the 3D imaging module is subjected to parameter inspection, the inspected parameters comprise the theoretical focal length of the optical device and the theoretical height of the central line of the sensing array.

Step S102: and performing depth detection on the reference plane by using the 3D imaging module to obtain the depth value of each block of the reference plane.

Step S104: and calculating and obtaining the calculation parameters of the 3D imaging module according to the depth value of the block and the space geometric relationship between the block and the array element corresponding to the block.

Specifically, based on the principle of pinhole imaging and the principle similar to triangle, depth1/fy is ToF _ Height/V1, and V1 is the difference between the theoretical Height V of the row of array elements corresponding to the b block compared with the reference plane and the theoretical Height V0 of the center row compared with the reference plane, in this case, ToF _ Height is (V1 × depth1)/fy is [ (V-V0) × depth1 ]/fy.

The ToF _ Height is the calculated Height of the center row and V0 is the theoretical Height of the center row.

Step S105: and judging whether the lens internal parameters and the position parameters of the 3D imaging module are accurate or not according to the calculation parameters.

When determining whether the lens internal parameters and the position parameters of the 3D imaging module are accurate according to the comparison result, please refer to fig. 3, which includes the following steps:

step S1051: and acquiring a difference function of the calculation parameters and the theoretical height of the center row of the sensing array compared with the reference plane.

The difference function is obtained according to the following equation:

wherein delta1(V) is the difference function, depth1 is the depth value of the tile x of the reference plane, V is the theoretical height of the tile x relative to the center line, V0 is the theoretical height of the center line of the sensing array relative to the reference plane, fy is the theoretical focal length of the lens, and H is the theoretical height of the lens center relative to the reference plane.

Step S1052: and judging whether the lens internal parameters and the position parameters are accurate or not according to the difference function.

The delta1(V) can be simplified to delta1(V) ═ Y/H) -1, which is a function of the difference between the ratio of the calculated height to the theoretical height of the center row of the sensor array and 1. Since the center of the lens is as high as the center row of the sensor array compared to the reference plane, the ratio of the measured height of the center row to the theoretical height is expressed as the ratio of the measured height of the center row to the theoretical height of the center of the lens in this formula.

If the calculated height of the center line is equal to the theoretical height, the difference function is-1, if the calculated height of the center line is greater than the theoretical height, the difference function is greater than 0, and if the calculated height of the center line is less than the theoretical height, the difference function is less than 0.

By the slope of the difference function, a comparison of the calculated height of the center row with the theoretical height can be obtained. Moreover, since the slope depth1/(fy × H) of delta1(V), where H is the theoretical height of the center of the lens, can be directly measured, and depth1 is also a measured value, the slope is mainly determined by the focal length fy.

Therefore, when judging whether the theoretical parameter is accurate according to the difference function, the method comprises the following steps: and acquiring the slope of the difference function, and judging that the theoretical focal length of the lens is accurate if the slope is within a first preset range, or else, judging that the theoretical focal length of the lens is inaccurate.

Specifically, when the slope of the curve of the difference function delta1 is within a first preset range, the focal length fy is considered to be satisfactory. And if the slope of the curve is greater than or equal to the first preset threshold, the theoretical focal length fy of the lens is considered to be too small, and corresponding correction is needed.

When judging whether the theoretical parameter is accurate according to the difference function, the method further comprises the following steps:

obtaining the difference value of each row of array elements through the difference function, and obtaining the average number of the difference values of each row of array elements;

and judging whether the theoretical height of the central row of the sensing array compared with the reference plane is accurate or not according to the average of the difference values.

And if the average value of the difference values is smaller than a second preset threshold value, judging that the theoretical height of the central row of the sensing array is accurate compared with the theoretical height of the reference plane, otherwise, judging that the theoretical height is inaccurate.

Specifically, the heights of the array elements in each row in the sensor array, compared with the height of the reference plane, are sequentially substituted into the difference function delta1(V), so as to obtain the difference values corresponding to the array elements in each row, sum the difference values of the array elements in each row, and divide the sum by the number of rows of the array elements, so as to obtain the average value of the difference values of the array elements in each row. And if the average value is smaller than a second preset threshold value, the theoretical height of the central row of the sensing array is considered to be in accordance with the standard, otherwise, the theoretical height is considered to be not in accordance with the standard.

When summing the various difference functions, the summation is according to the following equation:

where sum is the sum of the difference functions, V0 is the theoretical height of the center row of the sense array relative to the reference plane, Vmax is the theoretical height of the highest row of the sense array relative to the reference plane, and the theoretical height of each row array between V0 and Vmax relative to the reference plane will be calculated as V substituted in the sum equation; depth1 is the depth value of each tile, fy is the theoretical focal length of the lens, and H is the theoretical height of the center of the lens relative to the reference plane.

In this embodiment, since the parameters to be tested are embodied when the sensor array is located at different positions, at least two times of 3D imaging module parameter tests are performed when the parameters are calibrated, where each row of array elements in the sensor array is parallel to the reference plane when at least one 3D imaging module parameter test is performed, and each array element in the sensor array is parallel to the reference plane when at least one 3D imaging module parameter test is performed.

Therefore, in this embodiment, when performing the parameter verification, the following steps are further included:

and rotating the 3D imaging module to change the position of each array element of the sensing array relative to the reference plane, then performing depth detection, establishing a corresponding relation between each block on the reference plane and each array element in the sensing array after changing the direction, and calculating and obtaining the calculation parameters according to the depth value of each block and the space geometric relation between the block and each array element to judge whether the lens internal reference position parameters of the 3D imaging module meet the standard in other directions.

And when the parameters of the 3D imaging module are checked for the first time, each row array element in the sensing array is parallel to the reference plane. In the secondary 3D imaging module parameter inspection, the 3D imaging module is rotated by 90 degrees, so that the rows in the sensing array are parallel to the reference plane, then corresponding debugging operation is carried out, and the parameters of the rows in the sensing array are adjusted and converted into the adjustment of the rows in the sensing array.

At this time, the corresponding relationship between each array element in the sensing array and each block on the reference plane M needs to be determined again, and the slope, the average value, and the like of the second difference function delta2 are obtained.

In fact, the sensing array may also be rotated around an axis perpendicular to the sensing array and parallel to the reference plane, so that each row or column of array elements in the sensing array has another included angle with the reference plane, at this time, when calculating the calculation parameter, the included angle between the array elements and the reference plane needs to be considered, and a person skilled in the art can obtain the corresponding calculation mode without any creative labor based on the calculation mode disclosed in the present application.

An embodiment of the present application also provides a 3D imaging apparatus.

The 3D imaging device of the application is used for solving the calculation parameters of the 3D imaging module according to the corresponding relation between each array element in the sensing array of the 3D imaging module and the block on the reference plane, the depth value of the detected block, and the lens internal parameters and the position parameters of the 3D imaging module, and judging the accuracy of the lens internal parameters and the position parameters of the 3D imaging module according to the calculation parameters. The 3D imaging module parameter inspection method and the 3D imaging device can be used for inspecting the accuracy of the 3D imaging module parameters so as to carry out corresponding debugging according to an inspection result, and the 3D imaging module can shoot more accurate depth images.

The above-mentioned embodiments are only examples of the present application, and not intended to limit the scope of the present application, and all equivalent structures or equivalent flow transformations made by the contents of the specification and the drawings, such as the combination of technical features between the embodiments and the direct or indirect application to other related technical fields, are also included in the scope of the present application.