CN109460046B - A method for unmanned aerial vehicle natural landmark recognition and autonomous landing - Google Patents

一种无人机自然地标识别与自主着陆方法属机器视觉导航技术领域，本发明根据给定的预着陆坐标在卫星数字地图上确定着陆区域，利用无人机在预着陆坐标拍摄航拍图像，将卫星数字地图与航拍图像进行滤波、灰度化、二值化处理、边缘特征提取和Hough变换，提取连续几何曲线，并采用加权Hausdorff距离匹配算法将二者进行匹配，根据格林定理计算无人机航拍图像中区域的形心相对于无人机的坐标，根据投影关系计算区域形心的空间坐标，并引导无人机在区域形心的空间坐标自主着陆。本发明能保证无人机在指定范围内自主识别最佳着陆点，并精准着陆，可弥补GPS导航下自主着陆误差大的不足，且自主着陆的安全性和可靠性得到提高。

A method for recognizing natural landmarks and autonomous landing of an unmanned aerial vehicle belongs to the technical field of machine vision navigation. The present invention determines a landing area on a satellite digital map according to given pre-landing coordinates, uses the unmanned aerial vehicle to take aerial images at the pre-landing coordinates, The satellite digital map and aerial image are filtered, grayed, binarized, edge feature extraction and Hough transform, extracted continuous geometric curve, and the weighted Hausdorff distance matching algorithm is used to match the two, and the UAV is calculated according to Green's theorem The centroid of the area in the aerial image is relative to the coordinates of the UAV, and the spatial coordinates of the area centroid are calculated according to the projection relationship, and the UAV is guided to land autonomously at the spatial coordinates of the area centroid. The invention can ensure that the unmanned aerial vehicle can autonomously identify the best landing point within a specified range and land accurately, can make up for the large error of autonomous landing under GPS navigation, and improve the safety and reliability of autonomous landing.

Unmanned aerial vehicle natural landmark identification and autonomous landing method

Technical Field

The invention belongs to the technical field of machine vision navigation, and particularly relates to a natural landmark identification and autonomous landing method for an unmanned aerial vehicle.

Background

In recent years, with the development of micro inertial navigation systems, flight control systems, micro electromechanical systems and novel materials, the research on micro unmanned aerial vehicles has greatly progressed. The rotary wing type micro unmanned aerial vehicle has the advantages of being good in flexibility, compact in structure, low in cost, fast in data acquisition and the like, and the application range also covers various fields including but not limited to pesticide spraying, geological surveying, searching and rescuing, cargo transportation, mapping and the like. Due to the limitation of the response speed and the working efficiency of the people for acquiring the information, the tasks are automatically completed by the unmanned aerial vehicle as much as possible, the actions of automatic take-off and landing, path planning, obstacle avoidance, ground imitation flying and the like are realized through a set program or the automatic planning of the unmanned aerial vehicle, and the accuracy and the reliability of the operation process are ensured.

In the aspect of unmanned aerial vehicle autonomous landing, an autonomous landing mode based on GPS navigation is mostly used at present, namely, a GPS sensor carried by the unmanned aerial vehicle records the geographic coordinate of a take-off time body, or a certain geographic coordinate is specified artificially, and when the unmanned aerial vehicle lands, a GPS positioning system guides the unmanned aerial vehicle to hover over the recorded geographic coordinate and descend for landing. The GPS navigation has the defects of large interference by non-air media, low positioning accuracy and the like, so that the unmanned aerial vehicle has large autonomous landing error in remote areas or areas with a large number of shelters and cannot accurately complete a landing task.

An unmanned aerial vehicle autonomous landing method based on machine vision is one of approaches for solving inaccurate positioning of a GPS (global positioning system), and currently, an autonomous landing method based on artificial landmarks is more applied to a rotor unmanned aerial vehicle. And along with the application of unmanned aerial vehicles in various fields is more and more extensive, also higher and more high to unmanned aerial vehicle's environmental suitability requirement. Some specific tasks require the drone to land in places where artificial landmarks are not suitable, and even require the drone to autonomously find a suitable landing place in a specific area, which requires the drone to have the ability to recognize natural landmarks. Therefore, in order to provide accurate navigation information to the drone and complete a specific autonomous landing task, a natural landmark identification and autonomous landing method for the drone is urgently needed.

Disclosure of Invention

The invention aims to provide an unmanned aerial vehicle natural landmark identification and autonomous landing method based on machine vision and a satellite digital map, which aims to solve the problems in the prior art.

The natural landmark identification and autonomous landing method for the unmanned aerial vehicle comprises the following steps:

1.1 according to the given Pre-landing coordinates (X)₀,Y₀,Z₀) Determining a landing area P with a convex polygon outline on a satellite digital map, firstly carrying out filtering, graying and binarization processing on an image of the area P, then further carrying out edge feature extraction, removing part of miscellaneous points, reserving main edge features based on the area, and finally extracting a continuous geometric curve through Hough transformation to obtain an outline curve I and a reference image A of the area P on the satellite digital map, wherein the binarization processing adopts a maximum inter-class variance method;

1.2, flying an unmanned aerial vehicle to the air above a given pre-landing coordinate, carrying out filtering, graying and binarization processing on an aerial image, further carrying out edge feature extraction, removing part of miscellaneous points, reserving main edge features based on a region, and finally extracting a continuous geometric curve through Hough transformation to obtain a profile curve II and a measured image B of a region P on a satellite digital map, wherein the binarization processing also adopts a maximum inter-class variance method;

1.3, matching the reference image A obtained in the step 1.1 with the actual measurement image B obtained in the step 1.2, and confirming a landing area P in the aerial image of the unmanned aerial vehicle; the image matching adopts a weighted Hausdorff distance matching algorithm, and comprises the following steps:

1.3.1 in the reference image A and the actual measurement image B, the 3-4DT algorithm is adopted to carry out the distance conversion of the characteristic point set in the two-dimensional space, and an image distance conversion matrix J is obtained_AAnd J_B；

1.3.2 extracting branch points in a reference image A and a measured image B, and respectively storing the branch points in matrixes A and B;

1.3.3 according to J_A、J_BAnd A and B calculate the weighted Hausdorff distance:

H(A,B)＝max(h_WHD(A,B),h_WHD(B,A))

wherein: A. b is two point sets; n is a radical of_aIs the total number of feature points in point set a; a is a feature point belonging to A; d (a, B) is the distance from the characteristic point a on the point set A to the point set B; h is_WHD(A, B) represents the directed distance from point set A to point set B; h is_WHD(B, A) represents the directed distance from point set B to point set A;

the point with the minimum Hausdorff distance is the final matching point, so that the preliminary positioning information is obtained;

1.3.4, utilizing a least square algorithm to carry out one-to-one correspondence on all matching point pairs to obtain more accurate position information;

1.4 establishing a two-dimensional plane rectangular coordinate system by taking the unmanned aerial vehicle camera as the origin of coordinates, and calculating the coordinate (x) of the centroid of the area P in the aerial image of the unmanned aerial vehicle relative to the unmanned aerial vehicle according to the Green's theorem_c,y_c)；

1.5 calculating the coordinates (X) of the P centroid of the region from the projection relationship_c,Y_c,Z_c) The method specifically comprises the following steps:

1.5.1 calculating ground resolution GSD:

wherein: GSD represents ground resolution (m); f is the focal length (mm) of the lens; p is the pixel size (mm) of the imaging sensor; h is the corresponding flight height (m) of the unmanned aerial vehicle;

1.5.2 calculating the actual ground distance of the image diagonal, and obtaining the ground distance L between the image diagonal according to the width w and the height h of the image:

wherein: GSD represents ground resolution (m); w is the image width; h is the image height;

1.5.3 according to the longitude and latitude of the central point of the image, the distance and the direction angle of the area P centroid relative to the central point, the geographical coordinate of the area P centroid is obtained:

wherein: theta₀∈(0,2π)；Lon_aLongitude of the image center point; lat_aThe latitude of the central point of the image; r_i6378137m is taken as the equatorial radius; r_jTaking 6356725m as the extreme radius;

1.5.4 converting geographic coordinates into spatial coordinates to obtain spatial coordinates (X) of P centroid of region_c,Y_c,Z_c)：

Wherein: n is the curvature radius; lon is longitude; lat is latitude; h is elevation;

1.6 unmanned aerial vehicle flies to space coordinate (X)_c,Y_c,Z_c) And (5) landing in the vertical direction in the air.

The method can ensure that the unmanned aerial vehicle autonomously identifies the optimal landing point within the designated range, accurately lands, can make up for the defect of large autonomous landing error under GPS navigation, and improves the safety and reliability of autonomous landing.

Drawings

FIG. 1 is a flowchart of a method for identifying natural landmarks and autonomously landing for unmanned aerial vehicles

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below.

Step one, according to the given pre-landing coordinate (X)₀,Y₀,Z₀) Determining a proper landing area P (requiring the outline of the area to be a convex polygon) on a satellite digital map, firstly carrying out filtering processing, graying processing and binarization processing on the image of the area P, further carrying out edge feature extraction, removing partial miscellaneous points, reserving main edge features based on the area, finally extracting a continuous geometric curve through Hough transformation to obtain an outline curve of the area P on the satellite digital map, and obtaining a reference image A, wherein the binarization processing selects a maximum inter-class variance method, supposing that T is a selected global threshold, pixels of all pixel points of the image are divided into a foreground and a background according to T as a boundary, and omega is a global threshold₁And ω₂Respectively representing the proportion of pixels belonging to the background and the foreground in the whole image, then:

wherein: p (i) represents the probability of the pixel with the pixel value i appearing in the image.

μ₀And mu₁Respectively representing the average value of the pixels of the background pixel and the foreground pixel, and if mu is the average pixel value of all the pixels, then:

the variance σ between classes corresponding to the threshold²(T) is defined as:

σ²(T)＝ω₀(T)[μ₀(T)-μ(T)]²+ω₁(T)[μ₁(T)-μ(T)]²＝ω₀(T)ω₁(T)[μ₀(T)-μ₁(T)]²

and traversing each gray value, and finding out the threshold T corresponding to the maximum inter-class variance, namely the threshold.

And secondly, flying the unmanned aerial vehicle to the vicinity of the upper space of the given pre-landing coordinate, carrying out filtering processing, graying processing and binarization processing on the aerial image, further carrying out edge feature extraction, removing part of miscellaneous points, reserving main edge features based on the area, and finally extracting a continuous geometric curve through Hough transformation to obtain a contour curve of the area P on the satellite digital map so as to obtain a real-measurement image B. The binarization processing also selects a maximum inter-class variance method.

And step three, matching the reference image A with the actual measurement image B, and confirming the landing area P in the aerial image of the unmanned aerial vehicle. The image matching adopts a weighted Hausdorff distance matching algorithm, and comprises the following specific steps:

(1) in the reference image A and the actual measurement image B, the distance conversion of the feature point set in the two-dimensional space is carried out by adopting a 3-4DT algorithm to obtain an image distance conversion matrix J_AAnd J_B；

(2) Extracting branch points in the reference image A and the measured image B, and respectively storing the branch points in the matrixes A and the matrix B;

(3) according to J_A、J_BAnd A and B calculate the weighted Hausdorff distance:

H(A,B)＝max(h_WHD(A,B),h_WHD(B,A))

wherein: A. b is two sets of points, N_aIs the total number of feature points in the point set A, a is a feature point belonging to A, d (a, B) is the distance from the feature point a to the point set B on the point set A, h_WHD(A,B)、h_WHD(B, A) represent the directional distances from point set A to point set B and from point set B to point set A, respectively.

The point with the minimum Hausdorff distance is the final matching point to obtain preliminary positioning information.

(4) And performing one-to-one correspondence on all the matching point pairs by using a least square algorithm to acquire more accurate position information.

Establishing a two-dimensional plane rectangular coordinate system by taking the unmanned aerial vehicle camera as a coordinate origin, and calculating the coordinate (x) of the centroid of the region P in the aerial image of the unmanned aerial vehicle relative to the unmanned aerial vehicle_c,y_c)。

According to green's theorem, the closed contour along region P integrates:

after discretization, the above formula translates to:

step five, calculating the coordinates (X) of the P centroid of the region according to the projection relation_c,Y_c,Z_c):

(1) Calculating the ground resolution:

wherein: GSD represents ground resolution (m), f is lens focal length (mm), P is imaging sensor's pixel size (mm), and H is the corresponding flight height (m) of unmanned aerial vehicle.

(2) Calculating the actual ground distance of the image diagonal, and obtaining the ground distance between the image diagonal according to the width w and the height h of the image:

(3) according to the longitude and latitude of the central point of the image, the distance and the direction angle of the area P centroid relative to the central point, the geographic coordinate of the area P centroid is obtained:

wherein: theta₀∈(0,2π),Lon_a、Lat_aIs the longitude and latitude of the center point of the image, R_iTaking 6378137m, R as the equatorial radius_j6356725m was taken for the polar radius.

(4) Conversion between a geographical coordinate to an inter-spatial coordinate system

Wherein: n is curvature radius, Lon, Lat and H are longitude, latitude and elevation respectively, and space coordinates (X) of the centroid of the area P is obtained_c,Y_c,Z_c)。

Step six, the unmanned plane flies to a space coordinate (X)_c,Y_c,Z_c) Go to hang in the upper airLanding in a straight direction.

1.一种无人机自然地标识别与自主着陆方法，其特征在于包括下列步骤：1. a kind of unmanned aerial vehicle natural landmark identification and autonomous landing method, it is characterized in that comprising the following steps: 1.1根据给定的预着陆X₀,Y₀,Z₀坐标，在卫星数字地图上确定一块轮廓是凸多边形的着陆区域P，先对区域P的图像进行滤波、灰度化和二值化处理，再进一步实施边缘特征提取，剔除部分杂点，保留基于区域的主边缘特征，最后通过Hough变换提取连续几何曲线，得到卫星数字地图上区域P的轮廓曲线Ⅰ和参考图像A，其中，二值化处理采用最大类间方差法；1.1 According to the given pre-landing X ₀ , Y ₀ , Z ₀ coordinates, determine a landing area P whose outline is a convex polygon on the satellite digital map, and first filter, grayscale and binarize the image of the area P , and then further implement edge feature extraction, remove some noise points, retain the main edge features based on the region, and finally extract the continuous geometric curve through Hough transform to obtain the contour curve I of the region P on the satellite digital map and the reference image A. Among them, the binary value The maximum inter-class variance method is used for the processing; 1.2无人机飞行至给定的预着陆坐标上空，将航拍图像进行滤波、灰度化和二值化处理，再进一步实施边缘特征提取，剔除部分杂点，保留基于区域的主边缘特征，最后通过Hough变换提取连续几何曲线，得到卫星数字地图上区域P的轮廓曲线Ⅱ和实测图像B，其中，二值化处理同样采用最大类间方差法；1.2 The drone flies to the given pre-landing coordinates, and the aerial image is filtered, grayed and binarized, and then further edge feature extraction is performed, some noise points are removed, and the main edge features based on the region are retained. Finally, The continuous geometric curve is extracted by Hough transform, and the contour curve II of the area P on the satellite digital map and the measured image B are obtained, and the maximum inter-class variance method is also used for the binarization processing; 1.3将步骤1.1得到的参考图像A与步骤1.2得到的实测图像B进行匹配，在无人机航拍图像中确认着陆区域P；图像匹配采用加权Hausdorff距离匹配算法，包括下列步骤：1.3 Match the reference image A obtained in step 1.1 with the measured image B obtained in step 1.2, and confirm the landing area P in the UAV aerial image; the image matching adopts the weighted Hausdorff distance matching algorithm, including the following steps: 1.3.1在参考图像A与实测图像B中，采用3-4DT算法进行特征点集在二维空间中的距离转换，得到图像距离转换矩阵J_A和J_B；1.3.1 in the reference image A and the measured image B, adopt the 3-4DT algorithm to carry out the distance conversion of the feature point set in the two-dimensional space, and obtain the image distance conversion matrix J _A and J _B ; 1.3.2提取参考图像A和实测图像B中的分支点，并分别存储到矩阵A和B中；1.3.2 Extract the branch points in the reference image A and the measured image B, and store them in the matrix A and B respectively; 1.3.3根据J_A、J_B、A和B计算加权Hausdorff距离：1.3.3 Calculate the weighted Hausdorff distance from J _A , J _B , A and B: H(A,B)＝max(h_WHD(A,B),h_WHD(B,A))H(A,B)=max( _hWHD (A,B), _hWHD (B,A))

其中：A、B为两个点集；N_a为点集A中特征点的总数；a是属于A的一个特征点；d(a,B)是点集A上特征点a到点集B的距离；h_WHD(A,B)代表从点集A到点集B的有向距离；h_WHD(B,A)代表从点集B到点集A的有向距离；Among them: A and B are two point sets; Na is the total number of feature points in point set A; _a is a feature point belonging to A; d(a, B) is the feature point a to point set B on point set A ; h _WHD (A, B) represents the directional distance from point set A to point set B; h _WHD (B, A) represents the directional distance from point set B to point set A; 具有最小Hausdorff距离的点就是最终匹配点，由此获取初步的定位信息；The point with the smallest Hausdorff distance is the final matching point, from which preliminary positioning information is obtained; 1.3.4利用最小二乘算法对所有的匹配点对进行一一对应，来获取更为精确的位置信息；1.3.4 Use the least squares algorithm to perform one-to-one correspondence of all matching point pairs to obtain more accurate position information; 1.4以无人机摄像头为坐标原点建立二维平面直角坐标系，根据格林定理计算无人机航拍图像中区域P的形心相对于无人机的x_c,y_c坐标；1.4 Establish a two-dimensional plane rectangular coordinate system with the UAV camera as the coordinate origin, and calculate the x _c , y _c coordinates of the centroid of the area P in the UAV aerial image relative to the UAV according to Green's theorem; 1.5根据投影关系计算区域P形心的X_c,Y_c,Z_c坐标，具体包括:1.5 Calculate the X _c , Y _c , Z _c coordinates of the centroid of the region P according to the projection relationship, including: 1.5.1计算地面分辨率GSD：1.5.1 Calculate the ground resolution GSD:

其中：GSD表示地面分辨率(m)；f为镜头焦距(mm)；P为成像传感器的像元尺寸(mm)；H为无人机对应的航高(m)；Among them: GSD represents the ground resolution (m); f is the focal length of the lens (mm); P is the pixel size of the imaging sensor (mm); H is the flight height corresponding to the drone (m); 1.5.2计算影像对角线实际地面距离,根据影像的宽度w和高度h，得到图像对角线之间的地面距离L：1.5.2 Calculate the actual ground distance of the image diagonal, and obtain the ground distance L between the image diagonals according to the width w and height h of the image:

其中：GSD表示地面分辨率(m)；w为影像宽度；h为影像高度；Among them: GSD represents the ground resolution (m); w is the width of the image; h is the height of the image; 1.5.3根据影像中心点经纬度及区域P形心相对中心点的距离及方向角,求得区域P形心的地理坐标：1.5.3 According to the latitude and longitude of the center point of the image and the distance and direction angle of the centroid of the area relative to the center point, the geographic coordinates of the centroid of the area P are obtained:

其中：θ₀∈(0,2π)；Lon_a为影像中心点的经度；Lat_a为影像中心点的纬度；R_i为赤道半径，取6378137m；R_j为极半径，取6356725m；Where: θ ₀ ∈(0,2π); Lon _a is the longitude of the center of the image; Lat _a is the latitude of the center of the image; R _i is the equatorial radius, taking 6378137m; _Rj is the polar radius, taking 6356725m; 1.5.4进行地理坐标到空间坐标的转换,得到区域P形心的空间坐标X_c,Y_c,Z_c：1.5.4 Convert geographic coordinates to spatial coordinates to obtain the spatial coordinates X _c , Y _c , Z _c of the centroid of the region P:

其中：N为曲率半径；Lon为经度；Lat为纬度；H为高程；Where: N is the radius of curvature; Lon is the longitude; Lat is the latitude; H is the elevation; 1.6无人机飞行至空间坐标X_c,Y_c,Z_c上空，进行垂直方向着陆。1.6 The drone flies to the space above the space coordinates X _c , Y _c , Z _c for vertical landing.