CN114397911B - Unmanned aerial vehicle cluster countermeasure decision-making method based on multiple intelligent agents - Google Patents

The invention discloses a multi-agent-based unmanned aerial vehicle cluster countermeasure decision-making method, which belongs to the technical field of unmanned aerial vehicles and comprises the following steps of: collecting current environmental information and situation information of the unmanned aerial vehicle; modeling according to current environment information and situation information of the unmanned aerial vehicle, wherein the modeling comprises the steps of constructing a motion model of the unmanned aerial vehicle and a threat source model based on a graph; performing target hitting task distribution of the unmanned aerial vehicle by utilizing an ant colony algorithm based on a graph model; and determining the countermeasure decision according to the assigned hit targets. The invention can avoid adverse targets, support the inferior friend machine and find the beneficial targets when cruising, more effectively finish the cooperative antagonism of the unmanned aerial vehicle cluster, and improve the efficiency of the unmanned aerial vehicle cluster antagonism fight.

Unmanned aerial vehicle cluster countermeasure decision-making method based on multiple intelligent agents

Technical Field

The invention relates to the technical field of aircrafts, in particular to a multi-agent-based unmanned aerial vehicle cluster countermeasure decision-making method.

Background

Through decades of development, unmanned aerial vehicles have shown great advantages in replacing human execution flight tasks, particularly 4D (Dull, dirty, dangerous and Deep) tasks are complex, variable and difficult to predict, so that low-cost unmanned aerial vehicles with small volumes and high speed are outstanding, and casualties can be effectively avoided. For single unmanned aerial vehicle combat mode, the comprehensive defensive of single combat, unmanned aerial vehicle cluster combat mode has not only fused single unmanned aerial vehicle's powerful function to pay more attention to unmanned aerial vehicle cluster and fight in coordination, fight jointly, with the single combat ability advantage of gathering and cluster cooperation ability advantage.

The cluster countermeasure of the unmanned aerial vehicle can refer to the working form of a multi-agent system, the unmanned aerial vehicle is abstracted into an independent agent, and a direction is provided for automatic generation of an optimal formation algorithm in the unmanned aerial vehicle cluster flight and formation maintenance under the environments with obstacles and no obstacles. Wherein the external environment, such as terrain, weather, and unmanned aerial vehicle clusters, together comprise a multi-agent system. In the system, each unmanned aerial vehicle represents an agent. The task to be completed by the whole multi-agent system is to hit the target through the cooperation of all agents. Meanwhile, the real-time demand problem under the allocation of the multi-UAV cooperative tasks can be solved under the framework of the multi-agent system according to UAVs (unmanned aerial vehicles), targets and environmental information, and the communication behaviors such as knowledge sharing, planning, action coordination and the like among the multi-agents are realized. The multi-agent technology has self-organizing capability, learning capability and reasoning capability, and provides an effective way for the countermeasure control and decision among unmanned aerial vehicle clusters. However, the existing multi-agent algorithm is similar to the existing algorithm, and only a few agent scenes can be solved, so that the dynamic countermeasure of the unmanned aerial vehicle cluster is difficult to process.

Disclosure of Invention

The invention aims to provide a multi-agent-based unmanned aerial vehicle cluster countermeasure decision-making method so as to meet the demands of unmanned aerial vehicle cluster countermeasure.

In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:

the embodiment of the invention provides an unmanned aerial vehicle cluster countermeasure decision-making method based on multiple intelligent agents, which comprises the following steps:

collecting current environmental information and situation information of the unmanned aerial vehicle;

modeling according to current environment information and situation information of the unmanned aerial vehicle, wherein the modeling comprises the steps of constructing a motion model of the unmanned aerial vehicle and a threat source model based on a graph;

performing target hitting task distribution of the unmanned aerial vehicle by utilizing an ant colony algorithm based on a graph model;

and determining the countermeasure decision according to the assigned hit targets.

In a further optimized scheme, the step of performing unmanned aerial vehicle hitting target task allocation by using an ant colony algorithm based on a graph model comprises the following steps:

s1, constructing a graph according to a threat source model, calculating the total cost of each side in the graph, and giving an initial pheromone value to each side of the graph;

s2, enabling the unmanned aerial vehicle to start searching from a node closest to a departure point, selecting an edge according to a state transition rule, and ending searching by taking the node closest to a target as an end point;

s3, after all unmanned aerial vehicles in the unmanned aerial vehicle cluster complete respective hit target selection, calculating the cost of a feasible path according to a performance index function, updating the found optimal path, and updating the pheromones of each side in the graph according to a pheromone updating rule, wherein no side passed by the unmanned aerial vehicle is subjected to pheromone evaporation;

repeating steps S2-S3 until the end condition is reached.

The performance index function is an improved performance index function, t _i ＝λ ₁ P _T (d _i )+λ ₂ P _R (d _i )+λ ₃ P _M (d _i )，J ₂ ＝0.5*||h _i -h _j || ² s _ij wherein->To adjust the coefficient e _i Representing the energy cost of the ith unmanned aerial vehicle for executing tasks, n is the number of unmanned aerial vehicles, and omega, 1-omega respectively represent the weight coefficients of the energy cost and the threat cost, lambda ₁ ,λ ₂ ,λ ₃ Are weight coefficients, vector x _i 、h _i Respectively representing the states of the unmanned plane before and after executing tasks s _ij Is x _i And x _j Is a correlation degree of (a). In the scheme, the regular term describing the unmanned aerial vehicle model topological structure is added, and an improved performance index function is adopted, so that a more accurate and reliable optimal path can be found out.

The step of determining the countermeasure decision based on the assigned hit targets includes:

if no hit target is found in the flight process, executing a cruising decision, and flying towards a preset destination;

if a hit target is detected, but the hit target is out of the attack distance of the hit target, situation estimation is carried out, if the estimation result is dominant, a decision of approaching the target is carried out, and if the estimation result is inferior, a decision of keeping away from the target is carried out;

if the hit target is detected and the hit target is within the attack distance of the hit target, situation estimation is carried out, if the estimation result is dominant, decision of the attack target is carried out, and if the estimation result is inferior, decision of the attack target is carried out, wherein the decision of the attack target is far away from the target.

In a further preferred embodiment, the step of determining the countermeasure decision based on the assigned hit targets further comprises: after the behavior decision of all unmanned aerial vehicles in the unmanned aerial vehicle cluster is finished, the lowest threat situation estimated value of all unmanned aerial vehicles is checked, if the threat situation estimated value of a certain unmanned aerial vehicle compared with a certain enemy target is lower than a set dangerous alarm threshold, all friendly aerial vehicles which can detect the enemy target and are dominant in threat situation of the enemy target are searched for aiming at the enemy target, and the behavior decision of the unmanned aerial vehicle in the list is modified to approach or attack the enemy target according to a search result list.

Compared with the prior art, the invention has the following advantages:

the invention is oriented to the countermeasure decision of the unmanned aerial vehicle cluster, and aims to realize more efficient countermeasure decision oriented to the large-scale unmanned aerial vehicle cluster. The theory of a multi-agent system, an air combat situation assessment method and an air combat countermeasure idea are introduced, individual unmanned aerial vehicles are regarded as independent agents, corresponding behavior sets and decision methods are designed, an unmanned aerial vehicle cluster countermeasure decision model is built, and a cooperative countermeasure process among unmanned aerial vehicle clusters is completed in a self-adaptive mode. In addition, the graph model idea is added to the traditional ant colony algorithm by adding a regular term describing the unmanned aerial vehicle graph model topological structure. Meanwhile, the unmanned aerial vehicle cluster countermeasure method of the multi-agent system can avoid adverse targets, support inferior friends and find beneficial targets during cruising, more effectively complete cooperative countermeasure of unmanned aerial vehicle clusters and improve efficiency of unmanned aerial vehicle peak cluster countermeasure fight.

Other advantages that are also present with respect to the present invention will be more detailed in the following examples.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flowchart of an unmanned aerial vehicle cluster countermeasure decision-making method based on multiple agents in an embodiment of the invention.

Fig. 2 is a schematic diagram of a motion state of the unmanned aerial vehicle in a three-dimensional space.

Fig. 3 is a schematic diagram of the movement of the drone in a two-dimensional plane.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. The components of the embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present invention.

Referring to fig. 1, the multi-agent-based unmanned aerial vehicle cluster countermeasure decision-making method provided in the embodiment includes the following steps:

s10, collecting current environment information and situation information of the unmanned aerial vehicle.

The environmental information here includes other friendly states in the unmanned aerial vehicle cluster, enemy states in a perceivable range, terrain information, radar information, missile information, and the like.

The situation information of the unmanned aerial vehicle comprises the speed and the position of the unmanned aerial vehicle.

S20, modeling according to the current environment information and situation information of the unmanned aerial vehicle, including building a motion model of the unmanned aerial vehicle and a threat source model based on a graph.

S30, achieving target hitting task distribution of the unmanned aerial vehicle by utilizing an ant colony algorithm based on the graph model.

S40, determining the countermeasure decision according to the allocated targets.

The movement state of the unmanned aerial vehicle is represented by position and speed. As shown in fig. 2, it is assumed that the instantaneous state of the unmanned aerial vehicle in three-dimensional space can be expressed asWherein: p (P) _p ＝[x _p y _p z _p ]Representing location information->Representing a velocity vector. Use->Representing the state of motion at the next moment, +.>

Target position to be reached by unmanned aerial vehicle at next momentIn the known case, the motion in three-dimensional space is actually +.>At the right->Flying in the two-dimensional plane, and adjusting the speed and direction. As shown in FIG. 3, the current position of the unmanned aerial vehicle is taken as the origin, and the speed is +.>In the direction y _a Shaft, in->Is x _a A shaft. Wherein-> For the target position to be reached by the unmanned aerial vehicle at the next moment,/->Respectively indicate->At x _a And y _a Projection in axial direction, alpha is +.>And->Included angle of->

In y _a Projection in axial direction is +.> At x _a The projection in the axial direction isTherefore->The coordinates of (2) are expressed as +.>Included angleWherein->Velocity vector +.>Is a deflection angle of alpha _T ＝sgnαmin(α,α _max ) Wherein alpha is _max Is the maximum deflection angle within one decision step.

Is obtained from the relationship between the two-dimensional plane and the three-dimensional coordinate systemWherein the method comprises the steps ofAre respectively->And->Is +.about.new speed direction after the unmanned plane flying deflection angle>Unmanned aerial vehicle acceleration coefficient a for controlling speed of unmanned aerial vehicle _up And a deceleration coefficient a _down The expression is as follows:

speed control under unmanned aerial vehicle acceleration condition is

Speed control under unmanned aerial vehicle deceleration condition is

Wherein v is _max And v _min Respectively, the maximum speed and the minimum speed of the unmanned aerial vehicle.

The cluster behavior in the unmanned aerial vehicle cluster countermeasure comprises aggregation and separation of unmanned aerial vehicles. The aggregation of the unmanned aerial vehicle means that the unmanned aerial vehicle can automatically draw close towards the friend machine direction in the movement process. The relative cluster centers of all unmanned aerial vehicles except the local are set as the average value of all friendly machine positions, expressed asWherein P is _ci Represents the center point of the group of friends relative to the ith unmanned aerial vehicle, n represents the total number of unmanned aerial vehicles, and P _pj And (5) representing the position coordinates of other friendly machines except the ith unmanned plane. The separation of the unmanned aerial vehicle refers to a protection measure adopted in the spontaneous aggregation process of the unmanned aerial vehicle to avoid that the distance between partial unmanned aerial vehicles is too small or even collide.

The unmanned aerial vehicle is assumed to be unchanged in height in the flight process, and the enemy defense area is considered to be in a flat region. The method mainly considers the threats of terrains, radars and missiles, and models according to the specific characteristics of various threat sources. Modeling of various threat sources is as follows:

(1) Terrain threat: mainly refers to high peaks at a fixed flight height which may cause an obstacle to the flight of the unmanned aerial vehicle, and thus can also be understood as peak threats. The cone is used for approximately representing the peak, when the flying height of the unmanned aerial vehicle is fixed, the horizontal section of the peak is a circumference, and the radius of the peak and the distance from the unmanned aerial vehicle to the center of the peak are respectively d _T And d, probability of crashing P _T (d) Can be approximated as:

(2) Radar threat: when the threat is enemy Lei Dashi, the threat to the drone is inversely proportional to the fourth power of the distance of the radar. If the maximum detection radius of the radar is d _Rmax The horizontal distance between the unmanned aerial vehicle and the radar is d, so that the probability P of the unmanned aerial vehicle being detected by the enemy radar _R (d) Can be approximated as:

(3) Missile threat: an air missile is a main ground air defense weapon generally, according to the location of a killing area of the missile, the killing area can be approximately waist-drum-shaped, and the radius d of a horizontal cross-section circle is a function of the height and has the largest radius at a certain height. If d _Mmax Is the maximum radius of the missile killing area, and the probability P of the unmanned plane being hit by an enemy missile _M (d) Can be approximated as:

by using the models constructed for various threat sources, the task allocation of the target hitting of the unmanned aerial vehicle is realized by using an ant colony algorithm based on a graph model, wherein ants represent the target hitting task (namely, the unmanned aerial vehicle executing the target hitting task). Specifically, the method comprises the following steps:

s1, constructing a graph according to a threat source model, calculating the total cost of each side in the graph, and giving an initial pheromone value to each side of the graph. The total cost is referred to herein as a weighted sum of the energy cost and threat cost required for the drone to fly on each side.

S2, enabling ants (unmanned aerial vehicles) to start searching from the node closest to the departure point, selecting edges according to the state transition rule, and ending searching by taking the node closest to the target as the end point. The state transition rule here refers to selecting the edge corresponding to the largest state transition probability.

S3, after all ants finish respective hit target selection, calculating the cost of a feasible path according to the performance index of the improved ant colony algorithm, updating the found optimal path, and updating the pheromones on each side in the graph according to the pheromone updating rule, wherein the pheromones are evaporated on the sides without the ants passing through.

Repeating steps S2-S3 until the end condition is reached. The end condition is that the iteration number reaches a set value.

Constraint conditions to be met when completing task allocation refer to safety performance and energy performance of the unmanned aerial vehicle for completing tasks, so task execution costs comprise threat costs and energy costs suffered by the unmanned aerial vehicle. In the traditional ant colony algorithm, each constraint condition of executing tasks is independently considered and finally added into an integral objective function, and e is assumed to be used _i Representing the energy cost, t, of the ith unmanned aerial vehicle to perform tasks _i The threat cost of the i-th unmanned aerial vehicle execution task is represented, and then the performance index of the unmanned aerial vehicle cluster execution task can be written as:

wherein n is the number of unmanned aerial vehicles, and omega, 1-omega respectively represent the weight coefficients of energy cost and threat cost. Wherein the weight coefficient is determined according to the executed task, and if the task attaches importance to the safety of the flight, ω selects a smaller value; if the mission requires the quickness of the drone, ω selects a larger value.

Threat costs for each unmanned aerial vehicle to perform tasks include terrain, radar, missile costs, expressed as follows:

t _i ＝λ ₁ P _T (d _i )+λ ₂ P _R (d _i )+λ ₃ P _M (d _i )

wherein lambda is ₁ ,λ ₂ ,λ ₃ All are weight coefficients reflecting the relative importance of each threat source.

Since the performance index cannot well reflect the overall performance of the unmanned aerial vehicle and the interrelation between the unmanned aerial vehicles, in this embodiment, an improved ant colony algorithm based on a graph model is designed by adding a regular term describing the topological structure of the unmanned aerial vehicle model. Assume that the states of the unmanned aerial vehicle before and after executing tasks are respectively represented by a vector x _i 、h _i Is represented by x _i ,h _i A certain dimension of the vector may be used to characterize a certain index of the drone, such as energy. Consider the overall topology of an unmanned aerial vehicle cluster, topology constraint J ₂ ：

J ₂ ＝0.5*||h _i -x _i || ² s _ij

Wherein s is _ij Is x _i And x _j Is a correlation degree of (a). The improved performance index function of the ant colony algorithm is formed as follows:

wherein,for adjusting the coefficients.

Assuming that the number of pre-hit targets is m, the number of unmanned aerial vehicles is n,representing task T at time T _k State transition probability of transition from unmanned aerial vehicle i to unmanned aerial vehicle j:

wherein R is _c For task T _k The candidate unmanned aerial vehicle set represents all unmanned aerial vehicle sets that unmanned aerial vehicle i can reach; gamma is an information heuristic factor, represents the relative importance of the track, and reflects the role of the accumulated information of the task in the transfer process on the transfer of the task; beta is a desired heuristic factor, which represents the importance degree of heuristic information in selection and reflects the importance degree of heuristic information in the selection of task transfer paths in the transfer process. η (eta) _j (t) is a heuristic function:

d in _j (T _k ) Representing unmanned plane j and task T _k Distance between them. τ _ij (t) represents the value of the pheromone remained on the connection line of the unmanned aerial vehicles i and j at the moment t; all tasks can be found out by only completing one-time cyclic solution, and the optimal performer of the tasks can be found out and the pheromone value can be updated according to the following steps:

wherein ρ represents the pheromone volatilization coefficient, 1- ρ represents the pheromone residual factor, ρ E [0,1 ], Δτ _ij (T) represents T _k And R is _j The information element is enhanced, wherein Q is a constant representing the information intensity, T _k And R is _j All represent tasks.

In the unmanned aerial vehicle cluster countermeasure system, unmanned aerial vehicle individuals make independent decisions according to task and environment information, all unmanned aerial vehicles have a behavior set belonging to the unmanned aerial vehicle individuals, and the behaviors of the unmanned aerial vehicles are fullFoot: (1) the unmanned aerial vehicle can complete the cruising task of a specific destination, (2) the unmanned aerial vehicle and the unmanned aerial vehicle can keep a safe distance, do not fall behind and collide, (3) an attack target (unmanned aerial vehicle) can support the unmanned aerial vehicle of the friend if necessary. The fight behavior set of the unmanned aerial vehicle is as follows: action _i ＝{A ₁ ,A ₂ ,A ₃ ,A ₄ ,A ₅ -wherein: a is that ₁ Represents cruising, A ₂ Representing the approaching target, A ₃ Representing distant target, A ₄ Representing the attack target, A ₅ Representing a support friend machine.

The decision method of five behaviors in the individual behavior set Action of the unmanned plane is as follows:

①A ₁ -cruising: if no enemy target (hitting target allocated by a task) is found in the flight process of the unmanned aerial vehicle, and the behavior of the unmanned aerial vehicle is set to fly towards the preset destination at the moment, the target position vector of the unmanned aerial vehicle in the cruising mode is P _H ＝P _{presupposed destination} 。

②A ₂ -approaching the target: if an enemy target (a hit target allocated by a task) is detected in the cruising process of the unmanned aerial vehicle, but the target is still outside the self attack distance, the unmanned aerial vehicle needs to perform situation estimation on the enemy target. If the result of the situation estimation is dominant, a decision is made to approach the target. When the unmanned aerial vehicle detects a plurality of targets at the same time, situation estimation needs to be carried out on all the targets. Determining the weight of each target according to the situation estimated value, and then determining the decision value P _H ，Wherein: p (P) _Hi Representing the decided target position of the unmanned aerial vehicle i, wherein l represents the number W of multiple targets detected by the unmanned aerial vehicle i _j Weight factor representing target j, P _ij Representing the difference between the detected target j and the position vector of the unmanned aerial vehicle i,/and>the calculation formula of (2) is as follows:

weighting factor W of target j _j The calculation is as follows:

③A ₃ away from the target: if an enemy target (a hit target allocated by a task) is detected in the cruising process of the unmanned aerial vehicle, and the situation estimation result is a disadvantage, a decision of keeping away from the target must be made no matter whether the enemy target has entered the attack range of the unmanned aerial vehicle. When there are several detected targets, threat situation estimation is needed to all targets, each target weight factor is calculated, and then the decision value P is determined _H 。

④A ₄ -attack target: if the target (the hitting target of task allocation) enters the fire attack distance of the unmanned aerial vehicle and the situation evaluation result of the unmanned aerial vehicle and the target enemy plane is advantageous, the fire hitting of the target unmanned aerial vehicle is needed. If a plurality of targets meet the condition, selecting the target with the greatest dominance degree.

⑤A ₅ -support friend machine: when the unmanned aerial vehicle detects a plurality of targets (other enemy machines except for the hit targets allocated by the task), the targets which are extremely threatening to the friend mechanism are preferentially attacked. The specific practice is to make the computer execute according to the action A ₁ To A ₄ After finishing the behavior decision of all unmanned aerial vehicles, checking the lowest threat situation estimated value of all unmanned aerial vehicles, and setting a dangerous alarm value K _dangerous . If the threat situation estimated value of a certain unmanned aerial vehicle compared with a certain enemy target is lower than K _dangerous And searching all friends machines which can detect the dangerous target and are advantageous to the threat situation of the dangerous target aiming at the dangerous target. And according to the search result list, modifying the behavior decision of the unmanned aerial vehicle in the list to approach or attack the dangerous target.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention.