CN114373032B - Three-dimensional mesh deformation method based on contour skeleton and related device - Google Patents

Disclosure of Invention

The application aims to provide a three-dimensional grid deformation method based on a contour line skeleton and a related device, and aims to solve the technical problem that in the prior art, the reconstructed skeleton does not accord with basic characteristics of a graph due to blind reduction of feature vectors.

The first aspect of the application provides a three-dimensional grid deformation method based on a contour line framework, which comprises the following steps:

Constructing a Laplacian matrix of the three-dimensional grid model;

reconstructing the characteristic vector of the Laplace matrix to obtain a characteristic framework of the three-dimensional grid model;

Grouping and optimizing the characteristic frameworks based on the contour lines of the three-dimensional network model to obtain contour line frameworks of the three-dimensional network model, wherein certain characteristic values on the contour lines are equal;

And performing interactive editing on the contour line skeleton, and optimizing the contour line skeleton based on the difference value between the three-dimensional grid model and the contour line skeleton to obtain a target three-dimensional grid model.

Optionally, the reconstructing the feature skeleton of the three-dimensional grid model by using the feature vector of the laplace matrix includes:

performing feature decomposition on the Laplace matrix to obtain feature vectors of the three-dimensional grid model;

Extracting feature vectors of the three-dimensional grid model to obtain first k vectors;

reconstructing the three-dimensional grid model based on the first k feature vectors to obtain coordinates of the reconstructed three-dimensional grid model vertexes.

Optionally, a certain feature value on the contour line is one of an x-axis component, a y-axis component, or a z-axis component of coordinates of the vertex of the three-dimensional mesh model.

Optionally, grouping and optimizing the feature skeleton based on the contour line of the three-dimensional network model, and obtaining the contour line skeleton of the three-dimensional network model includes:

Generating a set of contour lines on the three-dimensional grid model based on a contour line generation algorithm;

Acquiring one vertex of a triangular patch in the three-dimensional grid model;

determining an interpolation point on one edge of a triangular patch of the three-dimensional grid model;

And further determining another contour point on the other two sides of the triangular surface patch, wherein the contour line is a closed curve, and two intersection points exist between the contour line and the triangular surface patch.

Optionally, the grouping and optimizing the feature skeleton based on the contour line of the three-dimensional network model, and obtaining the contour line skeleton of the three-dimensional network model further includes:

Taking a first non-zero vector of a Laplace matrix of the three-dimensional grid model as an equivalent scalar;

Calculating a contour image of the three-dimensional grid model according to an equivalent scale of a Laplace matrix of the three-dimensional grid model;

corresponding the contour lines in the contour line pattern and the vertexes of the triangular surface patches to obtain vertexes of a three-dimensional grid model based on contour line grouping;

And averaging the components of the eigenvectors of the Laplace matrix of the three-dimensional grid model according to the grouping.

Optionally, the step of associating the contour lines of the three-dimensional mesh model with the vertices of the triangular patch to obtain vertices of the three-dimensional mesh model based on contour line grouping includes:

Taking one contour point on one group of contour lines;

Judging whether the contour point is on the edge of the triangular patch or not;

If the contour point is on the edge of the triangular patch, adding the vertex to a Group _n;

And taking the next critical point of the contour point, and adding the vertex of the edge where the critical point is positioned to the Group _n until all contour points on the contour line are traversed.

Optionally, the step of associating the contour lines of the three-dimensional mesh model with the vertices of the triangular patches to obtain vertices of the three-dimensional mesh model based on contour line grouping, further includes:

a group of contour lines are taken, and when the vertex is added, whether the vertex has the corresponding contour line or not is judged;

if the vertex already has the corresponding contour lines, judging the distance between the vertex and the two contour lines;

if the vertex is closer to the other set of contours, then the vertex is deleted from Group _n and added to Group _n+1 corresponding to the other set of contours.

The second aspect of the present application provides a three-dimensional mesh deformation apparatus based on a contour skeleton, comprising:

a matrix construction unit for constructing a Laplacian matrix of the three-dimensional grid model;

The framework reconstruction unit is used for reconstructing the characteristic vector of the Laplace matrix to obtain the characteristic framework of the three-dimensional grid model;

The grouping optimization unit is used for grouping and optimizing the characteristic frameworks based on the contour lines of the three-dimensional network model to obtain contour line frameworks of the three-dimensional network model, wherein certain characteristic values on the contour lines are equal;

And the editing processing unit is used for interactively editing the contour line framework, and optimizing the contour line framework based on the difference value between the three-dimensional grid model and the contour line framework to obtain a target three-dimensional grid model.

A third aspect of the application provides a computer device comprising a memory and a processor, the memory having stored therein computer readable instructions which, when executed by the processor, cause the processor to perform the steps of causing the processor to perform the three-dimensional mesh morphing method based on a contour skeleton as described above.

A fourth aspect of the application provides a computer device, a storage medium storing computer readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of a three-dimensional mesh morphing method based on a contour skeleton as described above.

According to the three-dimensional grid deformation method based on the contour line skeleton, the contour line of the three-dimensional network model is adopted to group and optimize the characteristic skeleton, so that the contour line skeleton of the three-dimensional network model is obtained, certain characteristic values on the contour line are equal, then the contour line skeleton is subjected to interactive editing, the contour line skeleton is optimized based on the difference value between the three-dimensional grid model and the contour line skeleton, and a target three-dimensional grid model is obtained, so that more basic characteristics of the three-dimensional grid model can be reserved, part of grid details can be discarded, the skeleton model is visually closer to an original grid model, and a triangular patch is basically collapsed on one line, so that the structure is more compact.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.

As will be appreciated by those skilled in the art, the term "device" as used herein includes both devices that are wireless signal receivers, devices that have only a transmission signal receiver without transmission capability, and devices that include transmission and reception hardware, devices that have reception and transmission hardware capable of performing bi-directional communications over a bi-directional communication link. Such devices may include cellular or other communication devices having a single-wire display or a multi-wire display or cellular or other communication devices without a multi-wire display, PCS (Personal Communications Service, personal communication system) that may combine voice, data processing, facsimile and/or data communication capabilities, PDA (Personal DIGITAL ASSISTANT ) that may include a radio frequency receiver, pager, internet/intranet access, web browser, notepad, calendar and/or GPS (Global Positioning System ) receiver, conventional palm-top computer or other device having and/or including a conventional laptop and/or palm-top computer or other device of radio frequency receiver.

FIG. 1 is a flow chart of one embodiment of a three-dimensional mesh deformation method based on a contour skeleton.

The technical scheme of the application aims at the reconstruction and deformation of a three-dimensional grid model, wherein the three-dimensional grid model is stored in a database, the reconstruction of the three-dimensional grid model is generally based on the main characteristics of the three-dimensional grid model obtained by a framework, the framework characteristics are enabled to meet the deformation requirement of the three-dimensional grid model obtained in real time after the reconstruction, and then the reconstructed framework is filled and edited to obtain the deformed three-dimensional grid model.

In this embodiment, the three-dimensional grid deformation method based on the contour skeleton includes:

s1000, constructing a Laplace matrix of the three-dimensional grid model;

firstly, acquiring a three-dimensional grid model, wherein the three-dimensional grid model is derived from a three-dimensional grid model stored in a database, analyzing the three-dimensional grid model after acquiring the three-dimensional grid model, extracting a characteristic vector of the three-dimensional grid model, and constructing a Laplace matrix of the three-dimensional grid model, wherein the Laplace matrix is constructed by adopting a complementary weight.

S2000, reconstructing the characteristic vector of the Laplace matrix to obtain a characteristic skeleton of the three-dimensional grid model;

In this embodiment, after the laplace matrix is constructed, the laplace matrix is subjected to feature decomposition, and reconstruction is performed based on feature vectors, so as to obtain coordinates of grid vertices of the three-dimensional grid model after reconstruction.

The feature vectors and the low-frequency signals coded by the feature values mainly record the overall features of the grid, and the partially abandoned feature vectors enable the reconstructed feature skeleton to have inconsistent features with the original three-dimensional grid model.

S3000, grouping and optimizing the characteristic frameworks based on the contour lines of the three-dimensional network model to obtain contour line frameworks of the three-dimensional network model, wherein certain characteristic values on the contour lines are equal;

The three-dimensional mesh model may also be referred to as a triangular mesh model, which is a network model composed of a plurality of triangular surfaces, which may also be referred to as triangular patches.

In this embodiment, the feature skeleton is grouped and optimized based on the contour lines of the three-dimensional network model, and the purpose of the method is to reconstruct the three-dimensional network model by adopting fewer feature vectors at first, so that the feature skeleton can fully retain the basic features of the three-dimensional network model, and then further optimize the obtained feature skeleton by adopting other modes to obtain a more ideal feature skeleton.

In this embodiment, a triangular mesh contour line generation algorithm, which may also be referred to as a triangular mesh contour line generation algorithm, is used to generate a set of contour lines on the three-dimensional mesh model, where certain feature values of points on the set of contour lines are equal. The feature values are one of x-axis component, y-axis component or z-axis component of vertex coordinates of the original three-dimensional grid model, and can be a component of a certain feature value.

And grouping and optimizing the characteristic frameworks by constructing contour lines to obtain the contour line frameworks of the three-dimensional network model, namely the more ideal characteristic frameworks.

S4000, performing interactive editing on the contour line framework, and optimizing the contour line framework based on the difference value between the three-dimensional grid model and the contour line framework to obtain a target three-dimensional grid model.

After the contour line skeleton is obtained in this embodiment, the skeleton may be interactively edited by using the unit 3d, and details of the grid (a difference between the original grid and the skeleton) may be restored to the contour line skeleton, to obtain the target three-dimensional grid model.

The embodiment of the application adopts the contour lines of the three-dimensional network model to group and optimize the characteristic frameworks to obtain the contour line frameworks of the three-dimensional network model, wherein certain characteristic values on the contour lines are equal, then the contour line frameworks are interactively edited, and the contour line frameworks are optimized based on the difference value between the three-dimensional grid model and the contour line frameworks to obtain the target three-dimensional grid model, so that more basic characteristics of the three-dimensional grid model can be reserved, part of grid details can be discarded, the framework model is more similar to the original grid model in vision, and the triangular patch is basically collapsed to one line, so that the structure is more compact.

FIG. 2 is a schematic flow chart diagram of one embodiment of a method for reconstructing a feature skeleton according to the present application.

In this embodiment, the reconstructing the feature skeleton of the three-dimensional mesh model by using the feature vector of the laplace matrix includes:

S2100, performing feature decomposition on the Laplace matrix to obtain feature vectors of the three-dimensional grid model;

S2200, extracting feature vectors of the three-dimensional grid model to obtain first k vectors;

s2300, reconstructing the three-dimensional grid model based on the first k feature vectors to obtain coordinates of the reconstructed three-dimensional grid model vertexes.

First, a laplace matrix of the three-dimensional mesh model is calculated, wherein the laplace matrix of the three-dimensional mesh model is defined as l=d ^-1 W. In this embodiment, a laplace matrix of the mesh is constructed by adopting the complementary weights, and the construction mode of the weight matrix W is as follows:

d is a diagonal matrix, d=diag (a ₀,a₁,…,a_n-1),a_i represents the Voronoi area of the ith vertex.

After the Laplace matrix is constructed, carrying out feature decomposition on the Laplace matrix L, extracting feature vectors of the three-dimensional grid model, and carrying out reconstruction by using the first k feature vectors to obtain coordinates of the vertexes of the reconstructed three-dimensional grid model.

Further referring to fig. 8, fig. 8 (a) is an original three-dimensional mesh model, which has 7056 vertices, and fig. 8 (b), (c), and (d) are the results of reconstructing the original three-dimensional mesh model with 6, 10, and 15 feature vectors, respectively.

If three eigenvalues and eigenvectors are used for reconstruction, the difference between the reconstructed eigenvectors and the original three-dimensional grid model is increased. Because the frequency oscillation amplitude of the first few eigenvalue codes is small, the low-frequency signals of the first few eigenvalue codes are more important to record the integral characteristics of the original three-dimensional grid model, and partial eigenvectors are discarded to enable the reconstructed eigenvectors to be inconsistent with the characteristics of the original three-dimensional grid model.

Therefore, in the application, 6-15 feature vectors are generally adopted to reconstruct an original three-dimensional grid model, so that the reconstructed feature skeleton can fully reserve the basic features of the three-dimensional grid model, and then the feature skeleton is further optimized to obtain a more ideal feature skeleton.

According to the embodiment of the application, the feature vectors of the three-dimensional grid model are extracted, the three-dimensional grid model is reconstructed based on the first k feature vectors to obtain the coordinates of the vertexes of the reconstructed three-dimensional grid model, wherein the number of the feature vectors is generally 6-15, the feature skeleton obtained by the embodiment of the application can maintain the basic features of the original three-dimensional grid model, the large difference between the reconstructed feature skeleton and the original three-dimensional grid model caused by blindly reducing the feature vectors is avoided, and meanwhile, excessive feature vector calculation is not adopted in the reconstruction process, so that the three-dimensional grid deformation efficiency is effectively improved.

FIG. 3 is a schematic flow chart of one embodiment of the present application for obtaining a contour skeleton.

In this embodiment, the grouping and optimizing the feature skeleton based on the contour line of the three-dimensional network model, to obtain the contour line skeleton of the three-dimensional network model includes:

S3100, generating a group of contour lines on the three-dimensional grid model based on a contour line generation algorithm;

s3200, obtaining one vertex of a triangular patch in the three-dimensional grid model;

S3300, determining an interpolation point on one edge of a triangular patch of the three-dimensional grid model;

S3400, further determining another equivalence point on the other two sides of the triangular patch, wherein the contour line is a closed curve, and two intersection points exist between the contour line and the triangular patch.

In this embodiment, a contour line generating algorithm is first used to generate a set of contour lines on the three-dimensional grid model, where a method for drawing contour lines may be classified into an interpolation method and a curve fitting method according to different construction modes of the contour line algorithm.

Because the vertices of the three-dimensional grid are not coincident with the contour points on the contour lines in most cases, interpolation contour points need to be found on the edges of similar triangular patches.

Obtaining a vertex of a triangular surface patch in the three-dimensional grid model, wherein the vertex satisfies (z _d-z_a)(z_d-z_b) <0, z _d represents a value of a contour line, z _a and z _b represent z values of two vertices in a triangle, if the inequality is satisfied, it means that an interpolation point can be found on a triangle side (a, b) so that the value of the interpolation point is equal to the value of the contour line, and the position of the interpolation point in a three-dimensional Cartesian coordinate system is:

v=v_a+(v_b-v_a)(z_d-z_a)/(z_b-z_a)

The contour is a closed curve which has two intersecting points with the triangular surface patch, so if one contour interpolation exists on one side of the triangle, another equivalent point can be found on the other two sides of the triangle.

Referring to fig. 9, fig. 9 provides a contour constructor for the igl graphic library used in the present embodiment, the calculated 30 contour images.

Further referring to fig. 4, the grouping and optimizing the feature skeleton based on the contour line of the three-dimensional network model, to obtain the contour line skeleton of the three-dimensional network model further includes:

s3500, taking a first non-zero vector of a Laplacian matrix of the three-dimensional grid model as an equivalent scalar;

S3600, calculating a contour image of the three-dimensional grid model according to an equivalent scale of a Laplace matrix of the three-dimensional grid model;

S3700, corresponding the contour lines in the contour line pattern and the vertexes of the triangular surface patches to obtain vertexes of a three-dimensional grid model based on contour line grouping;

s3800, averaging components of eigenvectors of a Laplace matrix of the three-dimensional grid model according to the grouping.

In this embodiment, the first non-zero vector of the laplace matrix of the three-dimensional mesh model is used as an equivalent scalar, each contour line and each vertex are corresponding, a point corresponds to only one contour line and corresponds to the contour line closest to the plane of the three-dimensional mesh model, so that the vertices of the three-dimensional mesh can be divided into a plurality of groups according to the contour lines.

Further, components of eigenvectors of the laplace matrix of the three-dimensional mesh model are averaged in groups, i.e.:

u_i,k＝t_i,j,k∈Group_j

FIG. 5 is a flow chart illustrating one embodiment of grouping vertices of a model according to the present application in contours.

In this embodiment, the step of associating the contour lines of the three-dimensional mesh model with the vertices of the triangular patch to obtain vertices of the three-dimensional mesh model based on contour line grouping includes:

s3710, any one contour point on a group of contour lines is selected;

s3720, judging whether the contour point is on the edge of the triangular patch;

S3730, if the contour point is on the edge of the triangular surface patch, adding the vertex to a Group _n;

s3740, taking the next critical point of the contour point, and adding the vertex of the edge where the critical point is located to the Group _n until all contour points on the contour line are traversed.

Further, the step of associating the contour lines of the three-dimensional mesh model with the vertices of the triangular patches to obtain vertices of the three-dimensional mesh model based on contour line grouping, further includes:

S3750, a group of contour lines are taken in addition, and when the vertex is added, whether the vertex has the corresponding contour line or not is judged;

s3760, judging the distance between the vertex and two contour lines if the corresponding contour lines exist in the vertex;

And S3770, deleting the vertex from the Group _n and adding the vertex to the Group _n+1 corresponding to the another Group of contour lines if the vertex is closer to the another Group of contour lines.

Referring to fig. 7, fig. 7 is an algorithm for determining mesh vertices corresponding to each set of contours of corresponding points.

Specifically, if a point a is on a side (i, j) of the triangular patch, then vertex v _i、v_j is added to Group _n, then the next critical contour point b of a is taken, the vertex of the side where point b is located is added to Group _n, all contours on the contour are traversed all the time, and the vertices at both ends of the side where all contours are located are added to Group _n.

And repeating the steps by taking a Group of contour lines, adding the corresponding vertex to the Group _n+1, judging whether the corresponding contour line exists at the vertex of the side where the contour line exists or not when the corresponding vertex exists, judging the distance between the vertex and the two contour lines if the corresponding contour line exists, deleting the vertex from the original corresponding Group _n if the vertex is closer to the current contour line, adding the vertex to the current corresponding Group _n+1, and repeating the steps until all the contour lines are traversed.

The formula according to the reconstruction of the graph signals is as follows:

the feature frameworks are grouped and optimized based on the contour lines of the three-dimensional network model, and the framework obtained by the three-dimensional network model is called a contour line framework.

As shown in fig. 10, fig. 10 (a) is an original three-dimensional grid model, and fig. 10 (b) is a contour line skeleton obtained by grouping and optimizing the feature skeleton based on a contour line of the three-dimensional grid model, because components of feature vectors corresponding to each group of grid vertices are equal, the three-dimensional grid after reconstruction is simplified, and more features are reserved.

As shown in fig. 11, fig. 11 (a) is an original camel skeleton model, fig. 10 (b) is an optimized camel skeleton model, and it can be seen from the figure that the optimized camel skeleton model not only retains more basic features of the original camel skeleton model, but also can discard more grid details compared with a feature skeleton, so that the skeleton model is visually closer to the original three-dimensional grid model.

The three-dimensional grid deformation method based on the contour line provided by the embodiment of the application can keep basic characteristics of more original skeleton models, can discard more grid details, and can not blindly reduce feature vectors, so that the skeleton models are more similar to the original three-dimensional grid models in vision, and the structure is more compact.

Fig. 12 is a schematic structural diagram of a three-dimensional grid deformation device based on a contour skeleton according to an embodiment of the present application.

In this embodiment, the three-dimensional grid deformation device based on a contour skeleton includes:

a matrix construction unit for constructing a Laplacian matrix of the three-dimensional grid model;

The framework reconstruction unit is used for reconstructing the characteristic vector of the Laplace matrix to obtain the characteristic framework of the three-dimensional grid model;

The grouping optimization unit is used for grouping and optimizing the characteristic frameworks based on the contour lines of the three-dimensional network model to obtain contour line frameworks of the three-dimensional network model, wherein certain characteristic values on the contour lines are equal;

And the editing processing unit is used for interactively editing the contour line framework, and optimizing the contour line framework based on the difference value between the three-dimensional grid model and the contour line framework to obtain a target three-dimensional grid model.

Because the three-dimensional grid deformation device based on the contour line skeleton is a device corresponding to the three-dimensional grid deformation method based on the contour line skeleton one by one, the implementation principle is the same as that of the three-dimensional grid deformation method based on the contour line skeleton, and the description is omitted here.

In this embodiment, referring to fig. 13, a basic structural block diagram of a computer device is provided.

The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected by a system bus. The nonvolatile storage medium of the computer equipment stores an operating system, a database and computer readable instructions, the database can store a control information sequence, and when the computer readable instructions are executed by a processor, the processor can realize a three-dimensional grid deformation method based on a contour line framework. The processor of the computer device is used to provide computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may have stored therein computer readable instructions that, when executed by the processor, cause the processor to perform a three-dimensional mesh morphing method based on a contour skeleton. The network interface of the computer device is for communicating with a terminal connection. It will be appreciated by those skilled in the art that the structure shown in FIG. 13 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor may be implemented in at least one hardware form of DSP (DIGITAL SIGNAL Processing), FPGA (Field-Programmable gate array), PLA (Programmable Logic Array ). The processor 1001 may also include a main processor for processing data in the awake state, which is also called a CPU (Central Processing Unit ), and a coprocessor for processing data in the standby state, which is a low-power-consumption processor. In some embodiments, the processor may incorporate a GPU (Graphics Processing Unit, image processor) for rendering and rendering of content to be displayed by the display screen. In some embodiments, the processor 1001 may also include an AI (ARTIFICIAL INTELLIGENCE ) processor for processing computing operations related to machine learning.

The present application also provides a storage medium storing computer readable instructions that, when executed by one or more processors, cause the one or more processors to perform the three-dimensional mesh morphing method based on a contour skeleton as described in any of the above embodiments.

The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory is used to store at least one instruction for execution by a processor to implement a three-dimensional mesh morphing method based on a contour skeleton provided by an embodiment of the method in the present application.

Fig. 14 is a schematic structural diagram of a server according to an embodiment of the present application. The server is used for implementing the three-dimensional grid deformation method based on the contour line skeleton provided in the embodiment. Specifically, the present application relates to a method for manufacturing a semiconductor device.

The server includes a Central Processing Unit (CPU), a system memory including a Random Access Memory (RAM) and a Read Only Memory (ROM), and a system bus connecting the system memory and the central processing unit. The server also includes a basic input/output system (I/O system) to facilitate the transfer of information between the various devices within the computer, and a mass storage device for storing an operating system, application programs, and other program modules.

The basic input/output system includes a display for displaying information and an input device such as a mouse, keyboard for a user to input information. Wherein the display and the input device are connected to the central processing unit via an input-output controller connected to the system bus. The basic input/output system may also include an input/output controller for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus.

The mass storage device is connected to the central processing unit through a mass storage controller (not shown) connected to the system bus. The mass storage device and its associated computer-readable media provide non-volatile storage for the server. That is, the mass storage device may include a computer readable medium (not shown) such as a hard disk or CD-ROM drive.

The computer readable medium may include computer storage media and communication media without loss of generality. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory, or other solid state memory technology, CD-ROM, DVD, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the one described above. The above-described system memory and mass storage devices may be collectively referred to as memory.

In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.

Embodiments of the present invention are directed to, and combinations of, the flow diagrams and/or each block in the block diagrams of the methods, terminal devices (systems) and computer program products according to embodiments of the present invention. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In addition, each functional device in each embodiment of the present invention may be integrated in the same data processing device, each device may exist alone physically, or two or more devices may be integrated in the same device.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. Several means or computer means recited in the computer means claims can also be embodied by one and the same computer means by either software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.

The foregoing description of the preferred embodiments of the application is not intended to be limiting, but rather is intended to cover all modifications, equivalents, or alternatives falling within the spirit and principles of the application.