CN109802913B - Fusion conference implementation method and device, electronic equipment and readable storage medium - Google Patents

FIG. 1 is a schematic illustration of an implementation environment according to the present disclosure. The implementation environment includes: a

110, a

120 installed with a social Network application and a PSTN (Public Switched Telephone Network)

130 connected to the

110.

The association between the

110 and the

120 and the

130 includes the network association and/or protocol of hardware and the data association between the two. The

110 may receive Voice and/or video data transmitted by the social network application of the

120 through a VOIP telephone (Voice over Internet Protocol) and receive Voice and/or video data transmitted by the

130 through a public switched telephone network, respectively, and forward the Voice and/or video data to other

120 and the

130, thereby implementing convergence and interworking between the VOIP telephone of the social network application and the PSTN telephone, and implementing a convergence conference that can be participated in by the VOIP telephone of the social network application and the PSTN telephone.

Referring to fig. 2, fig. 2 is a schematic diagram of a server structure according to an embodiment of the present invention. The

200 may have a large difference due to different configurations or performances, and at least includes one or more Central Processing Units (CPUs) 222 (e.g., one or more processors) and a

232, and one or

230 storing an

242 or

244

(e.g., one or more mass storage devices).

232 and

230 may be, among other things, transient storage or persistent storage. The program stored in the

230 includes at least one or more modules (not shown), each of which includes at least a series of instruction operations for the

200. Further, the

222 is configured to communicate with the

230, and execute a series of instruction operations in the

230 on the

200.

200 also includes one or

226, one or more wired or wireless network interfaces 250, one or more input-

258, and/or one or

241, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so forth. The steps performed by the

110 described in the embodiments of fig. 3, 5-7 below may be based on the server architecture shown in fig. 2.

FIG. 3 is a flowchart illustrating a method for converged conferencing implementation, in accordance with an exemplary embodiment. The method is applicable to a scope and a body of execution of the converged conference implementation, for example, the method is used for the

110 of the implementation environment shown in fig. 1. As shown in fig. 3, the method for implementing a converged conference, which may be performed by the

110, may include the following steps.

The conference service refers to a conference resource that a social network application (such as instant messaging software APP) needs to use the

110. The

110 assigns a conference access interface for access to the social networking application. Specifically, the system architecture of the

110 is as shown in fig. 4, and the

110 may include a conference management server 41 and an audio/video

42

43.

The conference management server 41 is responsible for configuration management of each access conference service and call management work of conference members. The audio/video

42 is configured to perform relevant processing on the audio/video streams of each conference member during the call. The

43 is responsible for access of signaling and transit of media streams.

It should be further noted that the

110 may be deployed in a public cloud or a private cloud, and support different social network applications to share the resources of the

110. When deployed in a PAAS (Platform-as-a-Service) mode, the public cloud supports all social network applications to share conference resources of the

110 in an account isolation manner, and share conference services; when the conference server is deployed in a private cloud, the

110 is supported to be deployed in a local area network designated by a user, so that the sharing and isolation of conference resources are ensured.

The

110 may access a plurality of conference services at the same time, where the plurality of conference services refer to resources of the

110 for which a plurality of third-party clients (instant messaging software APPs) exist, and each conference service may create a plurality of conferences at the same time, so that the conference management server 41 may number each conference service for differentiation, and number each conference in each conference service for differentiation. For example, the number of each conference service may apply for the host number of the company for the service, and each conference in the service may specify a corresponding conference code (similar to an extension number).

After the

110 is deployed in the public cloud or the private cloud, the social network application may first come to the conference configuration management and call management cluster 41 to apply for a conference service, and obtain a conference access port. The social network application invokes the conference access interface to send VOIP signaling containing a converged conference creation request to the

43 of the

110, the request containing the designated conference member. Examples of the method include a speech encoding method, a sampling rate, and a packet transmission interval of a designated conference member, and an IP (internet protocol) address and a port of the designated conference member. If the designated conference member is a user of

130, a dial-in number (i.e., telephone number) or the like for the designated conference member may be included in the request.

it should be noted that the conference members specified for the converged conference creation request may include a plurality of users using the

130 and a plurality of users using the social networking application. To illustrate how a PSTN phone and a VOIP phone of a social networking application may be implemented to participate in a conference together, the present disclosure assumes that the conference members specified by the converged conference creation request are a plurality of

130 users and a plurality of users using the social networking application. The technical scheme provided by the disclosure is also applicable to the case of a VOIP telephone with only PSTN telephones or social network applications. Since the network type of the PSTN phone is a telephone network and the network type of the VOIP phone of the social networking application is an internet network, the network types of the obtained conference members include an internet network and a telephone network.

It should be noted that the conference link is a conference access address, each conference has a corresponding conference access address, the

43 issues the conference link to the social network application of the conference member, when the conference link is triggered by a user of the social network application, the

43 establishes a connection with the social network application, and the conference member using the social network application can access the conference. Therefore, the social network application can be accessed by using the most common http/https mode without using an embedded API (application programming interface), and the development difficulty of the social network application is greatly reduced. Whatever the social networking application, the conference may be accessed only according to the standard http protocol provided by the

110.

The telephone access information is information for conference members in the telephone network to access. For example, the telephone access information may be a dial-in number (similar to a host number) and a conference access code (similar to an extension number) corresponding to a certain conference, and a conference member in the telephone network may establish a data transmission channel between the

110 and the conference member in the telephone network by inputting the dial-in number and the conference access code in a conventional telephone dialing manner.

Specifically, after the conference management server 41 of the

110 generates a conference link for the conference members in the internet and generates conference access information for the conference members in the telephone network, the conference members using the social network application can establish a data transmission channel with the

43 by accessing the address provided by the conference link. For the conference members in the telephone network, the

43 of the

110 may call each conference member in the telephone network (i.e., issue a telephone call request, where the telephone call request is one of the telephone access information) according to the dial-in number of each conference member in the telephone network indicated in the merged conference creation request, at this time, the

130 device of the conference member in the telephone network starts to vibrate or ring, and the conference member in the telephone network establishes a data transmission channel with the

43 after answering.

After the conference members in the telephone network and the conference members using the social networking application establish data transmission channels with the

43, the

110 can be used as a transfer station for data transmission between the conference members in each telephone network and the conference members using the social networking application. For example, audio and video data sent by a certain conference member is forwarded to other conference members.

the deployed

43 issues the conference link to the social network application where the conference member is located through the internet;

when a conference member initiates a connection between itself and the

43 through the triggering of the conference link in the social network application in the internet network, the

43 responds to the connection initiated by the conference member in the internet network and establishes a connection with the social network application in which the conference member in the internet network is located.

That is, after the conference management server 41 of the

110 generates a conference link for a conference member whose network type is the internet, the

43 of the

110 issues the conference link to the social network application of the conference member in the internet through the internet. Each conference member may establish a connection with the

43 by clicking on the conference link in the social networking application to access the address provided by the conference link. After the conference member establishes the connection with the

43 in the internet, it indicates that the conference member in the internet accesses the

110, so that the

110 can receive the data sent by the conference member in the internet, where the data may be an audio/video media stream.

The internet uses the VOIP phone of the social network application, the signaling between the VOIP phone of the social network application and the

43 is transmitted by using a private protocol, the conference member uses the

130 in the phone network, and the

130 uses the SIP protocol to access the operator PSTN network. The data stream uploaded to the

110 by each conference member may be transmitted by using an RTP (Reliable Transport Protocol) or an encrypted RTSP (Real Time Streaming Protocol) Protocol.

The number of the

43 may be multiple, a plurality of the

43 form an access cluster, each

43 may be deployed all over the country, and the conference management server 41 may allocate the

43 to each accessed conference member according to the network type (VOIP type or PSTN type) of the conference member, the regional location of the conference member, the telecom operator information, and the load condition of each

43, so as to ensure the best access quality and call quality of each conference member.

that is, the

43 of the

110 initiates a signaling access procedure to conference members in the telephone network in two forms:

first, the conference management server 41 of the

110 generates a phone call request, and the

43 sends the phone call request to the conference member in the telephone network according to the dial-in number (i.e. phone number) of the conference member in the telephone network, and the conference member in the telephone network answers the phone call (i.e. responds to the phone call request), which indicates that the signaling access process is answered, and establishes a data transmission channel with the conference member in the telephone network.

In the second mode, the conference management server 41 of the

110 generates a conference access code (including a dial-in number and a conference code), the conference access code can be issued to a conference member in the telephone network by the

43 of the

110, the conference member in the telephone network sends an access request to the

43 according to the dial-in number and the conference code by dialing a telephone, at this time, the

110 is equivalent to a telephone, the conference member in the telephone network calls the

110, and when the

110 responds to the access request, it indicates that a signaling access process is answered, and a data transmission channel with the conference member in the telephone network is established.

When the signaling access procedure is answered, the

43 sets up a telephone channel in the telephone network between itself and the conference members in the telephone network, which are accessed via the telephone channel.

The signaling access process is sent in the above manner, and when the signaling access process is answered, the conference member in the telephone network accesses the

110, a data transmission channel between the conference member in the telephone network and the

110 is established, and through the data transmission channel, the

43 of the

110 can receive the audio/video media stream sent by the conference member in the telephone network.

Through the description of the above exemplary embodiment, a data transmission channel between the

110 and the social network application where the

130 and the conference member in the internet are located may be established, the

110 may receive the audio and video media stream transmitted by the social network application where the

130 and the conference member in the internet are located, and the

110 serves as an interface for connecting between each conference member, and may implement protocol conversion and signaling layer intercommunication between the

130 and the social network application where the conference member in the internet is located.

According to the actual conference requirement, under the access of each conference member, the conference management server 41 of the

110 can control the

43 to stop receiving the audio/video media stream sent by the designated conference member according to the signaling sent by the conference host, so as to forbid the designated conference member; or, limiting the access of the conference members, thereby locking the conference; or, the connection with a certain conference member is disconnected, so as to hang up the participation of the conference member; alternatively, the conference is terminated by disconnecting each conference member.

The conference management server 41 deployed by the

110 responds to the conference service access request, and allocates a conference access interface for the access of the social network application, so that the social network application calls the conference access interface to send a converged conference creation request to the

110. The conference management server 41 generates a configuration file for managing the conference service according to the conference service configuration information carried by the conference service access request.

Specifically, the configuration file may include a voice prompt management file, a conference function management file, and a conference capacity management file. The voice prompt management file is used for controlling the

43 to send voice prompt data to an accessed conference member by being called when any conference member accesses. Such as "welcome xxx teleconferences" played when conference members access. This voice prompt data may be provided by the social networking application to the conference management server 41, or default voice prompt data may be selected for use. When receiving the signaling for accessing any conference member, the

110 may send voice prompt data to the accessed conference member by calling the configuration file.

For example, when a conference member initiates an access request to the

43 through a conference access code and the access request is responded, indicating that the conference member is accessed, a voice prompt "welcome to xxx conference call" is sent to the conference member. For another example, when a conference member responds to a call request from the

43, indicating that the conference member is joined in the conference, a voice prompt "welcome to xxx conference call" may be sent to the conference member. And sending a voice prompt of 'XX joining the conference' to other conference members according to the requirement. It should be noted that the social networking application, when sending a conference service access request to the

110, provides a language prompt about what scenes to play in. And if the user just inputs the dial-in number, prompting the welcome word and prompting the input of the conference code, and when the user inputs the wrong conference code, continuing to prompt the next time and the like. IVR prompts are also typically configured when a new conference member joins. In brief, voice prompts can be configured in the change of the conference members or the interaction scenes with the conference members.

The conference function management file is used for processing data sent by each conference member through the called and controlled deployed audio and video

42 and controlling the

43 to send conference reminding information to each conference member when the conference members are accessed in the internet and the telephone network. The conference functions may include conference recording, conference reminders, conference time limits, encoding formats, conference encryption, and the like. The conference management server 41 of the

110 generates a conference function management file according to the conference service function (which may include one or more of the functions) indicated by the conference service access request. Under the access of each conference member, the

110 may implement a corresponding conference function by calling the conference function management file, for example: the audio/video

42 is controlled to record, encode, encrypt and the like the audio/video media streams sent by each conference member, the

43 is controlled to send conference reminding information to each conference member, the conference time is recorded, the connection with each conference member is disconnected at regular time, and the conference is ended.

The conference capacity management file is used for managing the number of conferences, the number of conference members, conference bandwidth, PSTN (public switched telephone network) paths and the like. The conference capacity management file is generated by the conference management server 41 according to the conference capacity configuration information indicated by the conference service access request, the number of concurrencies of the merged conference can be controlled by calling the conference capacity management file, and when the limited number of concurrencies is reached, the creation of a new conference is stopped. Specifically, after receiving the conference convergence request, the

110 calls the conference capacity management file, assuming that the conference service access request indicates that the maximum concurrent number of conferences is 300 (that is, only 300 conferences exist at the same time), and if 300 conferences exist at present, the conference to be created by the conference convergence request exceeds the maximum concurrent number, and an error is reported, and generation of conference link and telephone access information is stopped, so as to control the concurrent number of the conferences. Conversely, the maximum number of concurrencies is not exceeded, the generation of conference links and telephone access information for access by conference members may continue.

Similarly, when a conference member accesses, the

110 may also call the conference capacity management file to control the number of conference members accessing the same converged conference, and when the number of conference members exceeds the maximum number of conference members indicated by the conference service access request, the access of the conference members is prohibited, and in addition, the

110 may also call the conference capacity management file to control the bandwidth occupied by the converged conference.

in step 701, the deployed

43 receives media streams sent by conference members belonging to the social network application and conference members in the telephone network;

In step 702, the deployed audio/video

42 performs stream selection on the media stream to obtain stream selection data; the selected stream data is a single media stream or a multi-path combined media stream;

specifically, if the number of simultaneous speaking or video people in the conference exceeds the set number of people, the audio/video

42 deployed in the

110 may perform stream selection on multiple media streams sent by any conference member according to the configured stream selection rule to obtain stream selection data. The streaming data may be one of multiple media streams sent by any conference member or a combination of multiple media streams.

The simplest stream selection method is to select the 6 media streams received first to join in the audio mixing, or to select a route according to the volume of the speaker at different times, for example, to select the 6 media streams with the largest volume to join in the audio mixing, or of course, to send a route selection method for applying for speaking right to the

110, to determine the speaking right of different conference members by the conference host, to send the speaking right of a certain conference member to the

110 by the conference host, and to cancel the forbidden of the conference member by the conference management server 41 of the

110, to receive the media stream sent by the conference member.

Optionally, after the stream selection operation obtains the stream selection data, the audio/video

42 may further perform quality recovery processing such as jitter elimination and packet loss recovery on the stream selection data. The specific process of jitter elimination and packet loss recovery for the media stream may be implemented with reference to the prior art, and is not described herein again.

For each path of media stream in the selected stream data, a receiver conference member of the path of media stream is determined according to a sender conference member of the path of media stream, and since the

110 needs to forward the media stream to other conference members, the receiver conference member is other conference members except the sender conference member.

Taking an example that five conference members in a conference, ABCDE, carry out a voice conference, wherein four members in ABCD speak simultaneously, a forwards a media stream uploaded to the

110 to BCDE, B forwards a media stream sent to the

110 to ACDE, C forwards a media stream sent to the

110 to ABDE, and D forwards a media stream sent to the

110 to ABCE. Therefore, A can hear the speech of three BCD people, B can hear the speech of three ACD people, C can hear the speech of three ABD people, D can hear the speech of three ABC people, and E can hear the speech of four ABCD people.

In

705, the deployed

43 forwards each media stream to the receiving conference member corresponding to the media stream.

The exemplary embodiments of the present disclosure are illustrated with the AB speech coding format being silk and the CDE speech coding format being g.711u. As shown in fig. 8, which is a schematic diagram of a principle of implementing a convergence conference by the

110, in the diagram, the terminal on the left side of the audio/video

42, which is ABCD, sends an upstream media stream to the audio/video

42, and the terminal on the right side of the audio/video

42, which is processed by the audio/video

42, sends a downstream media stream to the ABCDE. Among them, in _ a (silk format), in _ b (silk format), in _ c (g.711u), in _ d

(g.711u) sequentially represents the upstream media stream of A, B, C, D, in _ b + in _ c + in _ d represents the media code stream after the upstream media stream of the BCD is fused, and in _ b + in _ c + in _ d is converted into a coding format (silk format) supported by a after being processed by the audio/video

42 and is sent to the terminal user a, namely the terminal user a can observe the speech of other speakers BCD except the terminal user a. By analogy with other end users, as shown in fig. 8, each media stream is forwarded to other conference members than the conference member of the media stream sender.

As shown in fig. 9, for an application scenario of the method for implementing a converged conference according to the present disclosure, a converged conference system, that is, the

110 in the foregoing exemplary embodiment, may connect to the social network application through the internet to establish a data transmission channel with the social network application, and may further connect to the voice switch through the internet to connect to the IP phone through the voice switch, where the voice switch is mainly responsible for distributing voice streams of each IP extension, similar to the distribution of IP packets by the switch in the lan. Conference members in the internet can access the converged conference system through social network applications or IP phones. The voice switch may also be connected to a voice gateway, where the voice gateway is used for performing data security check and data routing when interacting with a PSTN telephone network, and is a gateway in the internet, and accesses the PSTN network through the voice gateway to establish a data transmission channel with a PSTN telephone terminal 130 (a fixed telephone and a mobile phone). Conference members in the telephone network can access the converged conference system through the

130, so that the conference members in the internet can participate in the same conference with the conference members in the telephone network through the VOIP telephone applied by the social network, and barrier-free intercommunication between the VOIP telephone applied by the social network and the PSTN telephone is realized.

Fig. 10 is a flowchart of a method for implementing a converged conference, which is provided in another exemplary embodiment of the present disclosure, and is applicable to a scope and an execution subject of the method, for example, the method is used for the

120 in the implementation environment shown in fig. 1. As shown in fig. 10, the method for implementing a converged conference, which is executed by the

120, may include the following steps.

the social network application is the social network application in the above exemplary embodiment, and the social network application may be instant messaging software APP. It should be noted that, before the social network application initiates the converged conference creation request, a conference service is requested to be created from the

110, and the

110 allocates a conference access interface for the access of the social network application. Then, the

120 generates a meeting creation request and invokes a meeting access interface in the social network application according to a meeting creation operation triggered by the user (for example, click to open a meeting and add a meeting member).

As described in the foregoing exemplary embodiment, the personalized configuration refers to configuration management of an IVR (Interactive Voice Response) Voice prompt for each scene in a conference, configuration management of a conference function (including conference recording, conference reminding, conference time limit, encoding format, conference encryption, and the like), configuration management of a conference capacity (including conference number management, member number management, conference bandwidth management, PSTN route number management, and the like), and the like. The social network application of the

120 requests the

110 to create a personalized conference service according to the conference service access triggered by the user and the personalized configuration performed on the conference service, and the

110 responds to the request, allocates a conference access interface for the conference service access of the social network application and generates a personalized configuration file for managing the conference service.

Specifically, the

120 establishes a connection of the social network application with the

110 through the conference access interface.

In

1050, transmitting the converged conference creation request over a connection to the conference access interface;

the social network application of the

120 calls the conference access interface to send the generated converged conference creation request to the

110 via the conference access interface. Thus, the

110 can obtain the network types of the conference members as the internet and the telephone network, respectively, according to the conference member specified by the converged conference creation request. And generating a conference link for the conference member of which the network type is the internet, issuing the conference link to the corresponding social network application, and generating telephone access information for the conference member of which the network type is the telephone network.

specifically, the social network application of the

120 receives a conference connection issued by the

110 according to the fusion conference creation request, and then conference members in the internet can obtain a conference link for joining the conference through the social network application used by the conference members.

Specifically, the conference member triggers a conference link in the social network application used by the conference member, and the

120 controls the social network application to access the converged conference according to the address provided by the conference link, even if the conference member using the social network application joins the converged conference. Meanwhile, the phone members in the phone network may access the converged conference via the phone access information generated by the

110, which may be a phone call request or a conference access code with reference to the above-described exemplary embodiment. When the conference member in the telephone network sends an access request to the

110 by responding to the telephone call request sent by the

110 or by the conference access code, and the access request is responded, the conference member in the telephone network accesses the converged conference. When the conference member to which the social network application belongs and the conference member in the telephone network are accessed, the fusion conference with the conference member in the telephone network can be carried out through the social network application.

The following is an embodiment of the apparatus of the present disclosure, which may be used to execute an embodiment of a converged conference implementation method executed by the

110 of the present disclosure. For details not disclosed in the embodiments of the device of the present disclosure, please refer to the embodiments of the method for implementing a fusion conference of the present disclosure.

Fig. 11 is a block diagram illustrating a converged conference implementation apparatus, which may be used in the

110 of the implementation environment shown in fig. 1, to perform all or part of the steps of the converged conference implementation method shown in any one of fig. 3, 5-7, according to an exemplary embodiment. As shown in fig. 11, the apparatus includes, but is not limited to: a

1110, a

1130, an

1150, and a

1170.

A

1110, configured to be called by a social network application of a conference service created through a conference access interface, and receive a converged conference creation request initiated by the social network application and transmitted through the conference access interface;

a

1130, configured to obtain, according to the conference member specified by the converged conference creation request, that the network types of the conference members are an internet network and a telephone network, respectively;

An

1150, configured to generate a conference link for a conference member whose network type is an internet and send the conference link to a corresponding social network application, and generate telephone access information for the conference member whose network type is a telephone network;

and a

1170, configured to perform access to conference members in the internet and conference members in the telephone network through the telephone access information respectively by triggering the conference link in the social network application, and implement the converged conference under the access of the conference members belonging to the social network application running in the internet and the conference members in the telephone network.

The

1110 may be, for example, the wired or

250 in fig. 2.

The

1130, the

1150, and the

1170 may also be functional modules, and are configured to execute corresponding steps in the implementation method of the convergence conference. It is understood that these modules may be implemented in hardware, software, or a combination of both. When implemented in hardware, these modules may be implemented as one or more hardware modules, such as one or more application specific integrated circuits. When implemented in software, the modules may be implemented as one or more computer programs executing on one or more processors, such as programs stored in

232 for execution by

222 of FIG. 2.

Optionally, as shown in fig. 12, the

1170 includes but is not limited to:

the

1201 is used for the deployed access server to issue the conference link to the social network application where the conference member is located through the internet;

a

1202, configured to, when a conference member in an internet initiates a connection between the conference member and the access server through triggering of the conference link in the social network application, enable the access server to respond to the connection initiated by the conference member in the internet, and establish a connection with the social network application in which the conference member in the internet is located.

Optionally, the phone access information is a phone call request or a conference access code, as shown in fig. 13, the

1170 includes but is not limited to:

an

1301, configured to initiate a signaling access process by initiating the phone call request to a conference member in the phone network or issuing the conference access code, where the signaling access process is responded when the conference member in the phone network responds to the phone call request or initiates an access request through the conference access code and the access request is responded;

A

1302, configured to, when the signaling access process is answered, establish a telephone channel between the access server and a conference member in the telephone network, where the conference member in the telephone network is accessed via the telephone channel.

Fig. 14 is a block diagram illustrating a converged conference implementation apparatus, which may be used in the

120 installed with a social network application of the implementation environment shown in fig. 1, to execute the steps of the converged conference implementation method shown in fig. 10 according to an exemplary embodiment. As shown in fig. 14, the apparatus includes, but is not limited to: a

1410, a

1430, a

1450, a

1470, and a

1490.

The

1410 is configured to generate a meeting creation request and call a meeting access interface according to a meeting creation operation triggered in the social network application that has created the meeting service;

A

1430, configured to establish a connection of the social network application through the called conference access interface;

a

1450, configured to transmit the converged conference creation request through a connection with the conference access interface;

a

1470, configured to receive a conference link generated and issued according to the conference creation request, where a conference member specified by the conference creation operation obtains the conference link through a social network application where the conference member is located;

and the

1490 is used for controlling the social network application to perform the converged conference access of the conference members according to the trigger of the conference link, and the conference members in the telephone network perform the converged conference access through the telephone access information corresponding to the conference link so as to perform the converged conference with the conference members in the telephone network through the social network application.

Optionally, the present disclosure further provides an electronic device, which may be used in the

110 in the implementation environment shown in fig. 1 to execute all or part of the steps of the fusion conference implementation method shown in any one of fig. 3 and fig. 5 to 7. The electronic device may also be used in the

120 in the implementation environment shown in fig. 1 to execute the steps of the converged conference implementation method shown in fig. 8. The electronic device includes: