FI130232B - Method for distributed information processing and distributed information processing system - Google Patents

METHOD FOR DISTRIBUTED INFORMATION PROCESSING AND DISTRIBUTED

INFORMATION PROCESSING SYSTEM

FIELD OF THE INVENTION

The present invention relates to a method for distributed information processing using a dataflow graph and more particularly to a method according to the preamble of claim 1. The present invention further relates to a distributed information processing system using a dataflow graph and more particularly to a distributed information processing system according to the preamble of claim 7.

BACKGROUND OF THE INVENTION

In the prior art, distributed information processing using a dataflow graph is designed and executed in three major ways.

A first prior art method for distributed information processing using a dataflow graph relates to the scheduler /executor pattern, the scheduler/executor pattern being a system in which a computational node called the scheduler sends units of work, called jobs, to a node called an executor that executes said jobs.

The results of the jobs are then sent to a computational node that aggregates and further processes these results for use in a variety of industrial settings.

Examples of this pattern comprise: - Apache Spark schedules jobs described using MLib, and Apache n Mesos acts as a broker between Spark and one or more devices that are

S executors. se - Distributed TensorFlow schedules jobs on a master computational = unit and executors run the jobs, reporting back the results for use in aggregative

I algorithms like loss functions and back-propagation. - - Object oriented programming is used in order to enforce separation <Q of tasks between distributed nodes called actors and learners. This technology, 3 codenamed IMPALA, is experimental and is being developed by DeepMind.

N A second prior art method for distributed information processing using a dataflow graph relates to the siloed execution pattern, the siloed execution pattern being an arbitrary number of instances of one dataflow graph, for example a face-recognition algorithm, being distributed over an arbitrary number of nodes, for example, mobile phones, that have minimal communication with a central server. Examples of this pattern comprise: - Technologies like TensorFlow Lite and Amazon Greengrass that do edge computing and inference. Both of these solutions execute a small subset of an algorithm, such as predictions using a machine learning model or data preprocessing, on a mobile device that communicates sparingly with a central server to download new models or to report new data. - Federated learning uses machine learning algorithms on devices that are producers of their own data in order to execute a modified gradient descent algorithm that allows for more infrequent communication with a central server. - Certain research problems in theoretical physics, math and astronomy that require massive computational resources write portions of the algorithm in a way that can be executed in parallel with a certain level of planned or stochastic redundancy.

A third prior art method for distributed information processing using a dataflow graph relates to the parallel hardware pattern, the parallel hardware pattern being the use of specialized hardware with parallel lanes for operations such as floating point addition being matched to parts of a dataflow graph that use those operations. Examples of this pattern comprise: e - Chips like the Tensor Processing Unit (TPU) and Kirin 970 have built-

S in machine learning primitives such as matrix multiplication, convolution and 8 activation functions. - The Graphical Processing Unit (GPU) has built-in matrix z multiplication and parallel vertex manipulation, running parallel operations on so pixels of an image or vectorized representations of geometric forms. These are a also used by MLib, TensorFlow, Theano, Torch, Caffe and CNTK to execute parts = of Machine Learning algorithms.

N - Modern Central Processing Units (CPUs), such as Arm NEON , that have special chips for binary operations on matrices and vectors, FFTs and FIR filters.

While all of these patterns are useful, they pose disadvantages for certain categories of information processing problems.

The use of the prior art methods can result in the suboptimal use of computational infrastructure. For example, distributing an MLib data flow graph via Apache Mesos and Apache Spark relies on an algorithm that uses approximations of available computational resources to make estimations about the real-time availability of computational units in a datacenter. While this allows for a certain degree of optimization, it does not allow for the distribution of dataflow graphs to the most suitable devices that can execute certain operations.

Distributed parallelization and distribution algorithm of Tensorflow is similar to

Spark, but it improves upon this using the XLA Compiler, which fuses together certain sub-graphs, called “kernels”, of a dataflow graph and deploys them to nodes in a datacenter. However, the XLA Compiler uses a just-in-time compilation strategy has mitigated results in improving the speed and accuracy of the distributed information processing.

The prior art methods also may result in excessive communication between devices, which leads to infrastructure degradation. For example, a machine learning algorithm doing processing on data generated by a group of mobile devices capturing real-time video of a football match would need to, using current machine learning solutions, either use siloed inference that does not take

AM into account the feed of other devices, i.e. TensorFlow Lite, or transfer streamed

S video to a central server that aggregates all of the data into a model, draining 8 battery of the devices and, if done repeatedly, lowering the longevity of the mobile devices due to intensive network operations. No current solution can find z an optimal way for mobile devices to work together in the execution of a machine so learning algorithm, which would allow for more robust algorithms while lowering a the dependency on mobile network traffic, thus preserving the battery longer and = diminishing device degradation.

N Prior art methods are also disclosed in patent application documents

U 2010306753 Al and US 2014019949 Al. Teachings in these documents represent general state of the art relating to present invention.

Furthermore, the use of prior art methods can also lead to slow, inaccurate or incomplete predictive algorithms. The training of certain large machine learning algorithms, such as deep recurrent neural networks or convolutions on large images with small kernels running on processing units like

TPUs and GPUs in a datacenter, requires one to sacrifice either time, accuracy or model features. This is because the datacenter is physically limited in the number of computational units it contains.

BRIEF DESCRIPTION OF THE INVENTION

An object of the present invention is to provide a method and system for distributed information processing such that the prior art disadvantages are solved or at least alleviated.

The objects of the invention are achieved by a method for distributed information processing which is characterized by what is stated in claim 1. The objects of the present invention are further achieved by a distributed information processing system which is characterized by what is stated in claim 7.

The preferred embodiments of the invention are disclosed in the dependent claims.

The invention is based on the idea of providing a method for n distributed information processing using a dataflow graph comprising graph

S nodes and graph edges between the graph nodes. The method comprises

O distributing a dataflow graph to an infrastructure comprising infrastructure 7? nodes and infrastructure edges between the infrastructure nodes. The method = comprises segmenting the dataflow graph into dataflow sub-graphs, defining a

E pre-determined infrastructure having two or more infrastructure nodes and at <Q least one infrastructure edge between the infrastructure nodes, projecting the 3 dataflow sub-graphs to the pre-determined infrastructure by distributing the

N dataflow sub-graphs to the infrastructure nodes and infrastructure edges, and executing the dataflow graph by utilizing a communication protocol over the infrastructure edges between the infrastructure nodes in the execution of the dataflow graph and executing the distributed dataflow sub-graphs in the infrastructure nodes. The method allows splitting the dataflow graph into dataflow sub-graphs and distributing the sub-graphs to infrastructure which is defined before distributing the sub-graphs. This allows forming the sub-graphs and distributing the sub-graphs such that the infrastructure properties may be taken into account for efficient processing and efficient use of the infrastructure.

Segmenting the dataflow graph may comprise dividing the dataflow graph into the dataflow sub-graphs and grouping the sub-graphs into grouped dataflow sub-graphs based on the pre-determined infrastructure. Thus, the sub- graphs may be grouped taking into account the properties of the infrastructure.

The grouped dataflow sub-graphs may then be distributed to the pre-determined infrastructure.

The graph nodes of the dataflow graph comprise tensor operations.

Thus, the segmenting of the dataflow graph may comprise scalarization of the tensor operations, segmenting the graph nodes into scalar operations or cohorts of scalar operations, and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure. The scalar operation groups form the dataflow sub-graphs. This allows forming the dataflow sub-graphs based on the scalar operations and thus the forming of sub-graphs may take into account the scalar operations of the 0 dataflow graph and also the infrastructure.

S Alternatively, the segmenting of the dataflow graph may comprise 8 representing the tensor operations as scalar promises representing scalar 5 operations to be performed, receiving input data, fulfilling the scalar promises z based on the input data such that fulfilled scalar promises are formed from the so scalar promises, segmenting the fulfilled scalar promises into scalar operations or a cohorts of scalar operations, and grouping the scalar operations or cohorts of = scalar operations into scalar operation groups based on the pre-determined

N infrastructure, the scalar operation groups forming the dataflow sub-graphs.

The segmenting may further comprise repeating the following steps one or more times for fulfilling the scalar promises before the grouping of the scalar operations or cohorts of scalar operations: receiving input data, fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises, and segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations. This allows introducing information progressively into the dataflow graph and thus less time is required for the final step of graph segmentation due to the intermediary steps.

The segmenting of the dataflow graph may also comprise merging redundant equivalent dataflow sub-graphs such that merged dataflow sub-graphs are formed. This means that two or more equivalent sub-graphs or scalar operations or cohorts of scalar operations are merged into only one sub-graph or scalar operation or cohort of scalar operations which will be executed or performed only once, instead of executing or performing each separately.

Projecting the dataflow sub-graphs to the infrastructure may comprise compiling the dataflow sub-graphs into a segmented dataflow graph comprising the dataflow sub-graphs. Alternatively, the projecting of the dataflow sub-graphs to the infrastructure may comprise compressing the dataflow sub-graphs into compressed dataflow sub-graphs and compiling the compressed dataflow sub- graphs into a segmented dataflow graph comprising the dataflow sub-graphs.

Further alternatively, projecting the dataflow sub-graphs to the infrastructure may comprise compiling the dataflow sub-graphs into a segmented dataflow 0 graph comprising the dataflow sub-graphs and compressing the segmented

S dataflow graph into compressed segmented dataflow sub-graphs. This allows 8 forming the segmented dataflow graph which comprises the dataflow sub-graphs.

The segmented dataflow sub-graphs or the dataflow sub-graphs may be z compressed using any known compression algorithm for grouping similar scalar so operations. Thus, the projection and execution of the dataflow graphs may be 2 more efficient: 5 Defining the pre-determined infrastructure may comprise generating

N one or more sub-graph projection predictions based on the infrastructure, the dataflow sub-graph projection predictions representing the predicted optimal distribution of the dataflow sub-graphs to the infrastructure, and by defining the pre-determined infrastructure based on pre-defined infrastructure criteria, projecting the dataflow sub-graphs to the pre-determined infrastructure by projecting the one or more projection predictions to the infrastructure, executing the dataflow graph by executing the distributed dataflow sub-graphs according to one of the projection predictions.

Accordingly in the present invention, first is generated one or more predictions of how to distribute the sub-graphs to the infrastructure, then the used infrastructure is defined as the pre-determined infrastructure, and further the one or more projection predictions are projected to the infrastructure. One of the projection predictions is then decided to be executed in the infrastructure.

The one projection prediction to be executed may be decided based on the pre- determined infrastructure or properties of the pre-determined infrastructure or the infrastructure nodes and infrastructure edges of the pre-determined infrastructure.

Defining the pre-determined infrastructure may comprise generating the one or more projection predictions by creating a constraint optimization problem and utilizing a constraint optimization algorithm with pre-determined constraints, defining the pre-determined infrastructure from the one or more projection predictions by solving the constraint optimization problem, and projecting the dataflow sub-graphs to the hardware nodes and the network e connections of the pre-determined infrastructure.

S When the constraint optimization algorithm takes a prolonged time, 8 the determination of the pre-determined infrastructure may be accomplished by using the output of the constraint optimization algorithm as the target of the z neural network with the infrastructure description as the training input, training so the neural network and then using it to make projection predictions from a infrastructure descriptions as the prediction input. = In one embodiment, the step of defining the pre-determined

N infrastructure may further comprise analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the analysing of the infrastructure, generating one or more projection predictions by utilizing the constraint optimization programs with constraints obtained from the analysing of the infrastructure, and defining the pre- determined infrastructure as the infrastructure used to generate the one or more constraint optimization programs. In this embodiment, the available infrastructure is known such that the available infrastructure is used as the pre- determined infrastructure.

In another embodiment, the step of defining the pre-determined infrastructure may further comprise analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the analysing of the infrastructure, generating one or more projection predictions by utilizing the constraint optimization programs with constraints obtained from the analysing of the infrastructure, and defining the pre- determined infrastructure based on the pre-defined infrastructure criteria. In this embodiment, properties of the available infrastructure are known, and defining the pre-determined infrastructure comprises defining the infrastructure nodes and edges of the available infrastructure which are used, based on the pre- defined infrastructure criteria.

The pre-defined infrastructure criteria may comprise properties of the infrastructure nodes and edges or the like.

In a further embodiment, the step of defining the pre-determined e infrastructure may further comprise analysing a potential infrastructure by

S generating synthesized representations of the infrastructure and representing the 8 infrastructure as one or more constraint optimization programs based on the o synthesized representations, generating one or more projection predictions z based on the synthesized representations, and defining the pre-determined so infrastructure based on the pre-determined criteria. In this embodiment, the a available infrastructure is not known, and therefore synthesized representations = of the infrastructure are generated based on the anticipations of bandwidth

N between the nodes, the computational power of the nodes, or the like. Thus, the pre-determined infrastructure is defined based on the synthesized representations of the potential infrastructure or anticipations of the potential infrastructure.

In a yet alternative embodiment, the step of defining the pre- determined infrastructure may further comprise analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions by utilizing a neural network trained with the results of the one or more constraint optimization programs as expected output values and the synthesized infrastructure representations as input values and making projection predictions using the same or different synthesized infrastructure representations as input, and defining the pre-determined infrastructure based on the pre-determined criteria. In this embodiment, the neural network is utilized for generating the projection predictions when the constraint optimization problem execution is a slow process. When the neural network is used to make predictions, the infrastructure may be first described. A neural network is then further used to predict the distribution of the sub-graphs. The description of the infrastructure is received by the neural network as input and the output from the neural network is the projection prediction, or a prediction of the optimal distribution within a pre-determined threshold of accuracy.

Accordingly, the available infrastructure may be first analysed and the e pre-determined infrastructure may be defined based on the pre-determined

S criteria such as computational power of available infrastructure nodes and/or 8 available bandwidth of the infrastructure edges.

Projecting the dataflow sub-graphs to the infrastructure may comprise z projecting the dataflow sub-graphs or the segmented dataflow graph to the pre- so determined infrastructure, or projecting the dataflow sub-graphs or the a segmented dataflow graph to the pre-determined infrastructure such that each = dataflow sub-graph is projected to a separate infrastructure node. Thus, the

N dataflow sub-graphs may be distributed to most suitable infrastructure nodes and/or infrastructure edges.

The execution of the dataflow graph may comprise monitoring the infrastructure during the execution of the dataflow graph. Alternatively, the execution of the dataflow graph may comprise monitoring the infrastructure during the execution of the dataflow graph and receiving a monitoring output, and comparing the monitoring output and a pre-determined expected monitoring output. If, based on the comparison, the monitoring output is within the pre- determined expected monitoring output, the execution of the dataflow graph continues utilizing the pre-determined infrastructure. If based on the comparison, the monitoring output is outside the pre-determined expected monitoring output, the projection of the dataflow sub-graphs to the infrastructure is re-initiated.

In one embodiment, the execution of the dataflow graph may comprise monitoring the pre-determined infrastructure during the execution of the dataflow graph by spamming in the pre-determined infrastructure and receiving a spamming output. Alternatively, the execution of the dataflow graph may comprise monitoring the pre-determined infrastructure nodes and the infrastructure edges of the pre-determined infrastructure during the execution of the dataflow graph and receiving a monitoring output, and comparing the monitoring output and a pre-determined expected monitoring output, and based on the comparison. If, based on the comparison, the monitoring output is within the pre-determined expected monitoring output, continuing the execution of the e dataflow graph utilizing the infrastructure nodes and the infrastructure edges of

S the pre-determined infrastructure. If, based on the comparison, the monitoring 8 output is outside the pre-determined expected monitoring output, re-initiating the projection of the dataflow sub-graphs to the infrastructure nodes and the z infrastructure edges. so In another embodiment, the execution of the dataflow graph may a comprise monitoring the pre-determined infrastructure during the execution of = the dataflow graph by spamming in the pre-determined infrastructure and

N receiving a spamming output, and comparing the spamming output and a pre- determined expected output. If, based on the comparison, the spamming output is within the pre-determined expected spamming output, the execution of the dataflow graph is continued. If, based on the comparison, the spamming output is outside the pre-determined expected spamming output, the projection of the dataflow sub-graphs to the infrastructure nodes and the infrastructure edges is re-initiated.

Monitoring the infrastructure enables dynamic adjustment of the distributed processing as the used infrastructure may be changed if properties of the pre-determined infrastructure change during the execution of the dataflow graph.

The execution step of the method according to the present invention may further comprise progressively updating the dataflow sub-graphs during the execution of the dataflow graph. The updating may be carried out by utilizing the communication protocol over the infrastructure edges of the pre-determined infrastructure such that data may flow between the infrastructure nodes.

Progressively updating the dataflow sub-graphs during the execution of the dataflow graph may comprise exchanging data between the sub-graphs executed in the infrastructure nodes of the pre-determined infrastructure and modifying the sub-graphs based on the data exchange. The data exchange may be carried out by utilizing the communication protocol over the infrastructure edges of the pre- determined infrastructure. Accordingly, during the execution of the dataflow graph, data flows through and between the sub-graphs, and the sub-graphs may 0 be updated based on the data flowing through and between the sub-graphs. Thus,

S the sub-graphs may modify during the execution of the dataflow graph based on 8 the data. This makes the method evolve based on the data processed utilizing the dataflow graph. Accordingly, the communication protocol may be configured to z progressively update the dataflow sub-graphs during the execution of the so dataflow graph, as disclosed above. a The present invention also relates to a distributed information = processing system using a dataflow graph comprising graph nodes and graph

N edges between the graph nodes. The system comprises a distributed infrastructure comprising hardware nodes and network connections between the hardware nodes. The system is configured to distribute a dataflow graph to the infrastructure.

The system comprises a segmentation module configured to segment the dataflow graph into dataflow sub-graphs, a projection module configured to define a pre-determined infrastructure having two or more hardware nodes and at least one network connection between the hardware nodes, and to project the dataflow sub-graphs to the pre-determined infrastructure by distributing the dataflow sub-graphs to the hardware nodes and network connections for execution of the dataflow graph, and a communication protocol provided to the hardware nodes, the communication protocol being configured to execute the dataflow graph over the network connections by executing the distributed dataflow sub-graphs in the hardware nodes. The system of the invention is configured to split the dataflow graph into dataflow sub-graphs and distribute the sub-graphs to the hardware nodes and network connections determined before the distribution of the sub-graphs. Then the sub-graphs are executed by the pre- determined infrastructure in a distributed manner.

The system may comprise a segmentation server comprising the segmentation module, or a segmentation server comprising a segmentation memory configured to store the segmentation module and a segmentation processor configured to execute instructions of the segmentation module.

The system may also comprise a segmentation server comprising the e segmentation module and the projection module, a segmentation server

S comprising a segmentation memory configured to store the segmentation module 8 and the projection module, and a segmentation processor configured to execute instructions of the segmentation module and the projection module. z The segmentation server may be configured to segment, by utilizing so instructions of the segmentation module executed by the segmentation processor, a the dataflow graph by dividing the dataflow graph into the dataflow sub-graphs, = and grouping the dataflow sub-graphs into grouped dataflow sub-graphs based

N on the pre-determined infrastructure.

The segmentation server may be configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by scalarization of tensor operations, the graph nodes of the dataflow graph comprising the tensor operations, segmenting the graph nodes into scalar operations or cohorts of scalar operations, and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure, the scalar operation groups forming the dataflow sub-graphs.

Alternatively, the segmentation server may be configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by scalarization of tensor operations of the graph nodes into scalar operations, representing tensor operations as scalar promises representing scalar operations to be performed, the graph nodes of the dataflow graph comprising the tensor operations, receiving input data to the segmentation module, fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises, segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations, grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure, scalar operation groups forming the dataflow sub-graphs.

This enables forming the dataflow sub-graphs based on the pre- determined infrastructure and generating an efficient distribution and execution e of the dataflow sub-graphs.

S Furthermore, the segmentation server may be configured to segment, 8 by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by merging redundant eguivalent z dataflow sub-graphs such that merged dataflow sub-graphs are formed. This so enables decreasing the number operations to be performed as two or more a equivalent sub-graphs or scalar operations or cohorts of scalar operations may be 00 . . . 5 merged into only one sub-graph or scalar operation or cohort of scalar operations

N which will be executed or performed only once, instead of executing or performing each separately.

The segmentation server may be further configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by defining the pre- determined infrastructure by generating one or more sub-graph projection predictions based on the infrastructure, the dataflow sub-graph projections representing the distribution of the dataflow sub-graphs to the infrastructure, and by defining the hardware nodes and network connections of the pre- determined infrastructure based on pre-defined infrastructure criteria, and projecting the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure by projecting the one or more projection predictions to the hardware nodes and network connections of the pre-determined infrastructure. Further, the communication protocol may be configured to execute the dataflow graph over the network connections of the pre-determined infrastructure by executing the distributed dataflow sub-graphs according to one of the projection predictions in the hardware nodes of the pre- determined infrastructure.

The projection module may be configured to or the segmentation server may configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the e analysing of the infrastructure, generating one or more projection predictions by

S utilizing the constraint optimization programs with constraints obtained from the 8 analysing of the infrastructure, and defining the pre-determined infrastructure as the infrastructure used to generate the one or more constraint optimization = programs. a

O

2 Alternatively, the projection module may be configured to or the = segmentation server may configured to project, by utilizing instructions of the

N projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the analysing of the infrastructure, generating one or more projection predictions by utilizing the constraint optimization programs with constraints obtained from the analysing of the infrastructure, and defining the pre-determined infrastructure based on the pre-defined infrastructure criteria.

Accordingly, the pre-determined infrastructure may be formed based on the available hardware nodes and network connections.

Further alternatively, the projection module may be configured to or the segmentation server may configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions based on the synthesized representations, and defining the pre-determined infrastructure based on the pre-determined criteria.

Still further alternatively, the projection module may be configured to or the segmentation server may configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the e infrastructure as one or more constraint optimization programs based on the

S synthesized representations, training a neural using the results of the one or 8 more constraint optimization programs as targets and the synthesized representations of the infrastructure as inputs, and generating one or more z projection predictions by utilizing the trained neural network to generated one so or more projection projections as output from the input of one or more a synthesized infrastructure representations, and defining the pre-determined = infrastructure based on the pre-determined criteria. The projection module may

N be configured to or the segmentation server may configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure byanalysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions by utilizing a neural network trained with the results of the one or more constraint optimization programs as expected output values and the synthesised infrastructure representations as input values and making projection predictions using the same or different synthesised infrastructure representations as input, and defining the pre-determined infrastructure based on the pre-determined criteria.

The projection module may be configured to or the segmentation server may be configured to project, by utilizing instructions of the projection module and executed by the segmentation processor, the dataflow sub-graphs to the infrastructure by distributing the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure for execution of the dataflow graph, or by distributing the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure such that each dataflow sub-graph is projected to a separate hardware node.

The system of the present invention may further comprise a monitoring module configured to or the segmentation server may comprise a monitoring module configured to monitor, by utilizing instructions of the e monitoring module and executed by the segmentation processor, the

S infrastructure by monitoring the hardware nodes and the network connections of 8 the pre-determined infrastructure during the execution of the dataflow graph.

Alternatively, the system of the present invention may further z comprise a monitoring module configured to or the segmentation server may so comprise a monitoring module configured to monitor, by utilizing instructions of a the monitoring module and executed by the segmentation processor, the = infrastructure by monitoring the hardware nodes and the network connections of

N the pre-determined infrastructure during the execution of the dataflow graph and receiving a monitoring output, comparing the monitoring output and a pre-

determined expected output. If, based on the comparison, the monitoring output is within the pre-determined expected output, continuing the execution of the dataflow graph utilizing the hardware nodes and the network connections of the pre-determined infrastructure. If, based on the comparison, the monitoring output is outside the pre-determined expected output, re-initiating the projection of the dataflow sub-graphs to the hardware nodes and the network connections.

Further, the system of the present invention may further comprise a monitoring module configured to or the segmentation server may comprise a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph by spamming in the hardware nodes and the network connections and receiving a spamming output.

Further alternatively, the system of the present invention may further comprise a monitoring module configured to or the segmentation server may comprise a monitoring module configured to monitor, by utilizing instructions of the monitoring module and executed by the segmentation processor, the infrastructure by monitoring the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph by spamming in the hardware nodes and the network connections of the pre- e determined infrastructure and receiving a spamming output, and comparing the

S spamming output and a pre-determined expected output, and based on the 8 comparison. If, based on the comparison, the spamming output is within the pre- determined expected output, continuing the execution of the dataflow graph z utilizing the hardware nodes and the network connections of the pre-determined so infrastructure. If, based on the comparison, the spamming output is outside the a pre-determined expected output, re-initiating the projection of the dataflow sub- = graphs to the hardware nodes and the network connections.

N The monitoring module enables changing the used infrastructure if properties of the pre-determined infrastructure change during the projection or execution of the dataflow graph.

An advantage of the invention is that it allows the dataflow graph to be distributed as sub-graphs to the infrastructure nodes and infrastructure edges based on the pre-determined infrastructure. Furthermore, the pre-determined infrastructure may be defined based on the properties of the available infrastructure nodes and infrastructure edges. Therefore, a dynamic dataflow graph distribution may be achieved such that sub-graphs may be distributed to the infrastructure nodes and edges without sending the whole dataflow graph to each infrastructure node saving both data transfer amount infrastructure edges and computational power of the infrastructure nodes. Furthermore, defining the pre-determined infrastructure before distributing the dataflow sub-graphs enables choosing the dataflow sub-graphs to be distributed to certain infrastructure nodes and edges based on the computational power and bandwidth of each infrastructure node and edge. Therefore, this enables efficient use of computation power and bandwidth and dataflow graph may be executed faster.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is described in detail by means of specific embodiments with reference to the enclosed drawings, in which

Figure 1 shows a schematic view of a dataflow graph; n Figure 2 shows schematically steps of a method according to the

S present invention; se Figure 3 shows a schematic view of a system according to the present - invention; oO r Figure 4 shows a schematic view of a segmentation server of a system a according to the present invention; <Q Figure 5 shows a schematic view of another segmentation server of a 3 system according to the present invention;

Figure 6 shows a schematic view of a hardware device forming an infrastructure node.

DETAILED DESCRIPTION OF THE INVENTION

Complex computations of information may be expressed as a dataflow through a directed graph, called a dataflow graph. The dataflow graph comprises graph nodes and graph edges between the graph nodes. The graph nodes represent the components of the computation or tensors of the dataflow graph.

The graph edges represent dataflows between the nodes of the graph. For example, e?* (data — exponentiation — squared) is a dataflow graph with two graph edges between three graph nodes: data, exponentiation, and squared, of which the last two are tensors and the first one is a matrix.

Tensors are geometric objects that describe linear relations between geometric vectors, scalars, and other tensors. Examples of such relations include the dot product, the cross product, linear maps, and other mathematical operations. Geometric vectors, often used in physics and engineering applications, and scalars themselves are also tensors. The dataflow graph in the context of the present application be a tensor graph comprising the graph nodes and graph edges.

Figure 1 shows a schematic view a dataflow graph 50 comprising five nodes 52 each of which may comprise operations such as tensor operations. The dataflow graph 50 further comprises graph edges 54 between the graph nodes n 52. The graph edges 54 represent the relationship of the graph nodes 52. One

S graph node may comprise one or more operations or tensor operations. 0 Figure 2 shows schematically a diagram of the method for distributed 7 information processing using the dataflow graph 50 according to the present = invention. The method comprises distributing the dataflow graph 50 to an a infrastructure comprising infrastructure nodes and infrastructure edges between <Q the infrastructure nodes. 3 One exemplary embodiment of an infrastructure 1 is shown schematically in figure 3. The infrastructure 1 comprises hardware devices formed as the infrastructure nodes 6 of the infrastructure and carrying out the computation of the graph nodes 52 or operations of the graph nodes 52. The infrastructure nodes 6 may be for example computers, mobile phones, servers or the like hardware devices. Therefore, the infrastructure nodes 6 may be processors or computing elements having computing capacity and power capable of executing the graph nodes 52 or operations of the graph nodes 52, such as tensor operations. It should be noted that one infrastructure node 6 may be one hardware device or one computing element or alternatively one hardware device or one computing element may provide two or more infrastructure nodes 6. The latter may be accomplished by virtually dividing the computing element, the hardware device or a processor to two or more infrastructure sub-nodes.

Furthermore, it should be noted that one hardware device may comprise also two or more computing elements such as processors. Therefore, in the context of this application infrastructure node 6 or hardware node means one real hardware device, computing element or processor, or one virtually divided sub-element of the hardware device, computing element or processor. It may also comprise several connected real hardware devices that are, through an abstraction layer, treated as one single node. A single hardware node may contain both virtually divided sub-elements and be part of an abstraction that aggregates it onto a single virtual node. Furthermore, the infrastructure may comprise one or more different kind of infrastructure nodes 6 or hardware devices. The present invention is not limited any type infrastructure node(s) or hardware device(s), but they may vary e in different to embodiments.

S Figure 6 shows schematically one infrastructure node or hardware 8 node 6. The infrastructure node 6 may comprise an infrastructure node memory 32 configured to store information and instructions, such as information of the z sub-graph which is projected to the infrastructure node 6 and instructions to so execute the sub-graph. The infrastructure node 6 may comprise an infrastructure a node processor 34 configured to execute instructions of the sub-graph by utilizing = the instructions stored in the infrastructure node memory 32.

N The infrastructure node 6 may also comprise node application programming interface (API) 30 for communication with other infrastructure nodes 6 via the infrastructure edges 8. The node API 30 may be known as a kind of API module.

The infrastructure node 6 may further comprise a communication protocol provided to a communication protocol module 36. The communication protocol may be any known communication protocol, such as a Transmission

Control Protocol (TCP) or a peer-to-peer protocol, configured to execute the dataflow graph over the network connections or infrastructure edges 8 of the infrastructure 1 by executing the distributed dataflow sub-graphs in the infrastructure nodes 6.

The infrastructure nodes 6 and connected to each other with the infrastructure edges 8. The infrastructure edges 8 comprise network connections between the infrastructure nodes 6 enabling data transfer between the infrastructure nodes 6. The infrastructure edges 8 or the network connections may comprise any known network connection such as mobile phone network, local area network, wifi, wired network, fiber-optic network or the like. It should be noted that that the present invention is not limited to any specific network.

Furthermore, the infrastructure may comprise one or more different kind of infrastructure nodes 6 or hardware devices. The present invention is not limited any type infrastructure edge(s) or network connection(s), but they may vary in different to embodiments.

The method comprises segmenting the dataflow graph 50 into e dataflow sub-graphs. Segmenting means a process of splitting the dataflow graph

S 50 into sub-graphs based on a segmentation protocol. Accordingly, the dataflow 8 graph 50 is divided into two or more sub-graphs in the segmentation step 100, as shown in figure 2. z One sub-graph may comprise one or more graph nodes 52 and one or so more graph edges 8. Alternatively, the one sub-graph may comprise one or more a operations or tensor operations, or one or more tensors, and one or more graph > edges 8. &

Preferably, the dataflow graph 50 is divided into smallest possible sub- graphs such that the graph nodes of the sub-graph comprise only one operation, tensor operation or tensor.

Furthermore, the step 100 of the present invention may comprise segmenting two or more separate dataflow graphs 50 into sub-graphs. Continuing from that, the sub-graphs formed from the two or more dataflow graphs are processed further in the method of the present invention similarly manner together as when only one dataflow graph 50 is segmented in step 100.

The method further comprises defining a pre-determined infrastructure 1 having two or more infrastructure nodes and at least one infrastructure edge between the infrastructure nodes. The step 200 of defining the pre-determined infrastructure 1 comprises defining the infrastructure nodes 6 and infrastructure edges 8 which will be used for processing or executing he dataflow graph 50 or the dataflow sub-graphs. Accordingly, the step 200 comprises choosing the pre-determined infrastructure 1, meaning pre- determined infrastructure nodes 6 and infrastructure edges 8, from available infrastructure, meaning all available infrastructure nodes 6 and infrastructure edges.

In some embodiments, there is a fixed available infrastructure comprising fixed number of infrastructure nodes 6 and infrastructure edges 8. In this case, the pre-determined infrastructure defined in step 200 may comprise e the fixed infrastructure as such or it may comprise only part of the fixed

S infrastructure or part of the infrastructure nodes 6 and infrastructure edges. 8 In some other embodiment, there may be unpredictable or only partially anticipatable available infrastructure in which the number of z infrastructure nodes and infrastructure edges may vary in time or according to so circumstances. In this case, pre-determined infrastructure 1 always comprises a a separately defined number of infrastructure nodes 6 and infrastructure edges 8. = The method of the present invention further comprises projecting the

N dataflow sub-graphs onto the pre-determined infrastructure by distributing the dataflow sub-graphs to the infrastructure nodes and infrastructure edges in step

300, as shown in figure 2. In this step 300, the formed dataflow sub-graphs, or the graph nodes and graph edges of the dataflow sub-graphs are distributed or transmitted to the pre-determined infrastructure 1, meaning the infrastructure nodes 6 and infrastructure edges 8 of the pre-determined infrastructure 1. If the anticipated projections of sub-graphs onto a pre-determined infrastructure does not attain an appropriate level of anticipated efficient use of computational and bandwidth resources, an acceptable timeframe of execution or a desirable level of accuracy, the process of nodel design and segmentation may be re-initiated with a new model so as to attain the desired targets before the actual projection to the infrastructure.

Distributing the sub-graphs to the pre-determined infrastructure 1 usually comprises assigning or transmitting the sub-graphs to the pre- determined infrastructure 1 such that each sub-graph node is assigned to one infrastructure node 6 and each sub-graph edge is assigned to one infrastructure edge 8. Accordingly, the pre-determined infrastructure 1 may correspond the sub-graphs.

The method further comprises executing the dataflow graph in step 400 by utilizing a communication protocol over the infrastructure edges of the pre-determined infrastructure 1 between the infrastructure nodes of the pre- determined infrastructure 1 in the execution of the dataflow graph and executing the distributed dataflow sub-graphs in the infrastructure nodes of the pre- e determined infrastructure 1.

S The steps 100, 200, and 300 are usually carried out in central 8 computing unit 2, which may be server, a cluster of servers connected to each other via a communication network or direct data transfer connections. In the z context of this application the central computing unit is defined as a segmentation so server 2. The segmentation server 2 may project the dataflow sub-graphs to the a infrastructure nodes 6 utilizing projection network connections 4, as shown in = figure 3. The projection network connections 4 may be a known kind of network

N connection, similarly as disclosed above in connection with the infrastructure edges 8. The present invention is not limited to particular type of projection network connections 4.

The segmentation server may comprise a segmentation memory or segmentation memories 10, as shown in figure 4. The segmentation memory or memories 10 may be internal components of the segmentation server 2 or they may be external memories connected to the segmentation server 2. The segmentation memory 10 may further be known kind of memory or physical memory component capable of storing information. The segmentation server 2 may further comprise a segmentation processor or processor(s) 12. The segmentation processor(s) 12 may be internal components of the segmentation server 2 or they may be external processors connected to the segmentation server 2. The segmentation processor 12 may further be known kind of processor or physical computing component capable of executing operations.

The segmentation server application programming interface (SAPI) 14 for communication with the infrastructure nodes 6 via the projection network connections 4. The segmentation SAPI 14 may be known kind of SAPI module such as Apache HTTP Server, Microsoft IIS, and Oracle iPlanet Web Server.

The segmentation server 2 may also comprise a segmentation module 20 configured to segment the dataflow graph 50 according to the step 100, as shown in figure 5. The segmentation memory 10 may be configured to store the segmentation module 20 and the segmentation processor 12 may be configured e to execute instructions of the segmentation module 20.

S The segmentation server 2 may also comprise a projection module 22 8 configured to project the sub-graphs to pre-determined infrastructure 1 according to the step 300, as shown in figure 5. The segmentation memory 10 z may be configured to store the projection module 22 and the segmentation so processor 12 may be configured to execute instructions of the segmentation 2 module 20. 5 The segmentation server 2 may also comprise a projection module 22

N configured to define the pre-determined infrastructure 1 and project the sub- graphs to the pre-determined infrastructure 1 according to the step 300, as shown in figure 5. The segmentation memory 10 may be configured to store the projection module 22 and the segmentation processor 12 may be configured to execute instructions of the segmentation module 20.

The segmenting 100 of the dataflow graph may further comprise dividing the dataflow graph into the dataflow sub-graphs and grouping the sub- graphs into grouped dataflow sub-graphs based on the pre-determined infrastructure 1. This may be accomplished for example by dividing the dataflow graph 50 to separate operations or cohorts of operations, such as tensor operations, and grouping the operations into grouped dataflow sub-graphs based on the pre-determined infrastructure 1. Accordingly, the sub-graphs are formed based on the defined pre-determined infrastructure 1. This allows the grouping of the sub-graphs or operations to be carried out such that the pre-determined infrastructure 1 may be utilized based on the properties of the pre-determined infrastructure 1, meaning for example computation power of the infrastructure nodes 6 and bandwidth of the infrastructure edges 8. These properties may be expressed as known quantities or a range of possible quantities, in which case multiple projections may be made in order to acceptably cover the range of anticipated infrastructural mutations.

Accordingly, the segmentation server may configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph 50 by dividing the dataflow graph 50 into the e dataflow sub-graphs, and grouping the dataflow sub-graphs into grouped

S dataflow sub-graphs based on the pre-determined infrastructure 1. 8 In one embodiment, the graph nodes of the dataflow graph comprise tensor operations. Then the segmenting of the dataflow graph in step 100 may z comprise scalarization of the tensor operations whereby graph nodes are so segmented into scalar operations or cohorts of scalar operations. The method a further may comprise grouping the scalar operations or cohorts of scalar = operations into scalar operation groups based on the pre-determined

N infrastructure 1. The scalar operation groups form the dataflow sub-graphs.

Scalarization means a process of representing tensor operations in a graph as scalar operations.

In the embodiment above, the segmentation server may be configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by scalarization of tensor operations, segmenting the graph nodes into scalar operations or cohorts of scalar operations and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure. The scalar operation groups form the dataflow sub-graphs.

In another embodiment, the graph nodes of the dataflow graph comprise tensor operations. Then the segmenting of the dataflow graph in step 100 may comprise representing the tensor operations as scalar promises representing scalar operations to be performed, receiving input data and fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises. The segmentation may further comprise segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations and grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure. The scalar operation groups form the dataflow sub-graphs.

In the embodiment above, the segmentation server may be configured to segment, by utilizing instructions of the segmentation module executed by the segmentation processor, the dataflow graph by the scalarization of tensor e operations of the graph nodes into scalar operations, representing tensor

S operations as scalar promises representing scalar operations to be performed, the 8 graph nodes of the dataflow graph comprising the tensor operations, receiving input data to the segmentation module, and fulfilling the scalar promises based z on the input data such that fulfilled scalar promises are formed from the scalar so promises. The segmentation server may be further configured, by utilizing a instructions of the segmentation module executed by the segmentation processor, = to segment the fulfilled scalar promises into scalar operations or cohorts of scalar

N operations, grouping the scalar operations or cohorts of scalar operations into scalar operation groups based on the pre-determined infrastructure, scalar operation groups forming the dataflow sub-graphs

In the latter embodiment, the segmenting step 100 may further comprise repeating, by utilizing instructions of the segmentation module executed by the segmentation processor, the following steps one or more times for fulfilling the scalar promises before the grouping of the scalar operations or cohorts of scalar operations: receiving input data, fulfilling the scalar promises based on the input data such that fulfilled scalar promises are formed from the scalar promises, and segmenting the fulfilled scalar promises into scalar operations or cohorts of scalar operations. Input data to scalar promises are the data that reduces ambiguity of the promises, such as the dimensionality of tensors or constant values, such that scalar promises can become fulfilled scalar promises, otherwise defined as scalar operations or cohorts of scalar operations such that result from using input data to resolve scalar promises.

Accordingly, in this embodiment the segmentation step 100 or the segmentation module are configured to represent tensors in two layers: scalar promises, which are formulas for generating operations that will be performed on the scalars that comprise tensors, and fulfilled scalar promises, which are the operations themselves. In other words, scalar promises are an added level of indirection, relative to prior art, that represents the scalar arithmetic that will be executed once dimensions of the tensors are known.

When the new input data is received in the segmentation step 100, or e the segmentation server 2 receives input data, the scalar promises become

S fulfilled and fulfilled scalar promises are formed. Once the dimensionality of the 8 dataflow graph is entirely known, the scalar promises are fulfilled. The fulfillment of scalar promises represents the finest granularity into which the dataflow graph z can be segmented, as every operation, or tensor, is decomposed into an so unambiguous number of binary and unary scalar operations, or scalar tensors. a The method of the present invention may further comprise compiling = the dataflow sub-graphs into a segmented dataflow graph comprised of the

N dataflow sub-graphs. This means that the sub-graphs are compiled together such that the segmented dataflow graph is formed. The segmented dataflow graph comprises the sub-graphs and the relationship between the sub-graphs.

A scalar promise can be fulfilled either by in-memory representations of scalar operations to be performed (i.e. objects in a computer program that represent addition, subtraction, etc.) or an algorithm that will deterministically generate a representation of these operations (i.e. a transpiler that takes descriptions of fulfilled scalar promises as input and produces output in a given language like assembly or C).

Scalar promises allow for graph currying. Currying is progressive evaluation of a function with partial information instead of the traditional delayed evaluation of a function until all information is present. By scalarizing subsets of the dataflow graph 50 where scalar promises can be immediately fulfilled (i.e. sections of a dataflow graph where the dimensions of incoming and outgoing matrices or operations are entirely known), the introduction of additional dimensional information progressively into the dataflow graph requires less time for the dataflow graph to be created. This allows for a compilation time of the dataflow graph to be spread out as information, meaning the input data, enters the dataflow graph. In industrial circumstances, where dimensionality of the dataflow graph may elaborate progressively as new information, or new input data, arrives, this can speed up the fulfillment of the scalar promises, which allows for the dataflow graphs to be compiled faster than the traditional delayed compilation of a dataflow graphs. 0 The segmentation of the dataflow graph in step 100 may also comprise

S merging redundant eguivalent dataflow sub-graphs such that merged dataflow 8 sub-graphs are formed. Merging means that eguivalent or same sub-graphs or operations are first identified and these repetitive sub-graphs or operations are z executed only once during the execution of the dataflow graph. Thus, the so segmentation server 2 may configured to segment, by utilizing instructions of the a segmentation module 20 and executed by the segmentation processor 12, the = dataflow graph 50 by merging redundant eguivalent dataflow sub-graphs such

N that merged dataflow sub-graphs are formed.

Because dataflow graphs represent the future execution of an algorithm, compression of scalar promises or fulfilled scalar promises can result directly in optimized process efficiency, requiring less memory and less computational operations. For example, if the same arithmetic operation is present in many different parts of a graph, this operation is detected via a

Hamiltonian search, isolated, and executed only once in the actual process. This gain in efficiency is akin to optimized compilation of a computer program, allowing data scientists to write algorithms with redundancies that are subsequently made more efficient.

Gains in data compression through the merging come from the detection of dataflow sub-graph substructures using traversals of the scalarized graph. Unlike dataflow graphs generated by Keras or Tensorflow, scalarized sub- graphs do not represent aggregate operations (i.e. a dot product or a convolution) but rather the individual operations that comprise the aggregates behind this intention (i.e. addition, multiplication). Because of this, novel patterns in the dataflow graph that emerge beyond intention by analysing individual operations can be identified and grouped together to reduce size of the dataflow graph.

Accordingly, sub-graphs having reduced size may be created.

Dataflow graph segmentation and merging via scalar promises and fulfilled scalar promises allow for greater gains in data compression than traditional means of dataflow graph representation. This, in turn, allows for dataflow sub-graphs to be transmitted over a network faster and more efficiently 0 than traditional forms of compression.

S The present invention may also comprise compressing the dataflow 8 sub-graphs into compressed dataflow sub-graphs before projecting the dataflow sub-graphs onto the pre-determined infrastructure 1. Alternatively, the dataflow z sub-graphs may first be compiled to the to the compiled segmented dataflow so graph, and then the compiled segmented dataflow graph is compressed. The a compression of the dataflow sub-graphs or the compiled segmented dataflow = graph may be carried out by utilizing any known compression protocol, such as

N pickle, zip or tar. The compression by utilizing known data compression protocols may be carried out before or after the merging step.

According to the above mentioned, the step 100 of dataflow graph segmentation into to the subgraphs may achieve the segmentation of operations or tensors in two sub-steps - the formulation of the tensors as scalar promises, which represent a range of possible scalar operations to be performed, and the fulfilment of scalar promises, which represent the actual operations as scalar tensors. By adding this extra level of indirection, dataflow graphs may be curried over time, they can be compressed by grouping similar operations, and redundant processes can be merged to make execution in the infrastructure more efficient.

In addition to these benefits, segmentation allows for sub-graphs to be compiled or sescribed using arbitrary compilation or description processes and projected onto an infrastructure.

The invention may comprise defining the pre-determined infrastructure in step 200 by generating one or more sub-graph projection predictions based on the infrastructure, the dataflow sub-graph projections representing the distribution of the dataflow sub-graphs to the infrastructure, and by defining the pre-determined infrastructure based on pre-defined infrastructure criteria. Then in step 300 the dataflow sub-graphs may be projected to the pre-determined infrastructure by projecting the one or more projection predictions to the infrastructure, and in step 400 the dataflow graph may be executed by executing the distributed dataflow sub-graphs according to one of the projection predictions. e The execution step 400 may comprise sending the sub-graphs to the

S infrastructure nodes and edges of the pre-determined infrastructure. 8 The step 200 of defining the pre-determined infrastructure may comprise analysing the infrastructure and representing the infrastructure as one z or more constraint optimization programs based on the analysing of the so infrastructure, generating one or more projection predictions by utilizing the a constraint optimization programs with constraints obtained from the analysing of = the infrastructure, and defining the pre-determined infrastructure as the

N infrastructure used to generate the one or more constraint optimization programs. This may be carried out with the segmentation server by, utilizing instructions of the projection module executed by the segmentation processor.

Alternatively, step 200 of defining the pre-determined infrastructure may comprise analysing the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the analysing of the infrastructure, generating one or more projection predictions by utilizing the constraint optimization programs with constraints obtained from the analysing of the infrastructure, and defining the pre-determined infrastructure based on the pre-defined infrastructure criteria. Also this may be carried out with the segmentation server by, utilizing instructions of the projection module executed by the segmentation processor.

The analysing may comprise analysing for example available computations capacity and power of the hardware nodes 6 and/or the available bandwidth or transmission speed of the network connections 8.

The pre-defined infrastructure criteria for defining the pre- determined infrastructure from the one or more projection predictions may be based on instruction stored to the segmentation server memory 10 or the projection module of the segmentation server 2.

Analysing the available infrastructure 1 and creating one or more projection predictions allows choosing desired pre-determined infrastructure for each situation based on the properties of the hardware nodes 6 and network connections 8. Furthermore, when the pre-determined infrastructure is defined e the details of the pre-determined infrastructure may be received in step 201 of

S figure 2 as input to the segmentation step 100. Accordingly, the details of the pre- 8 determined infrastructure may be utilized the segmentation step 100 and the sub-graphs may be formed based on the pre-determined infrastructure making z the execution the dataflow graph and the use of the infrastructure efficient. so In one embodiment, the step 200 of defining the pre-determined a infrastructure may comprise analysing a potential infrastructure by generating 5 synthesized representations of the infrastructure and representing the

N infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions based on the synthesized representations, and defining the pre-determined infrastructure based on the pre-determined criteria, by utilizing instructions of the projection module executed by the segmentation processor.

In one embodiment, the step 200 of defining the pre-determined infrastructure may comprise predicting the range of possible mutations of an actual or anticipated infrastructure by creating a model capable of generating multiple projections based on a description of an infrastructure, generating one or more projection predictions using the model, defining the pre-determined infrastructure from the one or more projection predictions based on pre- determined criteria and projecting one or more of the projections of dataflow sub-graphs to the pre-determined infrastructure. The synthesized representation of the infrastructure may be carried out by generating one or more projections predictions by first synthesizing several projections based on different infrastructure descriptions using alternative infrastructure descriptions as input to a predictive model such as a neural network in order to train the neural network to output projection predictions. The neural network is then used to output projection predictions from synthesized infrastructure representations, and the pre-determined infrastructure is definedfrom the one or more projection predictions based on pre-determined criteria and projecting one or more outputs of the predictive model onto the infrastructure. The projection predictions generated by the neural network based on the synthesized representation of the e available infrastructure may be provided to the projection module. Therefore, the

S segmentation server may be configured to, by utilizing instructions of the 8 projection module and executed by the segmentation processor, model the current or future infrastructure in one or more possible states by anticipating one z or more configurations of the hardware nodes and the network connections, so generating one or more projections based on the analysis of the hardware nodes a and the network connections, using these configurations and projections to make = predictions about additional projections defining the pre-determined

N infrastructure based on pre-determined criteria, and projecting one or more projections of dataflow sub-graphs to the hardware nodes and the network connections of the pre-determined infrastructure. Generating the one or more projection predictions may be accomplished by feeding a representation of the hardware nodes and the network connections as well as the projection that results that representation from the segmentation server 2 or the projection module 22 to a neural network 16 and receiving the one or more projection predictions from the neural network 16 to the segmentation server 2 of the projection module 22.

According to the above mentioned, the step 200 may comprise analysing a potential infrastructure by generating synthesized representations of the infrastructure and representing the infrastructure as one or more constraint optimization programs based on the synthesized representations, generating one or more projection predictions by utilizing a neural network trained with the results of the one or more constraint optimization programs as output and the synthesised infrastructure representation as input, and defining the pre- determined infrastructure based on the pre-determined criteria.

The neural network 16 may be any known kind of neural network, such as many-to-many recurrent neural network. The neural network 16 may be one neural network or number of interconnected neural networks, and the neural network 16 may be integral part of the segmentation server 2 or separate from the segmentation server 2 and connected to the segmentation server via a neural network connection 18, as shown in figure 5. Instead of a neural network 16 any e known predictive algorithm may be used for the same purpose.

S The step 300 of projecting the dataflow sub-graphs, or the segmented 8 dataflow graph, to the infrastructure comprises projecting the dataflow sub- graphs or the segmented dataflow graph to the pre-determined infrastructure. In z one embodiment, this is carried out by projecting the dataflow sub-graphs or the so segmented dataflow graph to the pre-determined infrastructure such that each a dataflow sub-graph is projected to a separate infrastructure node. Accordingly, = the segmentation server 2 may be configured to project, by utilizing instructions

N of the projection module 22 and executed by the segmentation processor 12, the dataflow sub-graphs to the infrastructure by distributing the dataflow sub-graphs to the hardware nodes 6 and network connections 8 of the pre-determined infrastructure 1 for execution of the dataflow graph. In one embodiment, this is carried out by distributing the dataflow sub-graphs to the hardware nodes and network connections of the pre-determined infrastructure such that each dataflow sub-graph is projected to a separate hardware node.

It should be noted that the projecting phase 300 and projecting may comprise sending or transmitting the sub-graphs to the infrastructure nodes 6.

Alternatively, the projecting may comprise sending or transmitting instructions to create the sub-graphs in the infrastructure nodes 6. In the latter case, the sub- graph is first created in the infrastructure node 6 based on the instructions received from the segmentation server 2 or the projection module 22 and then the sub-graph may be executed. Alternatively, the projecting may comprise sending or transmitting variables as input to pre-defined functions residing on the infrastructure nodes to create the sub-graphs in the infrastructure nodes 6. In the latter case, the sub-graph is first created in the infrastructure node 6 using a pre-defined function whose input variables are received from the segmentation server 2 or the projection module 22 and then the sub-graph may be executed. All three of these alternatives, or any combination thereof, mean projecting the sub- graphs to the infrastructure.

The method of the present invention may further comprise monitoring the infrastructure during the projection and/or execution of the dataflow graph e or the sub-graphs.

S Therefore, the segmentation server 2 may comprise a monitoring 8 module 24, as shown in figure 5, configured to monitor, by utilizing instructions of the monitoring module 24 and executed by the segmentation processor 12, the z infrastructure by monitoring the hardware nodes 6 and the network connections so 8 of the pre-determined infrastructure 1 during the execution of the dataflow a and/or the projection of the sub-graphs. 5 In one embodiment the segmentation server 2 or the monitoring

N module may be configured to monitor the hardware nodes and the network connections of the pre-determined infrastructure during the execution of the dataflow graph and receiving a monitoring output and to compare the monitoring output and a pre-determined expected output. Based on the comparison, if the monitoring output is within the pre-determined expected output, the execution or projection may be continued by utilizing the hardware nodes and the network connections of the pre-determined infrastructure in step 300 or 400. If the monitoring output is outside the pre-determined expected output, based on the comparison, the projection of the dataflow sub-graphs to the hardware nodes and the network connections may be re-initiated in step 401, or alternatively defining of the pre-determined infrastructure 1 may be re-initiated in step 402. The step 402 may also result in re-initiating the segmenting of the segmentation of the data flowgraph.

In the following, more detailed embodiments of the invention are described.

Projection step 300 is the process of taking a segmented dataflow graph, or sub-graphs, and projecting it onto the pre-determined infrastructure.

Dataflow graph projection may take the segmented dataflow graph and recombine it in order using a polyglot approach to most efficiently use the set of hardware nodes and network connections onto which it is projected. Segmented dataflow graph projection results in more efficient computation, allowing for savings in cost of bandwidth, memory and computation power.

Projection of the segmented dataflow graph means the mapping of a 0 dataflow graph onto the pre-determined infrastructure. For example, if there are

S two desktop computers connected on a local ethernet and both are connected to a 8 server in the cloud over a 4G connection, all nodes of the segmented dataflow graph are projected onto these three devices and the edges of the segmented z dataflow graph are projected onto the ethernet and the 4G network. so The dataflow graph projection module first identifies fixed and a variable aspects of infrastructure onto which the projection occurs, such as the = number of hardware nodes present or the throughput of bandwidth under

N different conditions. Different configurations of these variables may be used to generate a constraint optimization problem in which the unknown values represent binary (1/0) decisions for the hardware node onto which a fulfilled scalar promise is mapped.

For example, six constraint optimization problems can be formulated for three configurations of hardware nodes (1, 100, or 1 000 for example) on a system with two levels of data throughput (peak and off-peak for example), each of which is used to determine an optimal projection of fulfilled scalar promises onto an infrastructure.

Constraint optimization problems are a category of problems where an optimal solution to a function is found given one or more constraints. An example of a constraint optimization program that can be used by the projection module is a sparse mixed integer programming is a linear programming problem in which some of the variables are integers and most of the integers are 0. In this case, integers represent binary activations that a fulfilled scalar promise will be assigned to a particular hardware node. Data synthesis is a technique in machine learning whereby data fed to a network is created synthetically.

Analysing the infrastructure, generating one or more projection predictions based on the analysed infrastructure, defining the pre-determined infrastructure from the one or more projection predictions based on pre- determined criteria may be carried out only using a constraint optimization program. In this case, there is no need for utilizing a neural network. When the constraint optimization program with a constraint optimizer is used for e generating the predictions, the constrain optimizer yields a local minimum of a

S deterministic function, which is always accurate insofar as the encoded 8 constraints accurately represent the infrastructure. However, in the context of this application the outputs from the constraint optimization program are also z considered predictions. so The neural network could work for example in the following manner. a Synthesized data is used to train a neural network, for example many-to-many = recurrent neural network, that learns a generic model for how to project fulfilled

N scalar promises onto infrastructure. The input to this neural network is the synthesized representation of physical infrastructure and the output is a sparse matrix representing the probability that a fulfilled scalar promise will be assigned to a hardware node. The output of the neural network is compared to the output of a linear program using a categorical cross entropy loss function and, by doing this iteratively using classic forward and back propagation, the model “learns” how to project fulfilled scalar promises onto an infrastructure. It is then able to make projection predictions from new synthesised infrastructure representations.

The resiliency of the method is bounded by the synthetic data, and predictions of projections for infrastructures or fulfilled scalar promises that vary considerably from the synthesized data will be outside the scope of the model and generate noise. However, by training the model using synthesised data based on reasonable assumptions about infrastructure configurations and network conditions, the method learns to make reliable predictions for the most common real-world scenarios.

The provisioning of computation power may represent a hurdle for companies with distributed infrastructures, as an excess of computation power and, inversely, a dearth of computation power can lead throttling of data because of insufficient capacity.

This problem is solved using a method to project dataflow graphs onto infrastructure because the model can determine how many servers or devices need to be provisioned and terminated to scale with network demand, decreasing e operational cost by using computational resources more efficiently.

S Accordingly, the dataflow graph projection step 300 is the process step 8 of taking a segmented dataflow graph and projecting it onto a given infrastructure comprised of hardware nodes and network connections. This is accomplished z using the two-step protocol of synthesizing data about projections in various so network configurations using mixed integer programming and feeding this a synthesized data to the neural network that predicts, for a given infrastructure, = the optimal projection of fulfilled scalar promises onto that infrastructure.

N Projected dataflow graphs can change for different bandwidth configurations, redistributing an algorithm as a function of increased or decreased network transmission time.

Furthermore, the dataflow graph execution step 400 is the process of executing an algorithm on a dataflow graph projection. In the best-case scenario, the pre-determined infrastructure onto which the dataflow graph is projected remains unchanged for the lifetime of the execution and all expected network conditions hold. However, the execution must also deal with dynamic, unexpected features of the pre-determined infrastructure including failover, throughput volatility and erratic distributions of data. Furthermore, the execution must be resilient to changes in the configuration of infrastructure nodes and edges, including the addition or subtraction of nodes and edges. Lastly, the execution of the projected dataflow graph must accommodate data being produced and consumed in a decentralized manner. The scheduling of data ingress and egress are often immediate, but can be delayed or throttled to accommodate irregularities in a projected tensor graph that would otherwise result in suboptimal performance.

Dataflow graph execution may be a queue-based producer/consumer messaging model implemented on top of a protocol negotiation library such as libp2p. Each node of the dataflow graph projection represents data ingress and egress as queues with timeout and throttling mechanisms to accommodate failover and overload. Unlike traditional producer/consumer tools, implementing dataflow graph execution on a transport-layer agnostic peer-to-peer protocol e removes the single, central broker and allows for information to flow between

S hardware nodes in an unmediated fashion. Because there is no central broker of 8 information, the execution of the distributed dataflow graph cannot be monitored by traditional logging mechanisms. Instead, the network must be periodically and z stochastically sampled by monitoring for an expected outcome. Deviations from so the expected outcome of the distributed dataflow graph becomes the subject of a investigation into network failure. This is also called spamming the network. = A producer is a hardware node that produces data sent to another

N hardware node. A consumer is a hardware node that consumes data.

When using a centralized broker there is a single point of failure - the datacenter. Even if there are multiple brokers in the datacenter to manage failover, inclement network conditions that block or slow down access to a datacenter create a bottleneck in a distributed tensor graph. Peer-to-peer execution of a dataflow graph projection solves this problem, making dataflow graphs more resilient.

Dataflow graph segmentation, defining the pre-determined infrastructure, dataflow graph projection and dataflow graph execution, while presented as separable processes, are interdependent and are thus presented as part of a single process called dynamic distribution. The dynamic part of the distribution process is manifested through the following cross correlations. A dataflow graph may be segmented into fulfilled scalar promises based on a given amount of input data and output predictions in a distributed network. These fulfilled scalar promises are projected onto the infrastructure based on a constraint optimization program that outputs a sparse matrix representing the most efficient distribution of operations to hardware nodes given the constraints.

The inputs and results of this constraint optimization program are used to train to a neural network that learns features of projections and will automatically generate a best-fit projection for a given infrastructure. The projected infrastructure may be executed via producer/consumer queues on a peer-to-peer network. The actual performance of the execution, monitored by network spamming, is used to re-evaluate the constants used in the projections. The new e projections reguire the genesis of different models that will contain different

S fulfilled scalar promises. 8 The main benefit of dynamic manner of dataflow distribution is that it allows for the elegant elaboration of algorithms that optimize infrastructure z usage and automate processes like evolutive machine learning, paving the way for so a new generation of artificial intelligence algorithms and industrial applications. a The invention has been described above with reference to the 5 examples shown in the figures. However, the invention is in no way restricted to

N the above examples but may vary within the scope of the claims.