Warehouse

From XtremWebCH Wiki

Description

XWCH supports direct communication between workers executing two communicating tasks. Direct communication can only take place when the workers can “see” each other. Otherwise (one of the two workers is protected by a firewall or by a NAT address), this kind of communication is impossible. In this case, it is necessary to pass by an intermediary (XWCH coordinator for example).

However, to avoid overloading the coordinator, one possible solution consists of installing “warehouse” nodes which acts as an intermediary. These nodes are used by workers to download input data needed to execute their allocated task and/or upload output data produced by the task.

A warehouse node acts as a repository or file server. It must be reachable by all workers contributing to the execution of a given application.

The protocol is the following:

• The list of available warehouses is received by a worker when it registers nearby a coordinator (Register Request)
• When a worker finishes the execution of a task it uploads its result in a one of the known warehouses (selected randomly). Thus, the result is stored in the worker and in the warehouse,
• The worker sends a work result to the coordinator with the two locations (IP address and path) of the result produced by the given task,
• When a worker sends a Work Request to execute a new task, it receives as a reply, the binary code of the allocated task and the two locations of its input data.