Distributed Computing — Approaches
Distributed Computing is extremely powerful — but sometimes architects get it wrong. If not implemented properly — solution can create few issues.
There are Multiple Architectures one can deploy for archiving Distributed Process. Let’s discuss some below.
In this article let’s discuss points around when and what kind of Distributed Computing Architectures, Architects should consider before starting the Fabulous Distributed Computing Journey.
Approaches of Distributed Computing Architecture:
I go with the assumption that you already know about Distributed Computing (DC) and why it is extremely powerful and useful.
From where I stand, I see (in practical world) there are 2 different kind of DC Architectures.
While we can sub-categorise them, but for sake of simplicity let’s consider Centralised and Decentralised approaches.
Centralised Computing
This kind of Architecture as the name suggests is based on centralised task distribution system. All the intelligence of
- who should work on the task
- when should one work on the task
- how many tasks can be worked upon parallely
etc. are with the master.
In this setup tasks are populated in a central location (say DB, Files etc.) that a master application polls (or get notified by Messages). Master then invokes various Slaves to perform the appropriate tasks/ while controlling overall flow of the tasks.
Go for this kind of architecture when you would like to have finer control over the overall processing. Some of the Examples where I have seen it highly beneficial are :
- Flexibility: Solution requires realtime reshuffling of the tasks based on certain parameters (Business Impact — hence re-prioritise the tasks)
- Monitoring Health Issues: Ensuring if a Slave is stuck processing the task, task gets reassigned to healthy Slave.
- Optimal Hardware Resource Usage: Since master controls which slave gets to work on which task, this kind of setup leads to optimal usage of available resources.
Biggest Drawback of this type of approach is Master being Single point of Failure. If Master goes down, entire system crumbles.
De-Centralised Distributed Computing
These is no concept of Master or Slave here.
This approach leads to all the Nodes unaware of presence of any other Nodes. They are programmed to work independently. They themselves decide on what to and when to pick up tasks.
Ideally all slaves should be able to perform all kind of tasks.
Go for this approach for:
- System Availability : When System Availability is of utmost Importance. Biggest positive of this approach is the System Availability. So long as one Node is up, work will not stop completely.
- Ease of Scaling Up/Down: Also scaling up/down is extremely simple/ its plug and play. Add a slave, and System scales up no other configurations required. Remove a slave and System scales down automatically (No Other Configurations would be Required).
Not discussing about Auto Scaling here.
Biggest Drawback of this solution is
- The complexity that would creep in — when you would try to make Nodes intelligent to ensure that slaves do not run into Race Conditions while pickup tasks.
- If a Node is Stuck, Task may never gets released automatically for another Slave. You will have to add external Monitoring solutions.
With Container Orchestration solutions such as Kubernetes — some of the Pros (such as Ease of Scaling) becomes Not so Pro; while some of Cons such as Node Getting Stuck — no more remains that big a CON.
For more details refer to my other Article on Decentralised Computing.
Overall:
Distributed Computing is Great as it can help in :
- System Performance
- System Availability
- System Reliability
If decided to use Distributed Computing, while Architecting the solution be sure what type of Architecture you would like to use. Use :
- Centralised Computing : For Optimal Resource Usage; and overall System Flexibility.
- Decentralised Computing : For High System Availability.
If you have any questions, comments — please share, I would love to listen.
