Distributed Computing Gets Spruced-Up with CCR/DSS!

  • Article

The reality of distributed systems development today is the complexity that eventually if not immediately bogs it down. Instead of being fun, exciting and easy to develop, distributed systems are often hard and unwieldy to design and work with. We continue to face the same old issues time and again yet never appear to come up with game-changing simplifying solutions to our common issues of complexity which, amongst others, are:

  • The large variety of participants: consumers, providers, intermediaries, administrators, etc.
  • Highly heterogeneous systems and architecture, implementation and deployment practices
  • Highly autonomous and independent systems & services
  • Highly distributed information sources and services

The challenges developers, architects, testers, etc. face are many. In general we need to be able to:

  • Apply (re)configuration in a live running system
  • Coordinate many simultaneous asynchronous interactions
  • Support high throughputs (and afford low latencies)
  • Provide simple lifecycle services like starting and stopping components dynamically
  • Monitor, interact with, and debug a running (distributed) system
  • Ideally be able to develop the system in simulated conditions or a test environment
  • Span multiple physically or logically distributed compute units
  • Re-use components across multiple hardware platforms & devices

A typical solution to these challenges might be to centralise & standardise, but that's a kind of anti-pattern for many types of distributed systems scenarios today and simply serves to:

  • Reduce the variety of systems, i.e. seeking economies of scale
  • Remove the positive benefits of heterogeneity in a distributed system
  • Eliminate the autonomy
  • Put demands on 'perfect information' which is really hard to do consistently and reliably
  • Hardwire the connections between components of the system

A better solution is to simply accept diversity and decentralize!

  • Accept variety, i.e. seeking economies of innovation
  • Encourage autonomy
  • Accept heterogeneity
  • Understand that that custom, personalized, networked information is normal
  • Imagine an adaptable, open, distributed information system

Recently I've been energized by the power of the Concurrency and Coordination Runtime (CCR) + Decentralized Software Services (DSS) combined features, and dare I say, paradigm shifting concepts to fix a lot of the problems we have today in building distributed systems.

  • CCR provides concurrent (multi-core) programming support by way of some key pattern implementations and developer conveniences
  • DSS provides higher level support to create, host, and discover lightweight services
  • DSS services are oriented to have a state and react to incoming request and updates, and sends update and request to other services
  • DSS services state can be serialized and a service can migrate to another host
  • DSS services are naturally RESTful
  • The Visual Programming Language designer can be used to define, compose, monitor and manage DSS services
  • CCR/DSS can be hosted in a device and is capable of running under .NET Compact Framework (and .NET Micro Framework coming soon)

Side Note - How does CCR/DSS relate to WCF?

WCF is more about communications infrastructure – standards-based ‘plumbing’ – whereas CCR/DSS is a high level programming model for distributed service-based and resource-oriented systems (however initially targeted to Robotics applications). Some key CCR features such as ports, memory queues, dispatchers, arbiters, etc. are hard to implement and beyond the scope of WCF. A better comparison would be the combo of WCF+WF, but still there would be many features and patterns missing from both sides. Apples to Oranges - just avoid trying to force a comparison and understand the core use cases for each technology set. In my opinion the patterns supported in CCR are also of immense value to WCF+WF developers. I wonder if anyone has implemented CCR patterns in WCF and furthermore provided a DSSP transport protocol binding for WCF? That would be really cool!

Resources

The Microsoft Robotics Studio includes the Concurrency and Coordination Runtime (CCR), Decentralized Software Services (DSS), Visual Programming Language (VPL), and a virtual worlds (robotics) simulation environment! If you want to dive straight into CCR/DSS check out the Wiki, watch a video or two and install Microsoft Robotics Studio.

As part of the Microsoft Robotics Studio (1.5) release, Microsoft has placed the Decentralized Software Services Protocol (DSSP) under its Open Specification Promise. This means that the Microsoft patents for the services-oriented protocol specification may be used by anyone in the world, at no cost, and for any type of development including free software, open source, academic, or commercial, enabling interoperability among a wide variety of devices and applications. The XSD schema for DSSP is also available.

MSRS Blog Main forum page DSS forum CCR forum
VPL forum Hardware configuration and troubleshooting forum Simulation forum Community examples forum

The creators of CCR/DSS are George Chrysanthakopoulos and Henrik Frystyk Nielsen. Search on their names and you'll be very impressed.

 

CCR/DSS is not just for Robotics - it'll help you spruce-up your enterprise distributed systems architectures too! Like I said earlier, I'm pretty jazzed by this technology and hope to be writing about it again very soon.