What is distributed programming?
Learn about distributed programming and why it's useful for the cloud, including programming models, types of parallelism, and symmetrical vs. asymmetrical architecture.
Learning objectives
In this module, you will:
- Classify programs as sequential, concurrent, parallel, and distributed
- Indicate why programmers usually parallelize sequential programs
- Explain why cloud programs are important for solving complex computing problems
- Define distributed systems, and indicate the relationship between distributed systems and clouds
- Define distributed programming models
- Indicate why synchronization is needed in shared-memory systems
- Describe how tasks can communicate by using the message-passing programming model
- Outline the difference between synchronous and asynchronous programs
- Explain the bulk synchronous parallel (BSP) model
- Outline the difference between data parallelism and graph parallelism
- Distinguish between these distributed programs: single program, multiple data (SPMD); and multiple program, multiple data (MPMD)
- Discuss the two main techniques that can be incorporated in distributed programs so as to address the communication bottleneck in the cloud
- Define heterogeneous and homogenous clouds, and identify the main reasons for heterogeneity in the cloud
- State when and why synchronization is required in the cloud
- Identify the main technique that can be used to tolerate faults in clouds
- Outline the difference between task scheduling and job scheduling
In partnership with Dr. Majd Sakr and Carnegie Mellon University.
Prerequisites
- Understand what cloud computing is, including cloud service models and common cloud providers
- Know the technologies that enable cloud computing
- Understand how cloud service providers pay for and bill for the cloud
- Know what datacenters are and why they exist
- Know how datacenters are set up, powered, and provisioned
- Understand how cloud resources are provisioned and metered
- Be familiar with the concept of virtualization
- Know the different types of virtualization
- Understand CPU virtualization
- Understand memory virtualization
- Understand I/O virtualization
- Know about the different types of data and how they're stored
- Be familiar with distributed file systems and how they work
- Be familiar with NoSQL databases and object storage, and how they work