On Parallel and Concurrent computation (Extracted from "Program on a Higher Level" course - EPFL)

A parallel program is a program that uses the provided parallel hardware to execute a computation more quickly. As such, parallel programming is concerned mainly with efficiency.
Parallel programming answers questions such as, how to divide a computational problem into sub-problems that can be executed in parallel. Or, how to make best use of underlying parallel hardware to obtain the optimal speed-up.

A concurrent program is a program in which multiple executions may or may not execute at the same time.
Concurrent programming serves to structure programs better in order to improve modularity, responsiveness to input and output events, or understandability of the program.
As such, concurrent programming is concerned with questions like, when can a specific execution start? When and how can two concurrent executions exchange information? And how do computations manage access to shared resources, such as files or database handles?

So parallel programming is mainly concerned with algorithmic problems, numerical computations, or big data applications. Examples of such applications include matrix multiplication, data processing, computer graphics rendering, or simulation of fluid movement.

Concurrent programming is targeted at writing asynchronous applications such as web servers, user interfaces, or databases.

While parallel programming is concerned mainly with speed-up, concurrent programming is concerned with convenience, better responsiveness and maintainability.


The two disciplines often overlap. Parallel programming may rely on insights from concurrent programming and vice versa. Concurrent programming may be used to solve parallel programming problems. However, neither discipline is the superset of the other.

No comments: