Serial and Parallel Processing
In this section we will introduce the concepts of serial and parallel processing
Serial Processing
In serial processing, sub-processes happen sequentially in time. Only one sub-process is active at any given time.
Serial processing:
The main drawback of serial processing is speed. Speed depends only on the rate at which each sub-process will occur (e.g. processing unit clock speed). The clock speed is the number of instruction cycles the CPU can deal with in a second. For more information about CPUs please see the reading Cores and Nodes.
Parallel Processing
In parallel processing multiple sub-processes can be active simultaneously, and is referred to as parallelism.
Parallel processing:
Speed depends on execution rate of each sub-process AND how many sub-processes can be made to occur simultaneously.
Why use parallelism?
Parallel processing is used for two main reasons: capacity and capability.
Parallelism for capacity
Example: Searching database of web-sites for some text (i.e. Google)
Searching sequentially through large volumes of text too time-consuming.
If this process is parallelised:
- Multiple servers hold different pages.
- Each server can report the result of each individual search.
- The more servers you add the quicker the search is.
Parallelism for capability
Example: Large scale weather simulation
Detailed description for atmosphere too large to run on today’s desktop or server PCs.
Therefore:
- Multiple servers are needed to hold all grid data in memory.
- Servers need to quickly communicate to synchronise work over entire grid
However:
- Communication between servers can become a bottleneck.
Large scale weather forecast:
All computers today are designed for parallel execution
But…