Channels ▼

Cameron and Tracey Hughes

Dr. Dobb's Bloggers

Parallelism Should Inspire You

September 29, 2009

We struggle to think in parallel and develop models and paradigms that imitate behavior with innate parallelism. Peer-to-peer, workpile, boss worker, consumer producer describe models for concurrent work and communication. But are these models sufficient to model the behavior of complex systems that utilize massive parallelism?We haggle over tools and blackbox parallelism struggling for a paradigm that will work for complex systems. Where do we get our inspiration? Peer-to-peer, workpile, boss worker are models we are familiar with in our day-to-day work environments. Are there any other work distribution models that can scale?

We are surrounded by parallelism even more so that serial activities. Social and biological systems are immersed in parallelism and these systems have worked out the kinks over millions of years. In biological systems, independent units perform simple computations or activities independently and in parallel. They create new states by combining internally generated information with signals received from other units. Yeah ... what's the big deal about that? We know and use that model. But this is also the basis for the activity of our nervous system. For our purposes the nervous systems' massive parallelism, work distribution, and signaling model is worth examining.

The nervous system is a complex information-processing system that regulates the body's actions and responses to internal and external stimuli. It is a complex nerve network consisting of the brain and spinal cord (this is the central nervous system) and the peripheral nervous system. Neurons are the core components. There are 10 ^11 neurons. Neurons generate and conduct electrochemical signals between them and between the CNS and the PNS. There are specialized neurons that respond to certain types of signals (touch, sound, light, etc.). Sensory and connector neurons make up the peripheral nervous system connecting it to the CNS. When a stimuli, a detectable change, (which can also be viewed as some event) has occurred in the system, the appropriate neuron responds to the stimuli, processes the information then sends a response to other parts of the system for action. Neurons also communicate to other neurons by releasing neurotransmitters (a special signal between neurons) into the chemical synapse. This is the basic concept that manages the communication and signaling in the nervous system. The analogy is obvious. There are threads, a hierarchy of different types of threads, specialized signals and events, and the domain specific actions and responses.

The ";swarm"; paradigm has been used to model a type of intelligence. It is an example of collective behavior. A swarm defines a group of mobile unsophisticated agents working in parallel that communicates with other agents and interacts with their environment. There is typically no centralized control that dictates the behavior of the mobile agents. But as a result of their interactions, a distributive collective problem solving strategy emerges. This collective problem solving strategy denotes a coherent functional global pattern considered to be the intelligence of the swarm. But swarm intelligence is just one type of collective intelligence. There are others such as plants and bacteria in which their behaviors are quite different from swarms.

Such paradigms/models of complex massive parallel systems have been used by computer scientists for many years and have influenced systems beyond just communication and work distribution. Here are a few more examples:

CHARACTERISTIC: Cell Aggregation

NATURE: Cells in the body join together to form tissue. Tissues form different organs like stomach, lungs etc. Organs form systems like Respiratory system, digestive system.

IMPLICATIONS FOR COMPLEX REALTIME SYSTEMS: Complex real-time systems would be a sea of state machine objects interacting with each other. There will be low level state machines which would work together to implement high level state machine behavior.


NATURE: Most biological systems are able to recover from a large number of fault conditions. These cases range from re-growing organs, recovering from a cut in the skin to fighting with invading virus and bacteria. Due to these "fault handling" techniques the human body is able to recover from most fault conditions. We need to see a doctor only if the natural defenses fail.

IMPLICATIONS FOR COMPLEX REALTIME SYSTEMS: As man made systems increase in complexity, the systems will need to be capable of recovering from all kinds of software and hardware failure conditions. This will become essential as cost of maintaining complex systems would be prohibitive if the operator is expected to fix every little problem in the system.

CHARACTERISTIC: Layered Architecture

NATURE: The biological systems have a layered architecture and interactions between objects at high level are very well defined. For example the interactions between the stomach and intestines are fairly well defined. However there are hardly any interactions between a cell in the stomach and a cell in the intestine. The interactions take place at clearly defined interface points.

IMPLICATIONS FOR COMPLEX REALTIME SYSTEMS: Complex systems should be designed in a layered fashion. The system is divided into sub-systems with clearly defined interfaces between sub-systems. Very few objects in a sub-system should interact with objects in the other sub-system.

Source: Anatomy of a Complex System, Nature vs. Man Made Systems

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.