Channels ▼
RSS

Parallel

Python NetWorkSpaces and Parallel Programs

Source Code Accompanies This Article. Download It Now.


NWS Iterators

It's also possible to iterate through values in NWS. ifind returns an iterator that returns each value bound to the specified variable:

for v in range(5): ws.store('v', v)
for v in ws.ifind('v'): print v

If you try this, notice that the second loop hangs after printing five values. This is because ifind is waiting for more values. If you use another process to store more values to v, it wakes up and prints them. To signal it to stop, you can delete the binding:

ws.deleteVar('v')

Alternatively, you can use ifindTry, which creates an iterator that terminates as soon as there are no more values. Finally, ifetch and ifetchTry are similar methods that consume the values as they iterate. NWS iterators can only be used on FIFO- or Single-mode variables.

Managing Workspaces

NWS supports multiple workspaces to help organize variables into related groups. We've seen that the constructor takes the name of a workspace as its first argument. It could also take a hostname and a port number to specify the server that hosts the workspace. (Something like this had to be coming. Without it how could we form ensembles with parts running on distinct computers?)

So if multiple processes each execute:

ws = NetWorkSpace('example', serverHost='myhost.mycorp.com')

they would all have access to the same workspace, which is managed by the NWS server accessible through a default port on myhost.

But who would own it? And why would that matter? By default, the process that first mentions the workspace to the server owns it. This matters because we eventually need to clean up. Workspaces and the data in them can be persistent, meaning that they exist until explicitly destroyed. In practice this can lead to a mess, so they are by default transitory—when the process that owns them exits, they are destroyed. Persistence is useful in certain settings and is available via an option to the workspace constructor.

Usually the default ownership policy does the right thing; a master process mentions the NWS first and becomes the owner. Then other processes that use it can come and go without destroying the NWS. Finally, when the master exits, the NWS is cleaned up.

But sometimes you want to be able to mention an NWS without inadvertently becoming the owner, as might happen if an auxiliary process beat the master to the first mention. Declaring the NWS with useUse true indicates that the process doesn't want to own the NWS:

ws = NetWorkSpace('not mine', useUse=True)

Workspace naming presents a bit of a challenge: How can we avoid name collisions? Consider, for example, multiple instances of an ensemble application using the same NWS server. Each NWS workspace object has associated with it a server object (encapsulating the connection to the server managing the workspace). This object provides a mktempWs method that works something like Linux's mkdtemp. It returns a reference to a workspace whose name is guaranteed to be unique on the server. Once we've got our own workspace, we can use variable names within it without worrying about those colliding with some other ensemble program.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video