Channels ▼
RSS

Web Development

S3 Meets R3 (Reliability, Robustness, and Resilience)


Test Case I: Writing Small Files

The first test case involves writing a set of small files to Amazon S3 Web Services and to a remote SCP enabled server. The set of files (10k to 100k) are relatively small in size given that one can write up to 5GB in a single operation to Amazon S3. The objective of this test case to observe how Amazon S3 behaves with files of small size. Figure 2 illustrates the average transfer time it takes to write multiple small files to Amazon S3 and an SCP server.

Figure 2: Amazon S3 vs. SCP Small Size File Writes (PUT).

Observations: For Small file size writes (PUT), we make the following observations:

ü      Latency increases almost linearly with an increase in file sizes from 10-100k. This linear increase in latency holds true for both Amazon S3 and the SCP.

ü      Throughput ranges from ~ 160 Kbps -- 270 Kbps.

Test Case II: Writing Large Files

The second test case involves writing a set of large files to Amazon S3 Web Services and to a remote SCP enabled server. Figure 3 illustrates the time it takes to write large multiple files to Amazon S3 and an SCP server.

Figure 3: Amazon S3 vs. SCP Large Size File Writes (PUT).

Observations: For Large File size writes (PUT), we make the following observations:

ü      Latency increases almost linearly with an increase in file sizes from 1-10 MB. This linear increase in latency holds true for both Amazon S3 and the SCP.

ü      Throughput ranges from ~ 256 Kbps -- 347 Kbps.

Test Case III: Reading Small Files

The third test case involves reading a set of small files from Amazon S3 Web Services and from a remote SCP enabled server.

Figure 3 illustrates the time it takes to read multiple small files from Amazon S3 and an SCP server.

Figure 4: Amazon S3 vs. SCP Small Size File Reads (GET).

Observations: For Small file size reads (GET), we make the following observations:

ü      The latency for Amazon S3 service is remarkably lower than SCP reads.

ü      Throughput ranges from ~ 0.8 Mbps -- 1.3 Mbps, well within the advertised download bandwidth of 6-8Mbps.

Test Case IV: Reading Large Files

The fourth test case involves reading a set of large files from Amazon S3 Web Services and from a remote SCP enabled server. Figure 5 illustrates the time it takes to read multiple large files from Amazon S3 and an SCP server.

Figure 5: Amazon S3 vs. SCP Large Size File Reads (GET).

Observations: For Large file size reads (GET), we make the following observations:

ü      The latency for Amazon S3 service is significantly lower than SCP reads.

ü      Throughput ranges from ~ 4.75 Mbps -- 7.5 Mbps. The S3 reads consume all available bandwidth advertised at 6-8 Mbps.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Web Development Recent Articles

Dr. Dobb's TV

Upcoming Events



Most Recent Premium Content