Channels ▼
RSS

Database

Synchronize Now!

Source Code Accompanies This Article. Download It Now.


The DataTransfer Abstract Class

Synchronize Files uses the DataTransfer abstract class to transfer files to and from S3 or an FTP site. The DataTransfer class includes nonabstract methods such as UploadFile that are independent of the transfer mechanism (S3 or FTP), and abstract methods such as UploadBuffer that are totally dependent on the transfer mechanism, and implemented in the S3DataTransfer and FTPDataTransfer classes. The sample app is able to transfer files using a DataTransfer object, without having to be aware of the differences between S3 and FTP.

DataTransfer maintains metadata for each file that is uploaded to the server. The metadata includes an SHA1 hash of the file's contents. This hash is a 160-bit binary number that is computed from the file's contents. Because it is extremely unlikely that two different files will have the same hash, the hash is used to avoid unnecessary file transfers. If the user requests a file download, the file will only be downloaded if the server's copy has a different hash than the local file. And a file will only be uploaded if its hash differs from the hash of the corresponding server file. The sample app also uses the hash to detect file corruption. When a file is downloaded, the sample app knows the file's contents have changed due to corruption or sabotage if its hash differs from the one stored on the server. To compute a hash from an array of bytes, instantiate an SHA1Managed object and call its ComputeHash method.

DataTransfer compresses and encrypts all files uploaded to the S3 or FTP server, and decrypts and decompresses all downloaded files. The Serialization class does the compression/decompression and encryption/decryption. For example, Synchronize Files calls Serialization.ObjectToBase64String (Listing Two) to compress and encrypt a file prior to uploading it. First the ObjectValue parameter is serialized to a MemoryStream using a BinarySerializer. The serialized data is extracted from the MemoryStream into a byte array. The byte array is then compressed using the Compression class. Thanks to the Enterprise Library's Cryptography Application Block, encrypting the byte array is just a simple call to Cryptographer.EncryptSymmetric. Decryption is just as easy, just a call to Cryptographer.DecryptSymmetric. In addition to simplifying encryption and decryption, the Cryptography application block decouples the cryptography algorithm from the code that uses it. The sample app is configured to use the Rijndael algorithm, but a different algorithm can be substituted with no code changes. Just run the Enterprise Library Configuration program. Select File/Open Application and open the sample app's app.config file. Right-click the Symmetric Provider and select Remove. Then right-click Symmetric Providers and select New/Symmetric Algorithm Provider. Specify a different algorithm (say, TripleDESCryptoServiceProvider), create a key, specify a key file path, and specify the data-protection mode. Finally, change the name of the TripleDESCryptoServiceProvider on the right to Symmetric Provider.

 ...

public static class Serialization
{
    public static string ObjectToBase64String(object ObjectValue)
    {
        string objectString = null;
        using (MemoryStream memoryStream = new MemoryStream())
        using (StreamReader streamReader = new StreamReader(memoryStream))
        {
            BinaryFormatter binaryFormatter = new BinaryFormatter();
            // Serialize the object into the memory stream.
            binaryFormatter.Serialize(memoryStream, ObjectValue);
            // Rewind the memory stream.
            memoryStream.Position = 0;
            // Extract the object's data into a byte array.
            byte[] buffer = new byte[memoryStream.Length];
            Array.Copy(memoryStream.GetBuffer(), buffer, buffer.Length);
            // Compress the object's data.
            buffer = Compression.Compress(buffer);

            try
            {
                int plainTextLength = buffer.Length;
                // Encrypt the object's data.
                buffer = Cryptographer.EncryptSymmetric(
                    StringLiterals.SymmetricProvider, buffer);
                int cipherTextLength = buffer.Length;
                Trace.WriteLine(string.Format(
                    "{0} bytes encrypted into {1} bytes", 
                    plainTextLength, cipherTextLength));
                // Convert the compressed, encrypted object data 
                // into a base 64 string.
                objectString = Convert.ToBase64String(buffer);
            }
            catch (Exception ex)
            {
                objectString = null;
                if (ExceptionPolicy.HandleException(ex, 
                        StringLiterals.CaughtExceptionPolicy))
                {
                    throw;
                }
            }
        }
        return objectString;
    }
    ...
}
Listing Two

Encrypting your files is useless if a bad guy gets hold of your private key. That's why the Cryptography Applicaton Block encrypts keys using DPAPI. This makes the key useless on any machine other than the one on which it was originally encrypted. This extra level of security makes it less convenient to transfer your key to other machines, which is exactly what you'll need to do if you intend to run Synchronize Files on multiple computers. To transfer a key to another machine, first make sure your key file is writable. Then run the Enterprise Library Configuration program, select File/Open Application, and open the sample app's app.config file. Right-click the Symmetric Provider on the left and select Export key. Specify a filename and a password for the exported key and press OK. Then copy the exported key file to the second machine. On the second machine, run the Enterprise Library Configuration program. Remove the Symmetric Provider on the left. Right-click Symmetric Providers on the left and select New/Symmetric Algorithm Provider. Choose RijndaelManaged and press OK. Then select the "Import a password-protected key file" radio button and press Next. Choose the exported key file and enter the password you used to encrypt it. Click Next and specify where the imported key file should be stored. After completing the key import process, change the Name field on the right to Symmetric Provider.

The Compression class (Listing Three) uses the .NET 2.0 GZipStream class to compress and decompress files. Compression.Compress instantiates a MemoryStream and associates it with a GZipStream with a CompressionMode of compress. When a byte array is written into the GZipStream, then retrieved using the MemoryStream's GetBuffer method, it is compressed. Attempting to compress very small files, or already compressed files such as Zip or JPEG files, will tend to result in larger, not smaller, files.

 ...
public static class Compression
{
    ...
    public static byte[] Compress(byte[] Buffer)
    {
        byte[] result = null;
        using (MemoryStream memoryStream = new MemoryStream())
        using (GZipStream gzipStream = new GZipStream(memoryStream, 
                                    CompressionMode.Compress, true))
        {
            // Write the buffer into the GZip stream.
            gzipStream.Write(Buffer, 0, Buffer.Length);
            gzipStream.Close();
            // Extract the compressed data from the GZip stream.
            result = new byte[memoryStream.Length];
            Array.Copy(memoryStream.GetBuffer(), result, result.Length);
        }
        if (Buffer.Length > 0)
        {
            Trace.WriteLine(string.Format(
              "{0} bytes compressed to {1} bytes ({2}%)", Buffer.Length, 
                 result.Length, ((double)result.Length / 
                    (double)Buffer.Length) * 100.0));
        }
        return result;
    }
    public static byte[] Decompress(byte[] Buffer)
    {
        byte[] result = null;
        using (MemoryStream memoryStream = new MemoryStream(Buffer))
        using (GZipStream gzipStream = new GZipStream(memoryStream, 
                                 CompressionMode.Decompress, true))
        {
            result = ReadBytes(gzipStream);
        }
        return result;
    }
}
Listing Three

The DataTransfer class includes two methods—EncodeName and DecodeName—that relax restrictions in S3 and FTP bucket and file names. For example, S3 Bucket Names currently only allow ASCII letters, numbers, underscores, and dashes. This is a surprising restriction considering the popularity of Unicode for web-service text encoding. The sample application can handle bucket names and filenames with no restrictions because EncodeName converts unsupported characters to supported characters, and DecodeName does the reverse. This lets the sample application display bucket and filenames in high market-share languages such as Chinese.

DataTransfer the Enterprise Library's Logging Application Block to log file transfers and errors. The log file is named trace.log and is stored in the same folder as the .exe. Most messages go to this log file. Serious errors are sent to the system event log that you can inspect by going to Control Panel/Administrative Tools/Event Viewer. The Logging Block provides two major advantages to applications that use it:


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video