Secure File Synchronization

DiskBoss Server provides secure file synchronization operations allowing one to synchronize files between servers without using network shares while encrypting transferred files using the AES-256 encryption algorithm. The user is provided with multiple one-way and two-way file synchronization modes, flexible performance optimization options, file matching rules allowing one to synchronize specific types of files or file groups, exclude directories and advanced file synchronization options.

Secure file synchronization operations may be performed via the network without using network shares between DiskBoss Ultimate and DiskBoss Server or between two DiskBoss Servers. In order to perform a secure file synchronization operation, the user needs to configure a secure file sync command on the source server and specify a source directory, the host name or an IP address of the destination server and a destination directory on the destination server. DiskBoss Server or DiskBoss Enterprise should be running on the destination server and secure file transfers should be enabled on the 'Options' dialog.

In order to enable encryption and/or compression of transferred files, open the secure file sync command dialog, select the 'Options' tab and enable all the required file transfer options. When the user enables encryption of transferred files, DiskBoss dynamically generates a random encryption key for each transferred file and encrypts each transferred data block using the AES-256 encryption algorithm. When the user enables verification of copied files, DiskBoss verifies all transferred data blocks using the SHA256 data signature algorithm.

Secure File Synchronization Commands

DiskBoss Server provides the user with the ability to configure an unlimited number of secure file synchronization commands with each one synchronizing files from a source directory on the source server to a destination directory on the destination DiskBoss Server. In addition, DiskBoss allows one to configure multiple secure file synchronization commands synchronizing files between the source server to multiple different destination DiskBoss Servers.

In order to add a secure file synchronization command, press the right mouse button over the commands view, select the 'Add New - Secure File Sync Command' menu item and specify a unique command name.

On the secure file synchronization command dialog, specify a source directory, a destination directory and the host name or an IP address of the destination DiskBoss Server. In addition, the user needs to specify the DiskBoss user name and password to login to the destination DiskBoss Server (default is admin/admin). Optionally, in the case the destination DiskBoss Server is configured to use a custom TCP/IP port (default is 8094), the user needs to configure an appropriate TCP/IP port number.

By default, due to security reasons, server-to-server file transfers are disabled and the user is required to explicitly enable server-to-server file transfers on the destination DiskBoss Server. In order to enable server-to-server file transfers, connect to the destination DiskBoss Server using the DiskBoss client GUI application, press the 'Options' button located on the main toolbar, select the 'General' tab, enable the 'Server-To-Server File Transfers' option and press the 'Save' button.

Secure File Synchronization Enable Transfer

In order to transfer files, the source DiskBoss Server will connect to the destination DiskBoss Server using the TCP/IP protocol and the configured TCP/IP port should be open in the destination server's firewall. Control messages sent from the source DiskBoss Server to the destination server are always encrypted using the AES-256 encryption algorithm with dynamically generated random encryption keys unique for each DiskBoss Server. Also, the user is provided with the ability to encrypt all transferred files using the AES-256 encryption algorithm with a dynamically generated random encryption key for each transferred file.

DiskBoss provides a number of different one-way and two-way file synchronization modes optimized for different usage scenarios. In order to select an appropriate file synchronization mode, open the secure file synchronization command dialog, select the 'Advanced' tab and change the 'File Sync Mode' combo box.

Secure File Synchronization Options

DiskBoss Server provides a large number of secure file sync options allowing one to customize the secure file synchronization operation for user specific needs. In order to customize secure file sync options, select the secure file synchronization command, press the right mouse button, select the 'Edit Command' menu item and select the 'Options' tab.

Sync File Attributes - this option enables synchronization of file attributes. If this option is disabled, files created in the destination directory will have default file attributes.
Sync File Timestamps - this option enables synchronization of the creation, last modification and last access dates for all transferred files. If this option is disabled, files transferred to the destination directory will have all timestamps set to the time of the file synchronization operation.
Compress Transferred Data Blocks - this option enables compression of transferred files. Depending on the speed of the network and the type of transferred files, it may be possible to increase the speed of the secure file synchronization operation and/or reduce the amount of transferred data. For example, text files and uncompressed bitmap images are highly compressible and when transferred with the compression option enabled the actual amount of transferred data may be significantly less than the size of uncompressed files thus increasing the performance of the secure file sync operation. On the other hand, JPEG images and compressed archive files are already compressed and therefore the actual amount of transferred data will be almost the same, but DiskBoss Server will need to use more CPU resources while trying to compress uncompressible files. Another point that should be considered is the speed of the network. When files are transferred over a fast network, the resulting impact may be not significant, but if files are transferred over a slow network, where the network speed is the major bottleneck, the resulting impact may be very significant.
Encrypt Transferred Data Blocks - this option enables encryption of transferred files using the AES-256 encryption algorithm with a dynamically generated random encryption key for each transferred file.
Verify Transferred Data Blocks - this option enables verification of all transferred data blocks using the SHA-256 data signature algorithm and retransmission of all corrupted data blocks. When this option is disabled, DiskBoss Server verifies transferred data blocks using a simple CRC32 checksum, which is good enough for reliable networks, but if the user needs to transfer important files through an unreliable network, it is recommended to enable verification of transferred data blocks using the SHA-256 data signature algorithm with subsequent retransmission of all corrupted blocks of data.

Secure File Synchronization Advanced Options

DiskBoss Server is optimized for modern multi-CPU/multi-core platforms, Gigabit Ethernet networks and RAID storage arrays. The user is provided with a large number of customization options allowing one to customize the behavior and performance of secure file synchronization operations for user-specific needs and hardware configurations.

File Sync Mode - DiskBoss provides a number of different one-way and two-way file synchronization modes optimized for different usage scenarios. In order to select an appropriate file synchronization mode, open the secure file synchronization command dialog, select the 'Advanced' tab and change the 'File Sync Mode' combo box.
Performance Mode - this option provides the ability to intentionally slow down the secure file synchronization operation in order to minimize the performance impact on running production applications. In the 'Full Speed' performance mode, DiskBoss will try to transfer files as fast as possible. In the 'Medium Speed' performance mode, DiskBoss will try to perform the secure file synchronization operation at 50% of the maximum possible speed. In the 'Low Speed' performance mode DiskBoss will try to perform the file synchronization operation at 20% of the maximum possible speed.
Dir Scan Threads - this option controls how many parallel directory scanning threads are used to scan the specified source directory. By default, DiskBoss scans directories using a single directory scanning thread and it is recommended to increase the number of parallel directory scanning threads only when synchronizing millions of files via a high-latency network.
File Sync Threads - this option controls how many parallel file transfer threads are used to transfer files to/from the destination DiskBoss Server. In order to reach the maximum possible speed when transferring files over a Gigabit network connection, it is required to use 2-4 parallel file transfer threads. Moreover, when transferring millions of files over a high-latency network, in order to mitigate the network latency, it is recommended to configure the secure file synchronization operation to use 4-8 parallel file sync threads.

Secure File Synchronization Rules

DiskBoss secure file synchronization operations provide the ability to sync files matching user-specified rules. For example, the user is provided with the ability to sync all types of document files with the file size more than X MB that were modification during the last month. Multiple different types of file matching rules may be used to precisely select files to be synchronized.

In order to add one or more file matching rules to a secure file sync command, open the command options dialog, select the 'Rules' tab, press the 'Add' button, select a rule type and enter all the required parameters. During the file synchronization process, DiskBoss will select files using the specified file matching rules and synchronize files matching the rules.

Secure File Synchronization Negative Rules

In addition to positive file matching rules allowing one to synchronize specific types of files, the user is provided with negative file matching rules capable of excluding files by the file type, size, last modification date, etc. For example, in order to exclude all types of images from the file synchronization process, add a file matching rule to process files 'Not Categorized as Images' to the secure file synchronization command.

Secure File Synchronization Exclude Directories

Sometimes, it may be required to exclude one or more subdirectories from a secure file sync operation. In order to exclude one or more directories, open the secure file synchronization command dialog, select the 'Exclude' tab and add directories that should be excluded from the secure file synchronization process.

In addition to the ability to exclude individual directories, DiskBoss provides a number of exclude directories macro commands allowing one to exclude multiple directories using a single macro command.

$BEGINS <Text String> - excludes directories beginning with the specified string.
$CONTAINS <Text String> - excludes directories containing the specified string.
$ENDS <Text String> - excludes all directories ending with the specified string.
$REGEX <Regular Expression> - excludes directories matching the specified regular expression.

For example, the exclude macro command '$CONTAINS Temporary Files' will exclude all directories with 'Temporary Files' at any place in the full directory path and the exclude macro command '$REGEX \.(TMP|TEMP)$' will exclude directories ending with '.TMP' or '.TEMP'.

Secure File Synchronization Command Line Utility

In addition to the DiskBoss GUI application, DiskBoss Ultimate and DiskBoss Server provide a command line utility allowing one to perform secure file synchronization operations from a source server with DiskBoss Ultimate or DiskBoss Server to a destination DiskBoss Server. The DiskBoss command line utility is located in the '<Install Dir>\bin' directory.

Command Line Syntax:

diskboss -ssync -source <Source Directory> -host <Host Name> -dest <Directory>

Executes a secure file sync operation with the specified command line parameters.

diskboss -execute <Preconfigured Secure File Sync Command>

Executes a preconfigured secure file synchronization command.

Required Parameters:

-source <Source Directory> -host <Destination Host> -dest <Destination Directory>

A source directory, the destination server host name or an IP address and a destination directory should be specified. In order to ensure proper parsing of command line arguments, directories containing space characters should be double quoted.

Optional Parameters:

-compress

This option enables compression of transferred data blocks.

-encrypt

This option enables encryption of transferred data blocks using the AES-256 encryption algorithm with a dynamically generated random encryption key for each transferred file.

-verify

This option enables verification of all transferred data blocks using the SHA-256 data signature algorithm and retransmission of corrupted data blocks.

-streams <Parallel File Sync Streams>

This parameter specifies the number of simultaneous file sync streams.

-time

This parameter enables synchronization of the creation, last modification and last access dates for all copied files. If this parameter is not specified, all files copied to the destination directory will have timestamps set to the time of the sync operation.

-port <Port Number>

This parameter specifies the destination DiskBoss Server TCP/IP port number to connect to. If this parameter is not specified, DiskBoss will try to connect to the default TCP/IP port 8094. The port should be open in the destination server's firewall.

-user <DiskBoss Server User Name>

This parameter specifies the user name to login to the destination DiskBoss Server. If this parameter is not specified, DiskBoss will use the default (admin) user name.

-password <DiskBoss Server Password>

This parameter specifies the password to login to the destination DiskBoss Server. If this parameter is not specified, DiskBoss will use the default (admin) password.

-v

This command shows the product's major version, minor version, revision and build date.

-help

This command shows the command line usage information.