- Saratoga Data Systems, Inc. - Securely and accurately move your distributed data at unprecedented speeds.
Saratoga Data Systems has provided their Flume accelerated file transfer technology on the CenturyLink Cloud platform. The purpose of this KB article is to help the reader take advantage of this integration to achieve rapid time-to-value for this network and security solution.
Saratoga Data Systems helps CenturyLink Cloud customers address the business challenge of secure and reliable transmission of data over challenged networks by providing fast reliable file transfer using the flume client server software - now available as part of the CenturyLink Cloud Blueprint Engine.
Flume Network Optimization is a patented network protocol for transferring data that is unaffected by high latency and very resilient in the face of errors and network interruption.
The 'flume' command line interface program implements this protocol wrapped in robust file management technology.
Data transmission is via a UDP data channel controlled and monitored by a separate TCP channel. By using UDP, 'flume' is able to transmit at full speed without the throttling effect on TCP caused by high latency and the requirement for each TCP data packet to be acknowledged before additional packets can be sent.
To achieve reliability with UDP, the TCP channel is used to indicate the need for replacement of missing or corrupted data. In addition to detecting packets have never arrived, two levels of CRC checksum are used for assurance that the data at the destination exactly matches that at the source.
In addition to the speed advantages of the protocol itself, the 'flume' program can optionally make use of additional technologies to reduce the actual amount of data transmitted:
- Sending only files that have a changed date / time stamp
- Sending only file differences ala rsync using the open source rsync library
- Data compression using the standard open source zlib library
Flume can also optionally encrypt all communications using a selectable encryption technology provided via the open source Botan library.
A unique feature of Flume among file transfer programs is the ability to be used in a UNIX "pipe". This means that source data can come via 'stdin' and/or output can be to 'stdout'. If either of these modes is used at the remote location, Flume allows specification of what program is to run there to either: push data to 'stdin' or pull data from 'stdout'. See the Flume User Manual for more information on this feature.
Saratoga Data is making their technology available for CenturyLink Cloud Users to deploy to their account. Installation of the Flume Blueprint includes a license good for one month (depending on the actual date of deployment). In order to purchase a license or entitlement, please contact email@example.com.
CenturyLink Cloud Users with high latency, challenged links, especially remote locations and via satellite-based connections.
After reading this article, the user should feel comfortable getting started using the partner technology on CenturyLink Cloud.
After executing the steps in this Getting Started document, the users will have a functioning Linux operating system that includes the 'flume' software.
- Access to the CenturyLink Cloud platform as an authorized user.
- Identify a Network VLAN you want the Saratoga Data servers to reside on.
- Possess a Flume license key.
If you want to access your VM over the internet, please perform the following tasks once you receive an email confirming you Blueprint completed successfully:
Allow incoming traffic for desired ports by clicking on the Servers Public IP through the Control Portal. The ports used by 'flume' are controlled by a configuration file /etc/flume/ports but the standard ports to open are:
Install Flume Blueprint
Locate the Flume Blueprint.
- Login to the Control Portal. From the Nav Menu on the left, click Orchestration > Blueprints Library.
- Search for “Flume” in the keyword search on the right side of the page.
- Locate the 'Install Flume File Transfer on Linux' Blueprint.
Choose and Deploy the Blueprint.
- Click the “Install Flume File Transfer on Linux” Blueprint.
Configure the Blueprint using the standard information.
- There is nothing special required.
Review and Confirm the Blueprint.
next: step 2.
Verify your configuration details.
Deploy the Blueprint.
- Once verified, click the
- You will see the deployment details along with an email stating the Blueprint is queued for execution.
- This will kick off the blueprint deploy process and load a page to allow you to track the progress of the deployment.
- Once verified, click the
Monitor the Activity Queue.
- Monitor the Deployment Queue to view the progress of the blueprint.
- To monitor progress, click Queue from the Nav Menu on the left.
- Once the blueprint completes successfully, you will receive an email stating that the blueprint build is complete. Please do not use the new server until you have received this email notification.
Accessing your Saratoga Data Flume Server
Access your Saratoga Data Flume server by connecting to the server via ssh to the public IP using root and the password created when deploying the blueprint.
- The flume data transfer program is invoked using the command line.
- For instruction on using Flume see the User Manual present on the newly created server.
- It is also available at ftp://public.saratogadata.com/pub/sw/flume_2.3/Flume_2.3_User_Manual.pdf
You can also access a simple 'man' page with:
- man flume
Or get integrated help using:
- flume --help
For example to push data to another system (which must also have flume installed:
flume --source_file(s) --remote-host-name-or-ip:/remote/file/system/directory/
The costs associated with this Blueprint deployment are for the CenturyLink Cloud infrastructure only. There are no Saratoga Data Flume license costs or additional fees bundled in.
Frequently Asked Questions
Where do I obtain my Saratoga Data License or entitlements?
Existing CenturyLink Enterprise Customers can contact their Account Representative for help obtaining a Saratoga Data license, or contact Saratoga Data directly:
- via Email: sales@SaratogaData.com
- via Phone: 1-408-898-4307
Who should I contact for support?
- For issues related to deploying the Blueprint, licensing, accessing or using the Saratoga Data application, please visit the Saratoga Data Support Website.
- For issues related to cloud infrastructure (VMs, network, etc.), or if you experience a problem deploying the Blueprint, please open a CenturyLink Cloud Support ticket by emailing firstname.lastname@example.org or through the support website