Speed up rsync with Simultaneous/Concurrent File Transfers?

 


Updated answer (Jan 2020)

xargs is now the recommended tool to achieve parallel execution. It's pre-installed almost everywhere. For running multiple rsync tasks the command would be:

ls /srv/mail | xargs -n1 -P4 -I% rsync -Pa % myserver.com:/srv/mail/

This will list all folders in /srv/mail, pipe them to xargs, which will read them one-by-one and and run 4 rsync processes at a time. The % char replaces the input argument for each command call.

Original answer using parallel:

ls /srv/mail | parallel -v -j8 rsync -raz --progress {} myserver.com:/srv/mail/{}




apt-get install moreutils to install parallel


apt-get install parallel

Comments

Post a Comment

Popular posts from this blog

cpanel exam CPSP Answers

 Which of the following options best describes the role of Shared Hosting on a web server? Providing hosting for a single website owned by different people on multiple servers. Providing hosting for a single website owned by one person on one server. Providing hosting for multiple websites owned by different people on one server. Providing hosting for multiple websites owned by one person on one server. 3 Which of the following capabilities can a cPanel account user perform easily from within the cPanel account interface without the aid of a system administrator? File and configuration management. Relocating the physical server. Upgrading server hardware and equipment. Installing new database software. 1 File and configuration management. Which of the following options best describes a core benefit to using cPanel & WHM, as a web hosting provider operating on a cPanel & WHM environment? cPanel users are able to create new virtualization resources to help host their web appl...
Read more

How to install zimbra collaboration suite 8.8.11 on CentOS 7

Check system requirements  https://wiki.zimbra.com/wiki/Zimbra_Releases/8.7.0/Single_Server_Installation#System_Requirements prerequisites Before we get started, here are a few things that are required before we proceed with the installation. A clean installation of Centos 7 A Fully Qualified Domain (FQDN) for your server(Example mail.geekybum.com) An external DNS server with both A and MX records for your server to point to your Zimbra mail server IP Address A static IP Address assigned to network interface. Step 1: Root login to server through SSH Step 2: Install required packages for zimbra coloboration suite installation yum -y install unzip net-tools sysstat openssh-clients perl-core libaio nmap-ncat libstdc++.so Initial server setup To get started we need to configure a hostname for our server and a static IP address. vi /etc/hosts 135.125.30.56 mail.geekybum.com mail Replace the system hostname and FQDN values accordingly in order to match your ow...
Read more

awstats installation

Set Up AWStats for Nginx on Ubuntu sudo apt-get install awstats add access log file to generate stats using this server { # ... all your other config ... access_log /var/log/nginx/yourdomain.com.access.log ; # ... all your other config ... } /etc/nginx/sites-available vi awstats.parleproducts.com.conf cd /etc/awstats/ vi awstats.parleproducts.com.conf #path to your nginx log file LogFile="/var/log/nginx/parleproducts.com.access.log" # Domain of your vhost SiteDomain="parleproducts.com" # Directory where to store the awstats data DirData="/var/lib/awstats/" # Other domains/subdomain you want included from your logs, for example the www subdomain HostAliases="www.parleproducts.com" # If you customized your log format above add this line: LogFormat = ""%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"" # If you did not, uncomment and use this line: #...
Read more