This post describes how you can limit the bandwidth usage during data transfer using lftp (File transfer program)

For those who don’t know what lftp is, You can check it out over here : http://lftp.yar.ru/lftp-man.html

For any data transfer between local servers (physically located at same location), one should make a practice to check with Network team whether both the server are connected to same switch or they are connected to two different switches. If they are connected between two different switches then make sure to rate limit the data transfer throughput during ftp. This is to ensure that bandwidth is not choked during data transfer and other Server/Applications do not suffer due to data transfer

Mostly every office has 1Gig network, If data transfer is done between servers connected to two different switches with uplink of 1G without bandwidth throttling then you may endup using entire bandwidth available resulting network issues and packet loss for other servers and applications.

To avoid this issue you should throttle bandwidth usage during data transfer. For Eg. You can throttle bandwidth usage to 40 Megabytes/Sec. To apply this limit using lftp follow below mentioned steps.

# vim /etc/lftp.conf

Append a line as below :

set net:limit-total-rate 41943040:0

•  This will limit the data transfer to 40 Megabytes/Sec. Of-course this has to be done before the transfer starts.
• As you notice 41943040:0 . The value before “:” is for download. i.e., GET (Mirror in lftp) for pulling data from other server. If you have to push the data i.e., PUT (Mirror -R in lftp) then change it to 0:41943040
•  If transfer is happening between two servers connected to same switch then no need to rate limit the transfer.

Caution : Simultaneous ftp transfer on same servers should not be done. If you wanna do it anyways then adjust the above mentioned limits accordingly to distribute the load between two connections.