LFTP多线程多(单)文件下载
由于mget命令下载速度太慢，就写了个脚本，可以用来进行多线程多(单)文件同步下载，速度比原来得快不少。供大家参考，欢迎转帖。#!/bin/bash
# PARM1: SOURCE DIRECTORY ON REMOTE HOST
# PARM2: HOST ADDRESS
# PARM3: DESTINATION PATH TO SAVE DOWNLOADED FILES
# PARM4: DYNAMICALLY CREATED SHELL FILE TO RETRIVE REMOTE FILES WITH "PGET" COMMAND
REMOTEPATH=$1
HOSTADDR=$2
DESTPATH=$3
DYNAFILE=$4
# test if REMOTEPATH parameter is specified
if [ "$REMOTEPATH" = "" ]; then
echo "Parameter: REMOTEPATH was not specified."
exit 1
fi
# test if HOSTADDR parameter is specified
if [ "$HOSTADDR" = "" ]; then
HOSTADDR=ftp.abc.com
echo "Parameter: HOSTADDR was not specified, use default hostname: $HOSTADDR."
fi
# test if REMOTEPATH parameter is specified
if [ "$DESTPATH" = "" ]; then
echo "Parameter: DESTPATH was not specified."
exit 1
fi
# make sure $REMOTEPATH $DESTPATH do not end with '/'
#
DESTPATH=`echo $DESTPATH | sed 's/\/$//g'`
REMOTEPATH=`echo $REMOTEPATH | sed 's/\/$//g'`
# test if destination path exists
if [ ! -d $DESTPATH ]; then
# if Destination Path: $DESTPATH does not exist, it will be created automatically.
mkdir $DESTPATH
if [ `echo $?` = "1" ]; then
echo "Cannot create directory: $DESTPATH."
exit 1
fi
fi
# test if destination path is writable
if [ ! -w $DESTPATH ]; then
echo "Destination Path: $DESTPATH is not writable."
exit 1
fi
tracelog=/tmp/trace.log.$$
USER=`whoami`
THIS_HOST=`hostname -s`
LOGDATE=`date +%d/%m/%Y`
# use traceroute to test connectivity
traceroute $HOSTADDR > $tracelog 2>&1
grep "unkown host" $tracelog >/dev/null 2>&1
if [ `echo $?` = "0" ]; then
echo "Host: $HOSTADDR is not reachable."
exit 1
fi
# retrieve file list
lftp $HOSTADDR<<FTPCMD
cd $REMOTEPATH
lcd $DESTPATH
nlist > $DYNAFILE
quit
FTPCMD
# multi file multi connections downloading, uncomment next line
# sed 's/^./pget &/g' $DYNAFILE | sed 's/$/& \&/g' > $DYNAFILE
#single file multi connections downloading, comment next line for multi file multi connections downloading
sed 's/^./pget &/g' $DYNAFILE | cat > $DYNAFILE
echo $DESTPATH | sed 's/^./lcd &/g' > $DYNAFILE.tmp
cat $DYNAFILE.tmp $DYNAFILE | cat > $DYNAFILE
echo $REMOTEPATH | sed 's/^./cd &/g' > $DYNAFILE.tmp
cat $DYNAFILE.tmp $DYNAFILE | cat > $DYNAFILE
echo $HOSTADDR | sed 's/^./lftp &/g' | sed 's/$/&<<FTPCMD/g' > $DYNAFILE.tmp
cat $DYNAFILE.tmp $DYNAFILE | cat > $DYNAFILE
echo '#!/bin/bash' > $DYNAFILE.tmp
cat $DYNAFILE.tmp $DYNAFILE | cat > $DYNAFILE
awk '{print $0} END { print "wait all"} ' $DYNAFILE | cat > $DYNAFILE
awk '{print $0} END { print "quit"} ' $DYNAFILE | cat > $DYNAFILE
awk '{print $0} END { print "FTPCMD"} ' $DYNAFILE | cat > $DYNAFILE
# remove temporary file
rm -f $DYNAFILE.tmp
# change file prvilage and excute the dynamically created shell file
chmod u+x $DYNAFILE
. $DYNAFILE
# rm -f $DYNAFILE
MichaelBibby编辑，以code格式格式化代码
----MichaelBibby 11/07/04 矣，lftp不是可以多线程下吗?
pget -n100 http://***************** 不错的脚本，加精鼓励 :mrgreen:
PS：liuzj兄，我编辑了你的帖子，以code格式格式化脚本，这样代码看起来很整洁8) 主要是用在自动下载目录下的文件，定制化的下载文件。
谢谢加的编辑。 嗯，有图形界面就好了
页:
[1]