Unable to download huge files like 4-5 GB?

[ant]

Hello!

I am having problems downloading big files like image ISO. sz <filename> fails without any errors. See below:

$ ls -all *.iso
-rwx------ 1 foo users 4303790080 Apr 1 07:49 image.iso

$ sz image.iso
$

As you can see, nothing happened. No error either. I tried strace with sz:

$ strace sz image.iso
execve("/usr/bin/sz", ["sz", "image.iso"], [/* 17 vars */]) = 0
uname({sys="Linux", node="mybox", ...}) = 0
brk(0) = 0x10021000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x30019000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=59873, ...}) = 0
mmap(NULL, 59873, PROT_READ, MAP_PRIVATE, 3, 0) = 0x3002a000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/tls/libnsl.so.1", O_RDONLY) = 3
read(3, "\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\24\0\0\0\1\0\0>@"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=85688, ...}) = 0
mmap(0xffc9000, 159360, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xffc9000
mprotect(0xffdd000, 77440, PROT_NONE) = 0
mmap(0xffec000, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13000) = 0xffec000
mmap(0xffee000, 7808, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffee000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/tls/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\24\0\0\0\1\0\1\314"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1390284, ...}) = 0
mmap(0xfe54000, 1461132, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfe54000
mprotect(0xff9d000, 113548, PROT_NONE) = 0
mmap(0xffac000, 45056, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x148000) = 0xffac000
mmap(0xffb7000, 7052, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffb7000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3001a000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x3001b000
mprotect(0xffac000, 24576, PROT_READ) = 0
mprotect(0xffec000, 4096, PROT_READ) = 0
mprotect(0x30028000, 4096, PROT_READ) = 0
munmap(0x3002a000, 59873) = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
fstat64(2, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
brk(0) = 0x10021000
brk(0x10042000) = 0x10042000
getuid() = 1005
geteuid() = 1005
rt_sigaction(SIGALRM, NULL, {SIG_DFL}, 8) = 0
rt_sigaction(SIGALRM, {SIG_DFL}, NULL, 8) = 0
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, TCSETSW, {B38400 -opost -isig -icanon -echo ...}) = 0
ioctl(0, TCGETS, {B38400 -opost -isig -icanon -echo ...}) = 0
rt_sigaction(SIGINT, {0x100016f0, [INT], SA_RESTART}, {SIG_DFL}, 8) = 0
rt_sigaction(SIGINT, {0x100016f0, [INT], SA_RESTART}, {0x100016f0, [INT], SA_RESTART}, 8) = 0
rt_sigaction(SIGTERM, {0x100016f0, [TERM], SA_RESTART}, {SIG_DFL}, 8) = 0
rt_sigaction(SIGPIPE, {0x100016f0, [PIPE], SA_RESTART}, {SIG_DFL}, 8) = 0
rt_sigaction(SIGHUP, {0x100016f0, [HUP], SA_RESTART}, {SIG_DFL}, 8) = 0
) = 3(1, "rz\r", 3rz
access("image.iso", R_OK) = 0
stat64("image.iso", {st_mode=S_IFREG|0700, st_size=4303790080, ...}) = 0
ioctl(0, TCFLSH, 0) = 0
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
ioctl(0, TCFLSH, 0) = 0
write(1, "**\30B00000000000000\r\212\21) = 20
write(2, "\r\n", 2
) = 2
write(2, "Can\'t open any requested files.", 31Can't open any requested files.) = 31
write(2, "\r\n", 2
) = 2
ioctl(1, TCFLSH, 0) = 0
write(1, "\30\30\30\30\30\30\30\30\30\30\10\10\10\10\10\10\10\10"..., 20) = 20
ioctl(0, TCSBRK, 0x1) = 0
ioctl(0, TCFLSH, 0x2) = 0
ioctl(0, TCGETS, {B38400 -opost -isig -icanon -echo ...}) = 0
ioctl(0, TCSETSW, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, TCXONC, TCOON) = 0
exit_group(128) = ?
$ **01000000039a32
-bash: **01000000039a32: command not found


Please note that I am still using SecureCRT v3.4.8 in Windows XP Pro. SP2 (all updates). I have no problems downloading smaller files. Does anyone know what's going on? Will the latest SecureCRT version fix this?

Are other SSH clients (Windows XP or Linux) that will let me download big files and be able to resume (scp can't. sz -r can). I cannot download nonstop since I need to take breaks like not download during the day time hours.
Thank you in advance.

[jjh]

Hello ant,

Sometimes a ZModem transfer will stop if the file being
transferred includes control characters. You can usually
use a switch to escape the control characters that appear in
the file being transferred.

Do you get better behavior if you use "sz -e" (or whatever
your sz client uses to escape control characters)?

If that doesn't work, does upgrading to the latest version
of SecureCRT help?

Thank you

JJH

[ant]

Originally Posted by jjh

Hello ant,

Sometimes a ZModem transfer will stop if the file being
transferred includes control characters. You can usually
use a switch to escape the control characters that appear in
the file being transferred.

Do you get better behavior if you use "sz -e" (or whatever
your sz client uses to escape control characters)?

If that doesn't work, does upgrading to the latest version
of SecureCRT help?

Thank you

JJH

This is on another Windows XP Pro. SP2 machine (work) and a different network.

$ sz -e image.iso

Can't open any requested files.
$ **01000000039a32
-bash: **01000000039a32: command not found

SecureCRT showed gave me an error: ZModem transfer canceled by remote side. I also notice file never gets written on the HDD.


I guess I will have to try the trialware of SecureCRT which I would like to avoid buying since I don't need the new features and enhancements. I only use basic telnet, SSH, and sz. Hence, why I wished PuTTY had zmodem transfer support.

[ant]

Originally Posted by jjh

Hello ant,

Sometimes a ZModem transfer will stop if the file being
transferred includes control characters. You can usually
use a switch to escape the control characters that appear in
the file being transferred.

Do you get better behavior if you use "sz -e" (or whatever
your sz client uses to escape control characters)?

If that doesn't work, does upgrading to the latest version
of SecureCRT help?

Thank you

JJH

JJH, I just tried SecureCRT v5.1 on a dummy test machine with XP Pro. SP0 and same problem. It didn';t even show me any errors with sz -e. I also checked /var/log/messages and it didn't mention any errors on sz with the big files. I know the log gets logged for sz transfers.

Here's what is weird. I tried another big file that was 4 GB, and that worked. Maybe it is not restricted to file sizes.

[jjh]

Hello ant.

Can you tell us what implementation of sz you are using and
which version of Linux?

Thanks

JJH

[ant]

Originally Posted by jjh

Hello ant.

Can you tell us what implementation of sz you are using and
which version of Linux?

Thanks

JJH

I assume how sz was installed for implementation. I believe the administrator installed it with apt-get command.

$ sz --version
sz (lrzsz) 0.12.21rc

$ uname -a
Linux mybox 2.6.16-1-powerpc #2 Thu Mar 23 07:02:15 UTC 2006 ppc GNU/Linux

[jjh]

Hi ant.

Thank you for providing that information.

We have done some testing and were able to reproduce the
problem, so I will create an entry for you in our bug
tracking database. 4GB does seem to be somewhat of a
boundary with some ZModem implementations. We need to take
a look at our implementation to see if there is a
work-around.

One thing you might consider is that if you are using the
SSH2 protocol we can offer a way to automate SFTP transfers
with SecureCRT.

JJH

[ant]

Originally Posted by jjh

Hi ant.

Thank you for providing that information.

We have done some testing and were able to reproduce the
problem, so I will create an entry for you in our bug
tracking database. 4GB does seem to be somewhat of a
boundary with some ZModem implementations. We need to take
a look at our implementation to see if there is a
work-around.

One thing you might consider is that if you are using the
SSH2 protocol we can offer a way to automate SFTP transfers
with SecureCRT.

JJH

I am indeed using SSH2. Does SFTP support resume file transfers? The reason why I use zmodem is because I can resume with sz -r. I can't download big files in one session due to disconnections, speed, etc. I know SCP doesn't support resume feature.

[jjh]

Hi Ant.

Actually the SFTP functionality in SecureCRT won't resume
transfers, so I have created an entry in our feature request
database for the resume. We will contact you should that
become available.

Our SecureFX SFTP client does have resume capability.

JJH

[ant]

Originally Posted by jjh

Hi Ant.

Actually the SFTP functionality in SecureCRT won't resume
transfers, so I have created an entry in our feature request
database for the resume. We will contact you should that
become available.

Our SecureFX SFTP client does have resume capability.

Sounds good to me. Can SecureFX SFTP client support huge files like 5 GB sizes?

[jjh]

Hello ant.

Yes, you can transfer very large files like that with
SecureFX.

SecureFX is available for download from our website as a
free 30 day trial, so I encourage you to try it out first.

Thanks

JJH

[ant]

Originally Posted by jjh

Hello ant.

Yes, you can transfer very large files like that with
SecureFX.

SecureFX is available for download from our website as a
free 30 day trial, so I encourage you to try it out first.

Thanks

JJH

Thanks, JJH!

[jjh]

Hi ant.

No problem. If you have any other questions, pleae feel
free to contact us on the forums or by sending e-mail to
support@vandyke.com

JJH

[jjh]

Hello ant.

I wanted to follow up with you about this. Our developers
were able to reproduce the problem you were experiencing
with ZModem and it appears to be related to a problem with
the ZModem installation on the server. If ZModem on the
server supported large file transfers, then SecureCRT
wouldn't have a problem transferring the large (greater than
4GB) files.

JJH