SFTP backup not completing

Hi @mustbeart,

I’m sorry to hear the SFTP uploads are not working for you, but I’m sure we can figure this out together.

1. Were the SFTP uploads every working, or are you getting this setup for the first time?

2. When uploading via SFTP, are you manually clicking the “Upload” button from the backup page, or are you uploading via SFTP via the scheduled backups?

3. To confirm…

But the created file is not a valid zip file and it is much too short (150K instead of 150M). The Total Upkeep panel says that the SFTP backup was uploaded.

On the original server, the backup shows as 150MB, but when uploaded via SFTP, only 150K is transferred?

On the original server, you can view the backup and browse the files via the archive browser?

Thanks,
– Brad

1. This is a first-time setup. I haven’t seen an SFTP backup work.

2. I tried both ways with the same results.

3. Yes, the backup shows as 150MB on the server but only 150K-ish is transferred to the SFTP host.

Yes, I can view the backup and browse the files on the server.

-Paul

Hi Paul,

I’ve added some sftp logging to Total Upkeep to help us determine what may be happening. I created a 3 minute video showing you how to grab the latest code and help us test. Please take a look here – https://youtu.be/gwsoYLv6Q2k

Thanks,
– Brad

OK, I have followed the instructions in the video. One glitch with the instructions is that the “Upload” button does not exist for backups in the archive that were already configured for SFTP. The system thinks it has already been uploaded successfully, and just says “Uploaded” with a check mark in front. (There probably ought to be a re-upload button in case the user has somehow lost the uploaded file.) No big problem for this experiment, though. I just chose a backup that was made before I configured SFTP.

Where the “Upload” button was, it now says “Unable to upload backup file. Unable to upload file.” with an X in front of it. And oops, that status does not stick if I leave the archive details screen and come back later. Now it says “Uploaded” with a checkmark, like the others.

Here is the ftp log file. I have redacted my username and site name.

[2020-08-24 19:43:31 UTC] PHP Version: 7.3.15
[2020-08-24 19:43:31 UTC] WordPress Version: 5.5
[2020-08-24 19:43:31 UTC] Total Upkeep version: 1.14.3
[2020-08-24 19:43:31 UTC] ——————————————————————————–
[2020-08-24 19:43:31 UTC] Beginning Boldgrid_Backup_Admin_Ftp::upload…
[2020-08-24 19:43:31 UTC] Local path: /home/XXXXXXXX/boldgrid_backup/boldgrid-backup-www.XXXXXXXX.XXX.dream_.website-9c4fcad2-20200823-033003.zip / 156589186
[2020-08-24 19:43:31 UTC] Remote path: boldgrid-backup-XXXXXXXX-WordPress/boldgrid-backup-www.XXXXXXXX.XXX.dream_.website-9c4fcad2-20200823-033003.zip
[2020-08-24 19:43:31 UTC] Connecting…
[2020-08-24 19:43:31 UTC] Logging in…
[2020-08-24 19:43:33 UTC] Beginning upload…
[2020-08-24 19:43:59 UTC] Last error: Array
(
[type] => 8
[message] => fputs(): send of 224 bytes failed with errno=32 Broken pipe
[file] => /home/XXXXXXXX/XXXXXXXX.XXX/wp-content/plugins/boldgrid-backup/vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php
[line] => 3965
)

[2020-08-24 19:43:59 UTC] Upload status:

==========
That’s it, the log file ends with “Upload status: ”

The resulting file was 163695 bytes, the exact same size as two of the others. (I’ve also seen 196434 bytes.)

-Paul

• This reply was modified 4 years, 3 months ago by mustbeart.

Hi Paul,

Thanks so much for all your help. It looks like we’ve got our culprit:

fputs(): send of 224 bytes failed with errno=32 Broken pipe in phpseclib/phpseclib/phpseclib/Net/SSH2.php

1. The sftp server that you’re using, have you successfully sftp’d large files to it before?

2. I’m going to reach out to DreamHost support to see if they have any suggestions. We have three players involved, (1) your DreamHost server, (2) Total Upkeep, and (3) your sftp server, and I’m hoping to run some tests to help further narrow down the issue. I’ll get back to you soon.

Thanks again Paul,
– Brad

Hi Paul,

I heard back from DreamHost on this, and they offered to look into this a little further on their end too. Can you email [email protected] and reference this post so I can get us all in touch?

Thanks,
– Brad

Hi Paul,

It’s been a few days, I just wanted to touch base and see how everything was going. Were you able to get in touch with DreamHost support?

– Brad

I haven’t tried to contact the person at DreamHost support yet. Been busy. Also, I’m in California so his working hours are not very convenient for me.

I wanted to accurately answer your question about previous success using SFTP. The answer seems to be that SFTP to my host works fine from most anywhere except DreamHost. It works from inside my LAN. It works from an Azure VM. But from DreamHost, a command-line SFTP fails the same way every time: the transfer starts, moves a small amount of data, and then stalls, eventually failing with a broken pipe error.

So this would appear to be entirely a DreamHost issue, and probably not a BoldGrid issue.

I probably should have guessed as much. Looking back at my emails with DreamHost, I see that over a year ago I reported a similar problem with SFTP *to* DreamHost, and it was never resolved. I can try again.

-Paul

Update. I spent a long while on chat with a DreamHost tech support person working in my timezone, and they were unable to solve the problem while I waited. Eventually they decided to let me go and email me the results of further testing. By that time they just wanted to blame it on my ISP. I can’t prove that’s wrong, but I don’t believe it’s right. I don’t think the tech I worked with was a networking expert.

One surprising new observation from the tech support session: SFTP initiated from home to DreamHost does work from Windows 8 using psftp (comes with PuTTY) OR FileZilla, even though it fails (stalls out during the transfer) for the traditional sftp client on Linux or macOS. Some difference in the protocol implementation? I don’t know.

At this point my plan is to take evasive action. I have plenty of room on my paid Dropbox account, and I’ve had good luck with <other plugin, redacted> which supports backup to Dropbox, so I’m going to use that for backup. Maybe consider this a vote for adding Dropbox support!

Hi @mustbeart, I’m sorry we weren’t able to get this one resolved. We’ve thought about adding Dropbox support in the past, and your vote here may push us to get it done. Even though you’ve gone with <other plugin, redacted> I still want to take a moment to say thanks for keeping me updated. You could have ghosted this thread all together, but instead you responded with some valuable info for us, and I appreciate it.

I wish we could have helped, but glad to hear you’ve got another solution, good luck Paul!
– Brad