Issue backing up to S3

  • Resolved beercity

    (@beercity)


    8 years, 1 month ago

    I set up BackUpWP for a side job, and it seemed to work until the customer started uploading images to the site. Now the backups seem to be completing based on the fact that I see them in S3, but I get a failure error log. I am not able to compare the generated backup file from the “Backups” tab in the plugin, as it seems to not list the failed backups so I can download the backup file on the webserver. However the backup file is in S3 and the file size seems to be correct. Additionally, since the backup job is not completing successfully as far as the plugin is concerned, it is not purging old backups from the S3 bucket.

    When the backups were around 45MB everything was fine. The backups are now around 200MB, and fail after about 900 seconds.

    Nothing has changed in regards to the IAM policies for access to S3. Since the jobs get uploaded, I believe the permissions are good.

    Thanks in advance for any help you can provide.

    Here is the error log for a failed backup, sanitized as needed:

    [INFO] BackWPup 3.3.4; A project of Inpsyde GmbH
    [INFO] WordPress 4.6.1 on https://site-name.org/
    [INFO] Log Level: Normal
    [INFO] BackWPup job: Daily Backup
    [INFO] Logfile is: backwpup_log_2016-09-30_08-15-49.html
    [INFO] Backup file is: backwpup_2016-09-30_08-15-50.zip
    [30-Sep-2016 08:15:50] 1. Try to backup database …
    [30-Sep-2016 08:15:50] Connected to database database on localhost
    [30-Sep-2016 08:15:50] Added database dump “database.sql.gz” with 229.56 KB to backup file list
    [30-Sep-2016 08:15:50] Database backup done!
    [30-Sep-2016 08:15:50] 1. Trying to make a list of folders to back up …
    [30-Sep-2016 08:15:50] Added “wp-config.php” to backup file list
    [30-Sep-2016 08:15:50] 511 folders to backup.
    [30-Sep-2016 08:15:50] 1. Trying to generate a file with installed plugin names …
    [30-Sep-2016 08:15:50] Added plugin list file “site-name.pluginlist.2016-09-30.txt.gz” with 937.00 B to backup file list.
    [30-Sep-2016 08:15:50] 1. Trying to generate a manifest file …
    [30-Sep-2016 08:15:50] Added manifest.json file with 4.97 KB to backup file list.
    [30-Sep-2016 08:15:50] 1. Trying to create backup archive …
    [30-Sep-2016 08:15:50] Compressing files as ZipArchive. Please be patient, this may take a moment.
    [30-Sep-2016 08:29:40] Backup archive created.
    [30-Sep-2016 08:29:40] Archive size is 202.69 MB.
    [30-Sep-2016 08:29:40] 4103 Files with 236.90 MB in Archive.
    [30-Sep-2016 08:29:40] 1. Trying to send backup file to S3 Service …
    [30-Sep-2016 08:29:41] Connected to S3 Bucket “bucket-name” in
    [30-Sep-2016 08:29:41] Checking for not aborted multipart Uploads …
    [30-Sep-2016 08:29:41] Starting upload to S3 Service …
    [30-Sep-2016 08:30:34] Backup transferred to https://s3.amazonaws.com/bucket-name/daily/backwpup_2016-09-30_08-15-50.zip.
    [30-Sep-2016 08:30:34] ERROR: S3 Service API: Access Denied
    [30-Sep-2016 08:30:34] 2. Trying to send backup file to S3 Service …
    [30-Sep-2016 08:30:34] ERROR: S3 Service API: The specified upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed.
    [30-Sep-2016 08:30:34] 3. Trying to send backup file to S3 Service …
    [30-Sep-2016 08:30:35] Connected to S3 Bucket “bucket-name” in
    [30-Sep-2016 08:30:35] Checking for not aborted multipart Uploads …
    [30-Sep-2016 08:30:35] Starting upload to S3 Service …
    [30-Sep-2016 08:31:04] Backup transferred to https://s3.amazonaws.com/bucket-name/daily/backwpup_2016-09-30_08-15-50.zip.
    [30-Sep-2016 08:31:05] ERROR: S3 Service API: Access Denied
    [30-Sep-2016 08:31:05] 4. Trying to send backup file to S3 Service …
    [30-Sep-2016 08:31:05] ERROR: S3 Service API: The specified upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed.
    [30-Sep-2016 08:31:05] ERROR: Step aborted: too many attempts!
    [30-Sep-2016 08:31:05] ERROR: Job has ended with errors in 916 seconds. You must resolve the errors for correct execution.

    • This topic was modified 8 years, 1 month ago by beercity.
    • This topic was modified 8 years, 1 month ago by beercity.
    • This topic was modified 8 years, 1 month ago by beercity.
    • This topic was modified 8 years, 1 month ago by beercity.
Viewing 4 replies - 1 through 4 (of 4 total)
  • Thread Starter beercity

    (@beercity)

    For reference here is the IAM profile I am using for access for this plugin

    {
    “Version”: “2012-10-17”,
    “Statement”: [
    {
    “Sid”: “Stmt1457967512000”,
    “Effect”: “Allow”,
    “Action”: [
    “s3:ListAllMyBuckets”
    ],
    “Resource”: [
    “*”
    ]
    },
    {
    “Effect”: “Allow”,
    “Action”: [
    “s3:GetBucketLocation”,
    “s3:ListBucketMultipartUploads”,
    “s3:ListBucket”
    ],
    “Resource”: [
    “arn:aws:s3:::rfl-backups”
    ]
    },
    {
    “Sid”: “Stmt1457967512001”,
    “Effect”: “Allow”,
    “Action”: [
    “s3:PutObject”,
    “s3:ListMultipartUploadParts”,
    “s3:Get*”,
    “s3:AbortMultipartUpload”
    ],
    “Resource”: [
    “arn:aws:s3:::rfl-backups/Ready-For-Life*”,
    “arn:aws:s3:::rfl-backups/Ready-For-Life/*”
    ]
    }
    ]
    }

    Thread Starter beercity

    (@beercity)

    I fixed this by fixing my permissions. I realized it was not deleting the old backups, the failures were likely because it did not have permissions to list the contents or delete items. I updated the permissions and all is good now.

    Thanks for posting your solution. I had tried to give the s3 user only permissions it should need for backwpup and had similar issues. It would be nice if they provided a recommended IAM policy.

    Thread Starter beercity

    (@beercity)

    I used these permissions. Might give too much access but I am not too concerned about it.

    {
    “Version”: “2008-10-17”,
    “Statement”: [
    {
    “Sid”: “”,
    “Effect”: “Allow”,
    “Principal”: {
    “AWS”: “arn:aws:iam::YOUR_IAM_USER_ID:user/YOUR_IAM_USER_NAME”
    },
    “Action”: [
    “s3:AbortMultipartUpload”,
    “s3:GetObjectAcl”,
    “s3:ListBucket”,
    “s3:DeleteObject”,
    “s3:GetObject”,
    “s3:PutObjectAcl”,
    “s3:ListBucketMultipartUploads”,
    “s3:GetBucketLocation”,
    “s3:PutObject”
    ],
    “Resource”: [
    “arn:aws:s3:::YOUR_BUCKET_NAME”,
    “arn:aws:s3:::YOUR_BUCKET_NAME/*”
    ]
    }
    ]
    }

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘Issue backing up to S3’ is closed to new replies.