I tried opening an S3 connection in Cyberduck with the access key and secret key I was given, but I can’t get to the bucket. I have its name, though I’m not sure whether that belongs in the server field or somewhere else. Where should I enter it?
That still leaves two separate things for me to check: whether I can access the S3 files, and whether the way I access them suits repeated work. I prefer the CloudMounter option for regular use because it mounts cloud storage as a drive on my computer. I would keep that preference separate from any conclusion about why a connection is failing.
Define what needs checking
I would start by defining the criteria for establishing the necessary access. For the access check, I want to connect, browse the buckets accessible to the account, and upload/downloading files. For the ongoing work, I also want to know whether I want to open the transfer application every time.
Those are linked criteria, but I would not consider them interchangeable. Mounting the drive is my preferable way to access files on S3 frequently. I would not confuse that detail with the initial access diagnostic, as it does not tell me if the credentials or bucket permission are correctly set.
Review the access details
In the case of a connection failure, my preliminary review would include the credentials, bucket permissions, and AWS region. I would also check whether the IAM user or role actually has access to the bucket I am trying to use.
I would keep those items separate rather than describe the problem as “S3 does not work.” The access key and secret access key are one part of the setup. The permission to access the intended bucket is another. The region also belongs on the checklist.
I would not read into the specific cause from the failure. Those are the checks that I would make for first, not the findings about what is wrong.
Make the connection attempt
For the Cyberduck connection, I would select Amazon S3 as the connection type and enter the AWS access key and secret access key.
Once connected, I should be able to browse the buckets to which the account has access permission. I would use this permission boundary as a criterion for checking the result rather than the presence of all buckets. Uploading and downloading files from there will be the next step of the verification process.
I would keep the result of the overall test in a simple form – whether the connection was established, whether the desired bucket was opened, and if I could upload/download the required file.
Keep the workflow decision separate
The returning to the everyday workflow should be another step in the process. Opening the Cyberduck application for individual transfers may be a way for some, but for me, the most convenient is to have the cloud storage mounted as a drive for frequent S3 operations.
I would define it as a preference rather than a connection repair or a requirement of one application over another. For those who still have problems, what step fails – connecting, opening the desired bucket, or uploading/downloading the file?
12 Likes
The ability to list the contents of a bucket does not necessarily give access to the account’s bucket list itself, so it’s not a solid proof of access. For your Cyberduck S3 bookmark, leave Server set to its default value and make sure to specify the desired bucket name in Default Path. I personally would go with CloudMounter, but there’s no need for switching tools if you don’t want to.
Do not take the two keys you have been given as the full set of keys to use for a login. If the two keys are the temporary AWS credentials, you will also need a session token to log in since these credentials are only temporary and will expire. Before you spend time updating your connection settings, ask the person giving them to you whether they are temporary or not.
In the case of the bucket location, it would be better to use a Cyberduck bookmark rather than just the Open Connection window. This is because bookmarks give you more options to update such as the Path. Choose Amazon S3, type in your access key and secret key, leave the Server option as default, and for the Default Path, type in the name of the bucket you want to connect to. It is important to type only the name of the bucket to connect to rather than a whole S3 address since Cyberduck can identify the region of the bucket on its own.
I disagree with @oleg74’s suggestion that browsing the account’s buckets should be part of the initial access test. You can have access to the named bucket without permission to retrieve the account’s bucket list. Open the bookmark directly instead of navigating down from the account root. Otherwise, you may be testing a permission you were never supposed to have.
If the credentials are temporary, Cyberduck supports a ‘S3 (Credentials from AWS Command Line Interface)’ connection profile, which reads AWS credentials profile with access key, secret key, and session token (don’t put the token in the password field along with the secret).
As for me, I would use CloudMounter, but I would avoid specifying the application for the time being. First, check what type of credential you have and try opening the bookmark for the bucket directly. If there is an error message, please post it here with the credential data replaced by their obfuscated versions (the actual string of the error message itself that you see on your screen, not the screenshot of an input form with the credential fields already filled in).
Access to a particular bucket and access to a certain folder in the bucket are two different things. Your credentials may grant you permission to the second while withholding the capability of opening the bucket as a whole, so it’s worth considering before labeling a failed directory listing attempt as a failed authentication. S3 permissions allow for limiting your ability to list contents of the bucket to a particular folder (which would appear as a folder on your client).
For your specific case, the bucket name goes to Server in the Bookmark field – no, wait, the instructions say the Path field. The Server field should have Amazon S3 selected, with its default value of s3.amazonaws.com, and you should input your Access Key and Secret Key accordingly. The bucket name goes into the Path field, and you go to that bookmark. It’s suggested as a method to connect directly if you are unable to list the account’s buckets.
Where I’d qualify @techwolf9511node’s advice is that the bucket name alone may not be the complete destination. Ask whoever supplied the credentials, “Do these allow browsing the bucket root, or only a specific folder?” If they supplied folder-restricted access, use the bucket plus that folder as the bookmark path, such as /example-bucket/incoming/. The intended folder matters because a prefix-restricted policy can reject a request to list the root.
I would leave the choice between Cyberduck and CloudMounter for that detail to be settled. The useful handoff here is the exact starting folder and what is expected to be done there, rather than a bucket name and a couple of keys.
Don’t use an upload as your first connection test unless you were told these credentials allow writing. I’d change that part of @oleg74’s advice: S3 can permit downloading while denying uploads, so a rejected upload doesn’t necessarily mean your connection settings are wrong.
First, create a Cyberduck bookmark and choose Amazon S3 in the connection-type dropdown. Leave Server at its default, enter the access key and secret key in their respective fields, and expand More Options to set Path to /your-bucket-name/. Open that bookmark directly. To clear up @real_panda’s wording, Amazon S3 is the connection type, not something you type into Server. The bucket belongs in Path for this setup.
Then test the operation that you actually need. If you have been given browsing and download access, open the destination, and download an existing file. Listing, downloading, and uploading require separate permissions, and you should test each one individually, instead of assuming that if one works, the others will, too. There’s no point in asking for write access just to verify that you can download.
Confirm whether this is Amazon-hosted storage or another provider’s S3-compatible service. @rust_kate79’s default-server advice fits Amazon S3, but a different provider may require its own endpoint in Server. The bucket goes in Path for this setup, so ask the person supplying the keys for the endpoint before assuming the default is correct.
Expect this to take multiple iterations with whomever handed you the keys instead of a single attempt. Half the thread is already circling that. It’s true, because you don’t yet know if it’s credentials, region, missing session token, or permission scoped down to one folder. Any of those will yield the same ‘can’t get to the bucket’ feeling from your side.
The practical thing everyone is doing right: Amazon S3 is in the connection type, server is default, keys in their fields, and bucket name in Path. What hasn’t come up is a nasty edge case. If your bucket name contains a dot in it (data.company.example), Cyberduck’s default virtual-hosted addressing can choke on TLS certificate validation and throw an SSL error that doesn’t look anything like a permissions problem. If this is the case, switching the bookmark to path-style access in the connection profile will usually clear it up. Worth checking before assuming the keys are bad.
@voyageurdubois already nailed the temporary-credentials angle, so I won’t repeat it, just second that you check it first because it’s the most common silent failure. For the day-to-day part, I’d mount it with CloudMounter and stop thinking about it, but that’s only worth doing once you’ve confirmed the plain Cyberduck bookmark opens. No point mounting a connection you haven’t proven works yet.