Allow others to upload data to your Amazon S3 bucket

When processing data on the cloud for somebody else (e.g. a customer) or you want to share large amounts of data for other reasons, a good option is to directly use cloud storage for this. Besides DropBox (which requires a contract with a monthly fee for this kind of data) and other offerings, Amazon provides S3: You already get 5 GB of free S3 storage as part of the “Free Tier“, for anything else you only pay for what you use. At the current price(August 2017) you pay $0.023 per GB for standard storage.Additionally you have to pay for access operations: PUT, COPY, POST, or LIST Requests $0.005 per 1,000 requests, GET and all other Requests $0.004 per 10,000 requests. Data transfers between S3 buckets or from S3 to any Amazon cloud service(s) within the same region are free, to other regions or download via the internet incurr additional prices. Please check back on the Amazon pages for curent prices and conditions.

Using the excellent Walk-through example from AWS, here is a condensed version of the set-up that worked for me. I needed to process large data files for a customer, let’s call them DataHeros Ltd. here, downloading to my machine and uploading to the cloud would have been highly inefficient. FTP or other access to the customer data was not possible.

The main steps are:

1. Create IAM profile for the customer
   – Sign in to AWS, create a new IAM user (let’s call him/her “data-heros-ltd”) with password but no permissions yet.
   – Also create a group (called “customers”)
2. Create S3 bucket with folder for the customer
   – Go to the S3 console.
   – Create a bucket “my-company-bucket” and a folder “customer-A”, “customer-B”, etc. within “customer-A” I create a subfolder “DataHeros-Ltd”. All customers will see the initial listing, but they don’t need to see the actual names of other customer. At the same time the customer is sure that the second-level folder is the correct one for his/her data.
   Let’s also create a folder “other-data” for just that, where only I have access to.
3. Set permission to allow access to this area only
   All customers (group) need list acess to the bucket, the current customer DataHeros-Ltd needs write access to the “customer-A / DataHeros-Ltd” area.
   The permissions are set using:
   – the inline policy of the user data-heros-ltd: Policy 1 below
   – a policy attached to the group customers: Policy 2
   – the general Bucket Policy in the Permissions tab of the bucket Policy 3, which includes your AWS account ID. You can find this on top of the billing console.
4. From the console you can now customize the link to give to customers along with their username and password. They should be able to upload data to their folder using the web interface which I can then process.

Policy 1:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowListBucketIfSpecificPrefixIsIncludedInRequest",
            "Action": [
                "s3:ListBucket"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::my-company-bucket"
            ],
            "Condition": {
                "StringLike": {
                    "s3:prefix": [
                        "customer-A/*"
                    ]
                }
            }
        },
        {
            "Sid": "AllowUserToReadWriteObjectDataInDevelopmentFolder",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::my-company-bucket/customer-A/*"
            ]
        }
    ]
}

Policy 2:

{
  "Version": "2012-10-17",                 
  "Statement": [
    {
      "Sid": "AllowGroupToSeeBucketListAndAlsoAllowGetBucketLocationRequiredForListBucket",
      "Action": ["s3:ListAllMyBuckets", "s3:GetBucketLocation"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::*"]
    },
    {
      "Sid": "AllowRootLevelListingOfCompanyBucket",
      "Action": ["s3:ListBucket"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::my-company-bucket"],
      "Condition":{
          "StringEquals":{"s3:prefix":[""]}
       }
    },
    {
      "Sid": "RequireFolderStyleList",
      "Action": ["s3:ListBucket"],
      "Effect": "Deny",
      "Resource": ["arn:aws:s3:::*"],
      "Condition":{
          "StringNotEquals":{"s3:delimiter":"/"}
       }
     },
    {
      "Sid": "ExplictDenyAccessToPrivateFolderToEveryoneInTheGroup",
      "Action": ["s3:*"],
      "Effect": "Deny",
      "Resource":["arn:aws:s3:::my-company-bucket/other-data/*"]
    },
    {
      "Sid": "DenyListBucketOnPrivateFolder",
      "Action": ["s3:ListBucket"],
      "Effect": "Deny",
      "Resource": ["arn:aws:s3:::*"],
      "Condition":{
          "StringLike":{"s3:prefix":["other-data/"]}
       }
    }
  ]
}

Policy 3:

{
    "Version": "2012-10-17",
    "Id": "Policy1502460168202",
    "Statement": [
        {
            "Sid": "Stmt1502460165827",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::MY-AWS-ID:root"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::my-company-bucket/*"
        }
    ]
}