API
The Gcore Customer Portal is being updated. Screenshots may not show the current version.
CDN
CDN
BillingCDN resources overviewOrigin groupPurgePrefetchReports
API
Chosen image
Home/CDN/CDN resource options/General/Private origin

Use a private bucket as an origin

Overview

Gcore allows private buckets in S3-compatible storage services, including Amazon and Gcore, as a CDN resource origin. However, to allow CDN servers to access the content stored in these private buckets, you must provide authentication data, including Access Key ID, Secret Access Key, Hostname, and Region. Without this information, the servers cannot access the content.

Authentication data

You can find the necessary authentication data in your personal S3 storage account.

Amazon AWS

To locate all the essential keys, refer to the Managing access keys guide.

Gcore

You can find the Hostname and Region in the “Details” section under the Storages tab.

Gcore S3 storage

We don’t store the Access Key ID and Secret access key for S3 storage. It is your responsibility to save them after creating the storage. If you forget them, click Generate new keys under the “Details” section.

Configure a private bucket as an origin

Customer Portal

There are two options for configuring a private bucket as an origin:

  1. Create an origin group during the CDN resource creation process;
  2. Add a group in the "Origins groups" tab and specify this group in the CDN resource setting.

The instructions below are relevant to the first option. The process for the second option is slightly different.

To configure a private origin:

1. Go to the CDN resource creation page, select "Accelerate and protect only static assets", and click Confirm.

CDN resource creation page

A new page will open. Complete the remaining steps of the manual in the "Origin" section on this page.

2. Select the Specify content origin option.

3. Select AWS signature V4 in the "Type of origin authentication" field.

4. Select the type of S3 storage: Amazon or Other.

5. Specify the needed authentication data. It depends on the type of storage selected in the previous step.

  • for Amazon: Access Key ID, Secret access key, and AWS region
  • for Other: Hostname (there's a bucket at the beginning), Access Key ID, Secret access key, and Region

To determine the region of your Gcore S3 storage, use the S3 service URLs and default region names guide.

More information about where to find the keys can be found in the "Authentication data" section.

6. Enter the Bucket name.

In the example below, we use the private bucket test-private in S3 Gcore storage with the endpoint (service URL) s-ed1.cloud.gcore.lu. Configure an origin group as follows:

Origin section

7. Continue the CDN resource creation according to the "Create CDN resource for only static files" guide.

8. Go to the resource settings and open the Host header option in the "HTTP headers" section. Specify the URL of your storage Hostname (the URL depends on your S3 provider) in the following format:

  • For Gcore S3 storage: {bucket-name}.s-ed1.cloud.gcore.lu
  • For most other storages (including Amazon): {bucket-name}.s3.{region-code}.{storage-hostname}

Note: The bucket in the "Host header" is very important to specify for the security of the storage data outside the bucket.

Click Save changes.

Save changes

API

Next, we will explain in detail how to specify private origin via API calls.

Method POST
Header Bearer {{access_token}}

application/json
Payload
{
"name": "YourOriginGroup",
"useNext": true,
"auth_type": "awsSignatureV4",
"auth": {
"s3_type": "amazon",
"s3_access_key_id": "EXAMPLEFODNN7EXAMPLE",
"s3_secret_access_key":
"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"s3_bucket_name": "bucket_name",
"s3_region": "us-east-2"
 }
}
Request parameter For more information, see Gcore’s API documentation
Response 201 Successful

400 Request for creation origin group is failed

403 Group creation is forbidden

To access the API and make authenticated requests, generate an access token. You can use a REST tool like cURL or Postman to send the requests. For this guide, we used Postman.

To specify private origin by URL request:

1. In Postman, open a new request tab.

a. Set the request method to POST.

b. Enter the API URL in the request URL field.

 API URL

2. Go to the Auth tab.

a. Select Bearer Token from the "Type dropdown".

b. Copy the generated access token and paste it into the "Token" field.

Token

3. Go to the Body tab.

a. Select raw as the data type.

b. Select JSON from the format dropdown.

c. Enter the payload in the text area. Replace the sample values indicated by {{ }} with your actual values.

4. Click Send.

Send

Troubleshooting Amazon S3 (403 Forbidden for HEAD requests)

When using private buckets in Amazon S3 storage as an origin, you might encounter a 403 Forbidden error for HEAD requests intended to retrieve metadata. However, a GET request may operate without any errors and return a 200 OK status. The disparity in the results of these requests can be attributed to several causes:

  1. Restricted object-level permissions: If the buckets or IAM policy permits public read access but the object's Access Control List (ACL) denies access to your user or role, AWS S3 will return a 403 status for HEAD requests.

  2. Different policies in effect: If the user making the HEAD request doesn't have the necessary permissions to read the object's metadata, AWS S3 will return a 403 status for HEAD requests.

  3. Incorrect parameters in presigned URLs: If a presigned URL for a HEAD request was generated with incorrect parameters, AWS will return a 403 error. The parameters for URL generation (access key, secret access key, bucket name, object key, etc.) must be consistent for both HEAD and GET requests.

  4. Different owners for bucket and object: If the bucket and object belong to different AWS accounts and the object owner hasn't granted the necessary permissions to the bucket owner, AWS will return a 403 error for a HEAD request and a 200 status for a GET request.

To resolve the issue, modify the bucket policy, IAM, or the object's ACL as required.

Was this article helpful?

Not a Gcore user yet?

Learn more about our next-gen CDN

Go to the product page