Migrate data from MinIO to Cloudflare r2

Oct. 30, 2023

Moving existing s3 compatible bucket data to Cloudflare R2

Clouflare R2 is s3 compatible object storage with zero egress fee, the goal is removing workload from self hosted minio for storing static and media file for this site to Cloudflare.

# first do backup on current minio bucket
$ mc alias set source https://storage.saktidwicahyono.name keyid accesskey
# the name of the bucket is 'public'
$ mc cp --recursive source/public .
$ ls
drwxr-xr-x@ - sakti 28 Oct 23:15 public

Setup R2 bucket for this case I named it web then create API key for it, from R2 menu - Manage R2 API Tokens - Create API token. For uploading large amount of files, one of the tool that can be used is rclone. And create the rclone configuration:

$ cat -p ~/.config/rclone/rclone.conf
type = s3
provider = Cloudflare
access_key_id = <keyid>
secret_access_key = <accesskey>
endpoint = https://<cloudflare account id>.r2.cloudflarestorage.com
acl = private


$ rclone copy public/* r2web:web/

then setup custom domain for public access, for this case storager2.saktidwicahyono.name.

The Django config

The Django app need some adjustment for media and static configuration,

AWS_S3_ENDPOINT_URL = "https://<cloudlare account id>.r2.cloudflarestorage.com"
AWS_S3_CUSTOM_DOMAIN = "storager2.saktidwicahyono.name"
MEDIA_URL = "https://storager2.saktidwicahyono.name/"

Done, now after deployment of the new version, all media and static files served by Cloudflare.

Return to blog