seaweedfs

Table of Contents

Example Nginx config

It's a common concept to put a proxy in front of S3 that handles requests. Nginx is well suited for this and can be used to handle TLS and virtual-hosted style bucket URLs (using subdomains instead of subfolders).

For virtual-hosted style URL buckets, you'll need to add a wildcard DNS record for your S3 subdomain.

Make sure the config sets the X-Forwarded-Host and optionally the X-Forwarded-Port if you are using a non-standard port. Otherwise, the AWS Signature v4 will not be correctly computed by the server, as it uses the Host header to compute the signature on the client side.

Additionally, make sure that proxy_request_buffering is off (default is on), as the proxy will buffer the request, and send the request to the backend as a whole instead of chunked, and again the signature computed by the client side will be different as it would have taken into account the Transfer-Encoding: chunked header that is dropped by the proxy when it buffers.

Example Nginx config

upstream seaweedfs { server localhost:8333 fail_timeout=0; keepalive 20;}

## Also you can use unix domain socket instead for better performance:
# upstream seaweedfs { server unix:/tmp/seaweedfs-s3-8333.sock; keepalive 20;}

server {
	listen 443 ssl;

	# Assumes that your subdomain is s3
	# The regex will support path style as well as virtual-hosted style bucket URLs
	# path style: http://s3.yourdomain.com/mybucket
	# virtual-hosted style: http://mybucket.s3.yourdomain.com
	server_name ~^(?:(?<bucket>[^.]+)\.)?s3\.yourdomain\.com;

	ignore_invalid_headers off;
	client_max_body_size 0;
	proxy_buffering off;

	proxy_set_header Host $host;
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $remote_addr;
	proxy_set_header X-Forwarded-Host $host;
	proxy_set_header X-Forwarded-Port $server_port;

	proxy_set_header X-Forwarded-Proto $scheme;

	proxy_connect_timeout 300;
	proxy_http_version 1.1;
	proxy_set_header Connection "";
	proxy_request_buffering off;
	chunked_transfer_encoding off;

	# If bucket subdomain is not empty,
	# rewrite request to backend.
	if ($bucket != "") {
		rewrite (.*) /$bucket$1 last;
	}

	location / {
		proxy_pass http://seaweedfs;
	}

	ssl on;
	ssl_certificate /{path_to_ssl_cert}/cert.pem;
	ssl_certificate_key /{path_to_ssl_cert}/key.pem;
}

Introduction

API

Configuration

Filer

Filer Stores

Advanced Filer Configurations

FUSE Mount

WebDAV

Cloud Drive

AWS S3 API

AWS IAM

Machine Learning

TensorFlow with SeaweedFS

HDFS

Replication and Backup

Async Replication to another Filer [Deprecated]
Async Backup
Async Filer Metadata Backup
Async Replication to Cloud [Deprecated]
Kubernetes Backups and Recovery with K8up

Example Nginx config

Introduction

API

Configuration

Filer

Filer Stores

Advanced Filer Configurations

FUSE Mount

WebDAV

Cloud Drive

AWS S3 API

AWS IAM

Machine Learning

HDFS

Replication and Backup

Metadata Change Events

Messaging

Use Cases

Operations

Advanced

Security

Misc Use Case Examples