SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

blob-storagecloud-drivedistributed-file-systemdistributed-storagedistributed-systemserasure-codingfusehadoop-hdfshdfskubernetesobject-storageposixreplications3s3-storageseaweedfstiered-file-system

Chris Lu d3b267bac2 Issue 15: Stress test corrupts volume ... Checked in for GThomas(tgulacsi78) I've also met with partial writes (pipe closed on localhost), and also met with real corruption: after some partial write, the data size in needle header was read as some huuuuge number, which resulted in memory panic. Please consider the attached patch for ensuring full writes (seeks back to the beginning on needle append error). Hope this helps. Tested with a small tmpfs, lot of "no space left on device" :) (sudo umount -lf /tmp/weed; mkdir -p /tmp/weed && sudo mount -o size=128M,mode=4777 -t tmpfs tmpfs /tmp/weed && bin/weed master -mdir=/tmp/weed -debug=true & bin/weed volume -dir=/tmp/weed -debug=true) GThomas		2013-02-06 14:30:44 -08:00
weed-fs	Issue 15: Stress test corrupts volume	2013-02-06 14:30:44 -08:00