lsm/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2025-04-05 07:31:23 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
11,392 Commits 35 Branches 301 Tags 164 MiB
master
Commit Graph

223 Commits

Author SHA1 Message Date
Tom Crasset
be2b389b81
add more logging for s3 signature (#6652)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
2025-03-21 07:37:15 -07:00
SmoothDenis
e60aa71890
fix: restore deletion audit of individual objects (#6644)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
2025-03-18 05:01:54 -07:00
Chris Lu
02773a6107
Accumulated changes for message queue (#6600)
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details 
* rename

* set agent address

* refactor

* add agent sub

* pub messages

* grpc new client

* can publish records via agent

* send init message with session id

* fmt

* check cancelled request while waiting

* use sessionId

* handle possible nil stream

* subscriber process messages

* separate debug port

* use atomic int64

* less logs

* minor

* skip io.EOF

* rename

* remove unused

* use saved offsets

* do not reuse session, since always session id is new after restart

remove last active ts from SessionEntry

* simplify printing

* purge unused

* just proxy the subscription, skipping the session step

* adjust offset types

* subscribe offset type and possible value

* start after the known tsns

* avoid wrongly set startPosition

* move

* remove

* refactor

* typo

* fix

* fix changed path
 2025-03-09 23:49:42 -07:00
chrislu
ddd67dd191 upgade rocksdb to 9.10.0 2025-02-17 20:26:07 -08:00
chrislu
5e46960651 use golang:1.23 image
Some checks are pending
go: build dev binaries / cleanup (push) Waiting to run
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Blocked by required conditions
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Blocked by required conditions
Details
docker: build dev containers / build-dev-containers (push) Waiting to run
Details
End to End / FUSE Mount (push) Waiting to run
Details
go: build binary / Build (push) Waiting to run
Details
Ceph S3 tests / Ceph S3 tests (push) Waiting to run
Details
2025-02-04 09:01:44 -08:00
Tom Crasset
c5f21b2b01
fix S3 per-user-directory Policy (#6443)
Some checks failed
go: build dev binaries / cleanup (push) Has been cancelled
Details
docker: build dev containers / build-dev-containers (push) Has been cancelled
Details
End to End / FUSE Mount (push) Has been cancelled
Details
go: build binary / Build (push) Has been cancelled
Details
Ceph S3 tests / Ceph S3 tests (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, linux) (push) Has been cancelled
Details
go: build dev binaries / build_dev_linux_windows (amd64, windows) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (amd64, darwin) (push) Has been cancelled
Details
go: build dev binaries / build_dev_darwin (arm64, darwin) (push) Has been cancelled
Details 
* fix S3 per-user-directory Policy

* Delete docker/config.json

* add tests

* remove logs

* undo modifications of weed/shell/command_volume_balance.go

* remove modifications of docker-compose

* fix failing test

---------

Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2025-01-17 01:03:17 -08:00
chrislu
8211b29689 fix s3tests.conf file name 2024-06-24 17:15:16 -07:00
chrislu
dd1571b95b add some delay when starting filers, to simulate real production 2024-05-05 14:18:11 -07:00
chrislu
30f8600a24 working 2024-05-03 21:27:06 -07:00
chrislu
cc1c69f312 Merge branch 'master' into mq-subscribe 2024-04-08 11:03:14 -07:00
Konstantin Lebedev
f9ec92dd83
[tests] add action s3 tests (#5427) 2024-03-29 07:57:27 -07:00
chrislu
707ff536a3 Merge branch 'master' into mq-subscribe 2024-03-25 11:21:19 -07:00
Nikita Borzykh
1f3742850d
Add TLS support, initial etcd connection check, fix key_prefix handling for etcd backend meta storage (#5403) 2024-03-24 08:36:21 -07:00
chrislu
aba934f0b5 adjust log level 2024-03-16 23:15:06 -07:00
chrislu
8f79bb3987 Merge branch 'master' into mq-subscribe 2024-03-07 10:42:29 -08:00
Konstantin Lebedev
170b63d6f8
[filer.backup] add param uploader_part_size for S3sink (#5352) 
* fix: install cronie

* chore: refactor configure S3Sink

* chore: refactor cinfig

* add filer-backup compose file

* fix: X-Amz-Meta-Mtime and resolve with comments

* fix: attr mtime

* fix: MaxUploadPartst is reduced to the maximum allowable

* fix: env and force set max MaxUploadParts

* fix: env WEED_SINK_S3_UPLOADER_PART_SIZE_MB
 2024-03-07 08:35:51 -08:00
chrislu
a52ce076fc Merge branch 'master' into mq-subscribe 2024-02-25 08:00:37 -08:00
Chris Lu
2ad363d984 upgrade go, rocksdb 2024-02-07 00:41:03 -08:00
Chris Lu
c471265837 build with pub sub clients for testing 2024-02-05 16:47:11 -08:00
chrislu
0aed16a9c4 add make brokers 2024-01-31 21:31:10 -08:00
Konstantin Lebedev
1169f94310
Fix filer sync set offset (#5197) 
* fix: compose 2mount with sync

* fix: DATA RACE
https://github.com/seaweedfs/seaweedfs/issues/5194
https://github.com/seaweedfs/seaweedfs/issues/5195
 2024-01-12 10:57:18 -08:00
chrislu
53188a1191 fix rocksdb build 2024-01-01 22:42:06 -08:00
Chris Lu
580940bf82
Merge accumulated changes related to message queue (#5098) 
* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* balance partitions on brokers

* prepare topic partition first and then publish, move partition

* purge unused APIs

* clean up

* adjust logs

* add BalanceTopics() grpc API

* configure topic

* configure topic command

* refactor

* repair missing partitions

* sequence of operations to ensure ordering

* proto to close publishers and consumers

* rename file

* topic partition versioned by unixTimeNs

* create local topic partition

* close publishers

* randomize the client name

* wait until no publishers

* logs

* close stop publisher channel

* send last ack

* comments

* comment

* comments

* support list of brokers

* add cli options

* Update .gitignore

* logs

* return io.eof directly

* refactor

* optionally create topic

* refactoring

* detect consumer disconnection

* sub client wait for more messages

* subscribe by time stamp

* rename

* rename to sub_balancer

* rename

* adjust comments

* rename

* fix compilation

* rename

* rename

* SubscriberToSubCoordinator

* sticky rebalance

* go fmt

* add tests

* tracking topic=>broker

* merge

* comment
 2023-12-11 12:05:54 -08:00
chrislu
556a242acf remove unused file 2023-11-30 08:20:01 -08:00
Konstantin Lebedev
ce9b822a8e
golang up version to 1.21.3 (#4911) 2023-10-13 05:31:43 -07:00
Konstantin Lebedev
f8b94cac0e
[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration (#4859) 
* move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface

https://github.com/seaweedfs/seaweedfs/issues/4519

* fix: test bucket acl default and
adjust the variable names

* fix: s3 api config test

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-25 08:34:12 -07:00
Konstantin Lebedev
a40888d343
fix: make dev_tls with volume https (#4866) 2023-09-25 06:47:38 -07:00
chrislu
1494221b08 reduce binary size 2023-09-21 11:58:03 -07:00
Konstantin Lebedev
a46f873edd
[s3acl] Step 0: Put bucket ACL only responds success if the ACL is private. (#4856) 
* Passing test:
test_bucket_acl_default
test_bucket_acl_canned_private_to_private

https://github.com/seaweedfs/seaweedfs/issues/4519

* Update weed/s3api/s3api_bucket_handlers.go

---------

Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
 2023-09-21 08:20:05 -07:00
chrislu
b0da8788a1 upgrade rocksdb versions 2023-05-29 23:14:45 -07:00
chrislu
1f7934a89c enable gocdk in the generated local image 
fix https://github.com/seaweedfs/seaweedfs/issues/4256
 2023-02-27 19:20:02 -08:00
Chris Lu
bac45912e3
go 1.20 (#4192) 2023-02-08 16:41:23 -08:00
CommanderRoot
c7c9d22f37
filer.store.mysql: Use utf8mb4 instead of 3 byte UTF8 (#4094) 2023-01-01 05:07:53 -08:00
Ryan Russell
949607fe90
refactor(compose)!: upgrade to v2 closes #3699 (#3705) 
* docs(compose): Update readme for compose v2 upgrade

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade `Makefile` for compose v2

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* refactor(compose): Upgrade yml from version 2 -> 3.9

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
 2022-10-16 14:02:33 -07:00
Konstantin Lebedev
b9933d5589
master server graceful stop (#3797) 2022-10-06 09:30:30 -07:00
Ryan Russell
b51880ed3b
docs(docker): readability fixes (#3684) 
Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
 2022-09-15 01:27:35 -07:00
chrislu
5afc20dbd3 add tmux for dev 2022-09-01 14:47:45 -07:00
chrislu
04300dc2ac build local rocksdb 2022-09-01 09:44:48 -07:00
chrislu
466bbe4cdd dockerfile fix directory 2022-09-01 09:43:41 -07:00
chrislu
9c76896e2d fix building rocksdb image 2022-09-01 00:20:23 -07:00
Patrick Schmidt
a73e177ecf
Add an End-to-End workflow for FUSE mount (#3562) 
* Add an e2e workflow to test FUSE mount

* Fix deadlocks during concurrent r/w
 2022-08-31 09:27:53 -07:00
Konstantin Lebedev
913a5d0dd4
add target dev_race to MakeFile (#3522) 2022-08-26 01:28:03 -07:00
Konstantin Lebedev
d8c54b527e
avoid Dockerfile Entrypoint: "file not found" (#3430) 
https://www.koskila.net/how-to-fix-exec-user-process-caused-no-such-file-or-directory-in-docker-on-windows/
 2022-08-11 15:07:44 -07:00
chrislu
8a61c124c7 go 1.19 2022-08-02 12:17:31 -07:00
chrislu
5fbe437ee9 update rocksdb version to 7.4.4 2022-07-31 19:47:27 -07:00
chrislu
1d05e7fcbd site move 2022-07-29 15:47:14 -07:00
chrislu
26dbc6c905 move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
Konstantin Lebedev
bcbdc4cb37 use const multipart uploads folder 
avoid error bucket NotEmpty if multipart uploads folder exist
 2022-06-29 16:21:16 +05:00
Chris Lu
e22c2c8012
Merge pull request #3188 from kmlebedev/fix_make_dev 
fix ip.bind cmd for docker compose
 2022-06-29 01:36:59 -07:00
Konstantin Lebedev
5b388ed6c1 initial advancedtls 2022-06-23 23:32:15 +05:00