Skip to content

Replace minio in test/dev environments #2120

Closed

Description

@JohnStrunk

Describe the feature you'd like to have.
With the changes to minio, it's difficult to continue using it for testing, and we should consider replacing it with a maintained project.


Activity

  1. JohnStrunk commented on Oct 14, 2026

    @JohnStrunk
    MemberAuthor

    Assessment

    Recommendation: RustFS is viable for a test/dev-only pilot as a replacement for VolSync's disposable S3 fixture. It is not a zero-work drop-in, and this would not establish production or full S3 compatibility. At review time, issue #2120 was open with no comments; it does not specify the exact MinIO change causing difficulty or define acceptance criteria.

    What it could simplify

    • The current setup builds MinIO and mc from source refs pinned to 2024 (Dockerfile, version pins), then builds/stages images across architectures in CI (workflow). The upstream MinIO server and mc repositories are archived. RustFS has a maintained project, an official Helm chart, and an official image workflow that builds amd64 and arm64 (workflow).
    • The VolSync test clients use S3 endpoints and credentials: Restic configures an s3: repository, and Rclone uses type = s3 with a MinIO provider (Restic role, Rclone role). RustFS documents SigV4, path-style addressing, and common bucket/object/list/multipart operations (S3 docs). This should keep the change in test/deployment tooling rather than VolSync's mover API.
    • Apache-2.0 (RustFS) versus AGPL-3.0 (MinIO) may ease review of the standalone test image, although VolSync itself is AGPL-licensed, so this is a secondary benefit.

    What it would complicate

    • Bucket and credentials setup: hack/run-minio.sh uses the MinIO chart to create the restic-e2e bucket and known test credentials. RustFS's chart needs explicit standalone mode, ingress disabled, credentials, and storage sizing; I did not find a MinIO-style buckets[] setting. Add an init job/client that creates the bucket through generic S3, and adapt the E2E roles that read the minio Secret and its rootUser/rootPassword keys. Keeping the Helm release/service DNS as minio could avoid changing many test fixtures.
    • TLS: Existing tests connect directly to the in-cluster S3 listener over HTTPS and trust a custom CA. RustFS native TLS uses RUSTFS_TLS_PATH and certificates named rustfs_cert.pem and rustfs_key.pem; chart Ingress TLS is not a substitute for TLS on the service endpoint (TLS docs). Certificate mounts, permissions, and the CA Secret roles need adapting.
    • OpenShift/scorecard: The current script has an OpenShift security-context branch. RustFS chart values default to runAsUser, runAsGroup, and fsGroup 10001 (chart values), which needs validation/overrides for restricted SCCs. The custom scorecard image also bundles MinIO and mc today (Dockerfile); CI image handling and readiness checks will need changes.
    • S3 compatibility: RustFS explicitly documents a tested subset, not complete AWS S3 or MinIO compatibility. Its published compatibility matrix is a snapshot from Aug. 9, 2026—before the 1.0 GA—and lists excluded or incomplete behavior. Test the exact pinned release against Restic and Rclone. Also, replacing the server does not remove mover-restic/minio-go: that is a Restic S3 client dependency, not the server.

    Long-term maintenance and suggested acceptance

    RustFS 1.0.0 was released Sept. 16, 2026, and 1.0.1 on Oct. 3. The project is active, but it is a young GA with a fast-moving release cadence; its security policy documents support for the latest version only. Pin the chart and image (and preferably image digest), then schedule regular stable-version updates with the E2E suite as the compatibility gate—avoid latest.

    I recommend a scoped pilot that passes: Restic empty-repository/backup/restore and Rclone tests; direct HTTP and HTTPS/custom-CA cases; bucket initialization and multipart uploads; the scorecard/OpenShift security path; and the existing Kubernetes/architecture CI coverage. Treat the store as disposable: do not reuse MinIO data directories or assume MinIO-encrypted object formats are portable to RustFS.

    (AI generated)

  2. JohnStrunk commented on Oct 14, 2026

    @JohnStrunk
    MemberAuthor

    This work should remove all minio references except minio-go that is used by restic. We're stuck with that until restic replaces it.

  3. self-assigned this
    on Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions