Skip to content

docker container stopped on the update #15

Open

Description

@mmihalko

I have repeating issue, that the container stops on update. It does not happen every update and happens randomly on different storagenode containers. For example last update to 1.78 two of my nodes stopped, on update to 1.79 different one (but only one updated yet). I have total 9 nodes, so the issue is pretty reliable.

Need to note, the "restart unless-stopped" directive for the container is set and confirmed by "docker inspect". I will run "docker inspect" on stopped container when it happens again.

I enclose tail of docker and node logs leading into stopped container.

I sent some logs and did a discussion on the forum. First post contains logs from node stopped on the update to 1.78.

I would like to note, I had these problems also on previous server. Only similarity I can think is, that both of them were running Debian, but not the same version.

Docker run command:

 docker run -d --restart unless-stopped --stop-timeout 300 --user $(id -u):$(id -g) \
     -p 3000$1:28967/tcp \
     -p 3000$1:28967/udp \
     -p 1400$1:14002 \
     -p 1410$1:14502 \
     -e WALLET=$(load_key "general" "wallet") \
     -e EMAIL=$(load_key "general" "email") \
     -e ADDRESS=$(load_snpar "van-ip"):3000$1 \
     -e STORAGE=$(load_snpar "size") \
     --sysctl net.ipv4.tcp_fastopen=3 \
     --mount type=bind,source=$(load_snpar "node-path")"/identity",destination=/app/identity \
     --mount type=bind,source=$(load_snpar "node-path")"/node",destination=/app/config \
     --name $SNNAME storjlabs/storagenode:latest \
     --operator.wallet-features=zksync \
     --storage2.piece-scan-on-startup=false \
     --log.output="/app/config/node.log" \
     --log.level=$(load_snpar "log-level")

docker.log
node.log

Activity

  1. storjrobot commented on Jun 1, 2023

    @storjrobot

    This issue has been mentioned on Storj Community Forum (official). There might be relevant details there:

    https://forum.storj.io/t/yet-another-node-offline-after-update-thread/22731/15

  2. mmihalko commented on Jun 2, 2023

    @mmihalko
    Author

    today, the issue happened again. Docker inspect of stopped container shows container has "unless-stopped" still enabled

    "RestartPolicy": {
                    "Name": "unless-stopped",
                    "MaximumRetryCount": 0
                },
    
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions