Posted | Started | Updated | Runtime |
Duration |
In Waiting |
Machine | Teuthology Branch | OS Type | OS Version | Nodes | Status |
---|---|---|---|---|---|---|---|---|---|---|---|
2021-11-15 18:25:47 | 2021-11-15 18:28:48 | 2021-11-15 18:56:45 | 0:27:57 | 0:17:54 | 0:10:03 | smithi | master | ubuntu | 20.04 | 3 | fail |
Description: orch:rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/calico rook/1.7.0}
Sentry event: https://sentry.ceph.com/organizations/ceph/?query=77dab582dded485fa8b0ef2b522847e9
Command failed on smithi059 with status 127: '\'toolbox()\' \'{\n\' \'\' \'\' \'\' kubectl -n rook-ceph exec -it deploy/rook-ceph-tools -- \'\' \'"$@"\n}\ntoolbox\' ceph orch osd rm 0 \'--force\nremoved_pv=""\nwhile\' \'[\' \'"$removed_pv"\' = \'""\' \']\ndo\n\' \'\' \'\' \'\' \'removed_pv=`kubectl\' get pv \'|\' grep Released \'|\' cut -f1 -d \'"\' \'"`\n\' \'\' \'\' \'\' sleep \'3s\ndone\ntarget_path=`kubectl\' get pv \'$removed_pv\' -o \'jsonpath=\'"\'"\'{.spec.local.path}\'"\'"\'`\ntoolbox\' ceph orch device zap \'`hostname`\' \'$target_path\' \'--force\nzap_completion="0"\nwhile\' \'[\' \'"$zap_completion"\' = \'"0"\' \'\' \']\ndo\n\' \'\' \'\' \'\' \'zap_completion=`kubectl\' get job -n rook-ceph rook-ceph-device-zap -o \'jsonpath=\'"\'"\'{.status.succeeded.path}\'"\'"\'`\n\' \'\' \'\' \'\' sleep \'3s\ndone\nkubectl\' patch pv \'$removed_pv\' -p \'\'"\'"\'{"spec":{"claimRef":\' \'null}}\'"\'"\'\ntoolbox\' ceph orch apply osd \'--all-available-devices\nkubectl\' delete job rook-ceph-device-zap -n \'rook-ceph\nnum_osd="0"\nwhile\' \'[\' \'"$num_osd"\' \'!=\' \'"$orig_num_osd"\' \']\ndo\n\' \'\' \'\' \'\' echo \'"waiting\' for osd to come back \'up"\n\' \'\' \'\' \'\' \'num_osd=`toolbox\' ceph osd stat \'|\' cut -f3 -d \'"\' \'"`\n\' \'\' \'\' \'\' sleep \'30s\ndone\n\''