今天偶然间发现了一个 kubectl 插件 kubectl foreach ,可以在多个集群(contexts)上执行 kubectl 命令。比如 kubectl foreach cluster-1 cluster-2 -- get po -n kube-system 。
插件安装和使用很简单,通过 krew 进行安装:
kubectl krew install foreach
使用也很简单:
kubectl foreach -h
Usage:
kubectl foreach [OPTIONS][PATTERN]... -- [KUBECTL_ARGS...]
Patterns can be used to match context names from kubeconfig:(empty): matches all contexts
NAME: matches context with exact name
/PATTERN/: matches context with regular expression
^NAME: remove context with exact name from the matched results
^/PATTERN/: remove contexts matching the regular expression from the results
Options:-c=NUM Limit parallel executions (default:0, unlimited)-I=VAL Replace VAL occurring in KUBECTL_ARGS with context name
-q Disable and accept confirmation prompts ($KUBECTL_FOREACH_DISABLE_PROMPTS)-h/--help Print help
Examples:
# get nodes on contexts named a b c
kubectl foreach a b c -- get nodes
# get nodes on all contexts named c0..9 except c1 (note the escaping)
kubectl foreach '/^c[0-9]/'^c1 -- get nodes
# get nodes on all contexts that has "prod" but not"foo"
kubectl foreach /prod/^/foo/-- get nodes
# use 'kubectl tail' plugin to follow logs of pods in contexts named *test*
kubectl foreach -I _ /test/-- tail --cnotallow=_ -l app=foo
接下来测试下,使用 k3d 创建 3 个集群 (k3d 貌似不支持同时创建多个集群,还是需要 for 脚本来操作):
for CLUSTER_NAME in cluster-1 cluster-2 cluster-3
do
k3d cluster create ${CLUSTER_NAME} \
--image docker.io/rancher/k3s:v1.23.8-k3s2 \--servers-memory 4g \--k3s-arg "--disable=traefik@server:0" \--no-lb \--timeout 120s \--wait
done
集群安装完成:
k3d cluster list
NAME SERVERS AGENTS LOADBALANCER
cluster-11/10/0false
cluster-21/10/0false
cluster-31/10/0false
kubectl foreach -q /cluster/-- create deploy pipy --image flomesh/pipy -n default
Will run command in context(s):- k3d-cluster-1- k3d-cluster-2- k3d-cluster-3
k3d-cluster-1| deployment.apps/pipy created
k3d-cluster-3| deployment.apps/pipy created
k3d-cluster-2| deployment.apps/pipy created
然后查看下 pod:
kubectl foreach -q /cluster/-- get pod -n default
Will run command in context(s):- k3d-cluster-1- k3d-cluster-2- k3d-cluster-3
k3d-cluster-1| NAME READY STATUS RESTARTS AGE
k3d-cluster-1| pipy-df659b55f-bnr27 1/1 Running 025s
k3d-cluster-3| NAME READY STATUS RESTARTS AGE
k3d-cluster-3| pipy-df659b55f-p9j49 1/1 Running 025s
k3d-cluster-2| NAME READY STATUS RESTARTS AGE
k3d-cluster-2| pipy-df659b55f-9bjgf 1/1 Running 025s
查看日志:
kubectl foreach -q /cluster/-- logs -l app=pipy -n default --tail 3
Will run command in context(s):- k3d-cluster-1- k3d-cluster-2- k3d-cluster-3
k3d-cluster-2|2022-11-3010:40:56.520[INF][listener] Listening on TCP port 8080 at 0.0.0.0
k3d-cluster-2|2022-11-3010:40:56.520[INF][listener] Listening on TCP port 8081 at 0.0.0.0
k3d-cluster-2|2022-11-3010:40:56.520[INF][listener] Listening on TCP port 8082 at 0.0.0.0
k3d-cluster-1|2022-11-3010:40:56.551[INF][listener] Listening on TCP port 8080 at 0.0.0.0
k3d-cluster-1|2022-11-3010:40:56.551[INF][listener] Listening on TCP port 8081 at 0.0.0.0
k3d-cluster-1|2022-11-3010:40:56.551[INF][listener] Listening on TCP port 8082 at 0.0.0.0
k3d-cluster-3|2022-11-3010:40:55.813[INF][listener] Listening on TCP port 8080 at 0.0.0.0
k3d-cluster-3|2022-11-3010:40:55.813[INF][listener] Listening on TCP port 8081 at 0.0.0.0
k3d-cluster-3|2022-11-3010:40:55.813[INF][listener] Listening on TCP port 8082 at 0.0.0.0