DNS插件在Kubernetes中输入crashloopbackoff 1.4.5

作为kubernetes的新手，我使用“packstack”群集跟随了以下教程。本教程使用kubernetes 1.2.1版，我还算了解是很老：DNS插件在Kubernetes中输入crashloopbackoff 1.4.5

http://kubernetes.io/docs/getting-started-guides/coreos/coreos_multinode_cluster/

一切似乎好，我是能够推出吊舱，所以我虽然我会尝试安装DNS插件作为kubernetes服务/ RC。我搜索了一下，看到DNS添加需要kubernetes版本1.3或更高版本。

我在我的主节点云节点中将Kubernetes版本撞到了1.4.5，然后再次尝试。再次，一切似乎工作 - 除了当我尝试启动DNS复制控制器（和服务），我在日志中看到的错误开始：

Expected to load root CA config from  /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory

我不能确定如何进行。我试图创建另一个服务帐户，但秘密中没有证书。我的云配置没有引用任何根证书，所以我想知道这是否是问题？

我已经附加了我的主节点和节点云配置，我的DNS附加yaml文件和DNS服务器日志。

云少爷配置：

#cloud-config 

--- 
write-files: 
    - path: /etc/conf.d/nfs 
    permissions: '0644' 
    content: | 
     OPTS_RPC_MOUNTD="" 
    - path: /opt/bin/wupiao 
    permissions: '0755' 
    content: | 
     #!/bin/bash 
     # [w]ait [u]ntil [p]ort [i]s [a]ctually [o]pen 
     [ -n "$1" ] && \ 
     until curl -o /dev/null -sIf http://${1}; do \ 
      sleep 1 && echo .; 
     done; 
     exit $? 

hostname: master 
coreos: 
    etcd2: 
    name: master 
    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001 
    advertise-client-urls: http://$private_ipv4:2379,http://$private_ipv4:4001 
    initial-cluster-token: k8s_etcd 
    listen-peer-urls: http://$private_ipv4:2380,http://$private_ipv4:7001 
    initial-advertise-peer-urls: http://$private_ipv4:2380 
    initial-cluster: master=http://$private_ipv4:2380 
    initial-cluster-state: new 
    fleet: 
    metadata: "role=master" 
    units: 
    - name: etcd2.service 
     command: start 
    - name: generate-serviceaccount-key.service 
     command: start 
     content: | 
     [Unit] 
     Description=Generate service-account key file 

     [Service] 
     ExecStartPre=-/usr/bin/mkdir -p /opt/bin 
     ExecStart=/bin/openssl genrsa -out /opt/bin/kube-serviceaccount.key 2048 2>/dev/null 
     RemainAfterExit=yes 
     Type=oneshot 
    - name: setup-network-environment.service 
     command: start 
     content: | 
     [Unit] 
     Description=Setup Network Environment 
     Documentation=https://github.com/kelseyhightower/setup-network-environment 
     Requires=network-online.target 
     After=network-online.target 

     [Service] 
     ExecStartPre=-/usr/bin/mkdir -p /opt/bin 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/setup-network-environment -z /opt/bin/setup-network-environment https://github.com/kelseyhightower/setup-network-environment/releases/download/v1.0.0/setup-network-environment 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/setup-network-environment 
     ExecStart=/opt/bin/setup-network-environment 
     RemainAfterExit=yes 
     Type=oneshot 
    - name: fleet.service 
     command: start 
    - name: flanneld.service 
     command: start 
     drop-ins: 
     - name: 50-network-config.conf 
      content: | 
      [Unit] 
      Requires=etcd2.service 
      [Service] 
      ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.244.0.0/16", "Backend": {"Type": "vxlan"}}' 
    - name: docker.service 
     command: start 
    - name: kube-apiserver.service 
     command: start 
     content: | 
     [Unit] 
     Description=Kubernetes API Server 
     Documentation=https://github.com/kubernetes/kubernetes 
     Requires=setup-network-environment.service etcd2.service generate-serviceaccount-key.service 
     After=setup-network-environment.service etcd2.service generate-serviceaccount-key.service 

     [Service] 
     EnvironmentFile=/etc/network-environment 
     ExecStartPre=-/usr/bin/mkdir -p /opt/bin 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-apiserver -z /opt/bin/kube-apiserver https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-apiserver 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-apiserver 
     ExecStartPre=/opt/bin/wupiao 127.0.0.1:2379/v2/machines 
     ExecStart=/opt/bin/kube-apiserver \ 
     --service-account-key-file=/opt/bin/kube-serviceaccount.key \ 
     --service-account-lookup=false \ 
     --admission-control=NamespaceLifecycle,NamespaceAutoProvision,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota \ 
     --runtime-config=api/v1 \ 
     --allow-privileged=true \ 
     --insecure-bind-address=0.0.0.0 \ 
     --insecure-port=8080 \ 
     --kubelet-https=true \ 
     --secure-port=6443 \ 
     --service-cluster-ip-range=10.244.0.0/16 \ 
     --etcd-servers=http://127.0.0.1:2379 \ 
     --public-address-override=${DEFAULT_IPV4} \ 
     --logtostderr=true 
     Restart=always 
     RestartSec=10 
    - name: kube-controller-manager.service 
     command: start 
     content: | 
     [Unit] 
     Description=Kubernetes Controller Manager 
     Documentation=https://github.com/kubernetes/kubernetes 
     Requires=kube-apiserver.service 
     After=kube-apiserver.service 

     [Service] 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-controller-manager -z /opt/bin/kube-controller-manager https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-controller-manager 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-controller-manager 
     ExecStart=/opt/bin/kube-controller-manager \ 
     --service-account-private-key-file=/opt/bin/kube-serviceaccount.key \ 
     --master=127.0.0.1:8080 \ 
     --logtostderr=true 
     Restart=always 
     RestartSec=10 
    - name: kube-scheduler.service 
     command: start 
     content: | 
     [Unit] 
     Description=Kubernetes Scheduler 
     Documentation=https://github.com/kubernetes/kubernetes 
     Requires=kube-apiserver.service 
     After=kube-apiserver.service 

     [Service] 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-scheduler -z /opt/bin/kube-scheduler https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-scheduler 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-scheduler 
     ExecStart=/opt/bin/kube-scheduler --master=127.0.0.1:8080 
     Restart=always 
     RestartSec=10 
    update: 
    group: alpha 
    reboot-strategy: off

节点云配置

#cloud-config 
write-files: 
    - path: /opt/bin/wupiao 
    permissions: '0755' 
    content: | 
     #!/bin/bash 
     # [w]ait [u]ntil [p]ort [i]s [a]ctually [o]pen 
     [ -n "$1" ] && [ -n "$2" ] && while ! curl --output /dev/null \ 
     --silent --head --fail \ 
     http://${1}:${2}; do sleep 1 && echo -n .; done; 
     exit $? 
coreos: 
    etcd2: 
    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001 
    advertise-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001 
    initial-cluster: master=http://10.10.1.31:2380 
    proxy: on 
    fleet: 
    metadata: "role=node" 
    units: 
    - name: etcd2.service 
     command: start 
    - name: fleet.service 
     command: start 
    - name: flanneld.service 
     command: start 
    - name: docker.service 
     command: start 
    - name: setup-network-environment.service 
     command: start 
     content: | 
     [Unit] 
     Description=Setup Network Environment 
     Documentation=https://github.com/kelseyhightower/setup-network-environment 
     Requires=network-online.target 
     After=network-online.target 

     [Service] 
     ExecStartPre=-/usr/bin/mkdir -p /opt/bin 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/setup-network-environment -z /opt/bin/setup-network-environment https://github.com/kelseyhightower/setup-network-environment/releases/download/v1.0.0/setup-network-environment 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/setup-network-environment 
     ExecStart=/opt/bin/setup-network-environment 
     RemainAfterExit=yes 
     Type=oneshot 
    - name: kube-proxy.service 
     command: start 
     content: | 
     [Unit] 
     Description=Kubernetes Proxy 
     Documentation=https://github.com/kubernetes/kubernetes 
     Requires=setup-network-environment.service 
     After=setup-network-environment.service 

     [Service] 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-proxy -z /opt/bin/kube-proxy https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-proxy 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-proxy 
     # wait for kubernetes master to be up and ready 
     ExecStartPre=/opt/bin/wupiao 10.10.1.31 8080 
     ExecStart=/opt/bin/kube-proxy \ 
     --master=10.10.1.31:8080 \ 
     --logtostderr=true 
     Restart=always 
     RestartSec=10 
    - name: kube-kubelet.service 
     command: start 
     content: | 
     [Unit] 
     Description=Kubernetes Kubelet 
     Documentation=https://github.com/kubernetes/kubernetes 
     Requires=setup-network-environment.service 
     After=setup-network-environment.service 

     [Service] 
     EnvironmentFile=/etc/network-environment 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/kubelet -z /opt/bin/kubelet https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kubelet 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/kubelet 
     # wait for kubernetes master to be up and ready 
     ExecStartPre=/opt/bin/wupiao 10.10.1.31 8080 
     ExecStart=/opt/bin/kubelet \ 
     --address=0.0.0.0 \ 
     --port=10250 \ 
     --hostname-override=${DEFAULT_IPV4} \ 
     --api-servers=10.10.1.31:8080 \ 
     --allow-privileged=true \ 
     --logtostderr=true \ 
     --cadvisor-port=4194 \ 
     --healthz-bind-address=0.0.0.0 \ 
     --healthz-port=10248 
     Restart=always 
     RestartSec=10 
    update: 
    group: alpha 
    reboot-strategy: off

DNS附加组件YAML

apiVersion: v1 
kind: Service 
metadata: 
    name: kube-dns 
    namespace: kube-system 
    labels: 
    k8s-app: kube-dns 
    kubernetes.io/cluster-service: "true" 
    kubernetes.io/name: "KubeDNS" 
spec: 
    selector: 
    k8s-app: kube-dns 
    clusterIP: 10.244.0.5 
    ports: 
    - name: dns 
    port: 53 
    protocol: UDP 
    - name: dns-tcp 
    port: 53 
    protocol: TCP 


--- 


apiVersion: v1 
kind: ReplicationController 
metadata: 
    name: kube-dns-v20 
    namespace: kube-system 
    labels: 
    k8s-app: kube-dns 
    version: v20 
    kubernetes.io/cluster-service: "true" 
spec: 
    replicas: 1 
    selector: 
    k8s-app: kube-dns 
    version: v20 
    template: 
    metadata: 
     labels: 
     k8s-app: kube-dns 
     version: v20 
     annotations: 
     scheduler.alpha.kubernetes.io/critical-pod: '' 
     scheduler.alpha.kubernetes.io/tolerations: '[{"key":"CriticalAddonsOnly", "operator":"Exists"}]' 
    spec: 
     containers: 
     - name: kubedns 
     image: gcr.io/google_containers/kubedns-amd64:1.8 
     resources: 
      limits: 
      memory: 170Mi 
      requests: 
      cpu: 100m 
      memory: 70Mi 
     livenessProbe: 
      httpGet: 
      path: /healthz-kubedns 
      port: 8080 
      scheme: HTTP 
      initialDelaySeconds: 60 
      timeoutSeconds: 5 
      successThreshold: 1 
      failureThreshold: 5 
     readinessProbe: 
      httpGet: 
      path: /readiness 
      port: 8081 
      scheme: HTTP 
      initialDelaySeconds: 3 
      timeoutSeconds: 5 
     args: 
     - --domain=cluster.local. 
     - --dns-port=10053 
     ports: 
     - containerPort: 10053 
      name: dns-local 
      protocol: UDP 
     - containerPort: 10053 
      name: dns-tcp-local 
      protocol: TCP 
     - name: dnsmasq 
     image: gcr.io/google_containers/kube-dnsmasq-amd64:1.4 
     livenessProbe: 
      httpGet: 
      path: /healthz-dnsmasq 
      port: 8080 
      scheme: HTTP 
      initialDelaySeconds: 60 
      timeoutSeconds: 5 
      successThreshold: 1 
      failureThreshold: 5 
     args: 
     - --cache-size=1000 
     - --no-resolv 
     - --server=127.0.0.1#10053 
     - --log-facility=- 
     ports: 
     - containerPort: 53 
      name: dns 
      protocol: UDP 
     - containerPort: 53 
      name: dns-tcp 
      protocol: TCP 
     - name: healthz 
     image: gcr.io/google_containers/exechealthz-amd64:1.2 
     resources: 
      limits: 
      memory: 50Mi 
      requests: 
      cpu: 10m 
      memory: 50Mi 
     args: 
     - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 >/dev/null 
     - --url=/healthz-dnsmasq 
     - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 >/dev/null 
     - --url=/healthz-kubedns 
     - --port=8080 
     - --quiet 
     ports: 
     - containerPort: 8080 
      protocol: TCP 
     dnsPolicy: Default

DNS插件日志

E1118 17:33:10.140677  1 config.go:265] Expected to load root CA config from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory 
I1118 17:33:10.141079  1 server.go:94] Using https://10.244.0.1:443 for kubernetes master, kubernetes API: <nil> 
I1118 17:33:10.141596  1 server.go:99] v1.5.0-alpha.0.1651+7dcae5edd84f06-dirty 
I1118 17:33:10.141728  1 server.go:101] FLAG: --alsologtostderr="false" 
I1118 17:33:10.141840  1 server.go:101] FLAG: --dns-port="10053" 
I1118 17:33:10.141931  1 server.go:101] FLAG: --domain="cluster.local." 
I1118 17:33:10.142073  1 server.go:101] FLAG: --federations="" 
I1118 17:33:10.142171  1 server.go:101] FLAG: --healthz-port="8081" 
I1118 17:33:10.142260  1 server.go:101] FLAG: --kube-master-url="" 
I1118 17:33:10.142345  1 server.go:101] FLAG: --kubecfg-file="" 
I1118 17:33:10.142433  1 server.go:101] FLAG: --log-backtrace-at=":0" 
I1118 17:33:10.142522  1 server.go:101] FLAG: --log-dir="" 
I1118 17:33:10.142605  1 server.go:101] FLAG: --log-flush-frequency="5s" 
I1118 17:33:10.142688  1 server.go:101] FLAG: --logtostderr="true" 
I1118 17:33:10.142771  1 server.go:101] FLAG: --stderrthreshold="2" 
I1118 17:33:10.142853  1 server.go:101] FLAG: --v="0" 
I1118 17:33:10.142932  1 server.go:101] FLAG: --version="false" 
I1118 17:33:10.143056  1 server.go:101] FLAG: --vmodule="" 
I1118 17:33:10.143247  1 server.go:138] Starting SkyDNS server. Listening on port:10053 
I1118 17:33:10.143455  1 server.go:145] skydns: metrics enabled on : /metrics: 
I1118 17:33:10.143556  1 dns.go:166] Waiting for service: default/kubernetes 
I1118 17:33:10.144214  1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0] 
I1118 17:33:10.144358  1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0] 
I1118 17:33:10.154429  1 dns.go:172] Ignoring error while waiting for service default/kubernetes: Get https://10.244.0.1:443/api/v1/namespaces/default/services/kubernetes: x509: failed to load system roots and no roots provided. Sleeping 1s before retrying. 
E1118 17:33:10.159852  1 reflector.go:214] pkg/dns/dns.go:155: Failed to list *api.Service: Get https://10.244.0.1:443/api/v1/services?resourceVersion=0: x509: failed to load system roots and no roots provided 
E1118 17:33:10.171051  1 reflector.go:214] pkg/dns/dns.go:154: Failed to list *api.Endpoints: Get https://10.244.0.1:443/api/v1/endpoints?resourceVersion=0: x509: failed to load system roots and no roots provided 
I1118 17:33:11.157527  1 dns.go:172] Ignoring error while waiting for service default/kubernetes: Get https://10.244.0.1:443/api/v1/namespaces/default/services/kubernetes: x509: failed to load system roots and no roots provided. Sleeping 1s before retrying. 
E1118 17:33:11.166217  1 reflector.go:214] pkg/dns/dns.go:155: Failed to list *api.Service: Get https://10.244.0.1:443/api/v1/services?resourceVersion=0: x509: failed to load system roots and no roots provided 
E1118 17:33:11.181840  1 reflector.go:214] pkg/dns/dns.go:154: Failed to list *api.Endpoints: Get https://10.244.0.1:443/api/v1/endpoints?resourceVersion=0: x509: failed to load system roots and no roots provided

来源

2016-11-18 Gapmeister66

我会尝试删除机密，或干脆服务帐户： Kubernetes如果没有，将创建一个默认的服务帐户。但是，如果您这样做，则需要重新启动所有窗格，以便获取新的服务帐户信息。 – MrE

我也会确保你的集群有适当的容器版本。检查回购正确的图像版本，因为这些事情发生了很大的变化，DNS是这个难题的重要部分。 – MrE

出于礼貌我张贴我的解决方案，这可能也有助于某人在相同的情况。我正在使用cloud-init启动K8S服务并在一个pod中运行DNS。我意识到pod运行在不同的网络中，因此修改我的dns-addon.yaml以将正确的网络地址传递给kubedns容器：“ - --kube-master-url = http://10.10.1.31:8080”。错误消息有点误导！

来源

2016-11-22 17:56:05 Gapmeister66

尽管您可以像您在答案中提出的那样，与不安全的端口通话，从而绕过任何身份验证并解决了kubeDNS问题，但这不会解决集群中使用服务帐户的其他任何内容的秘密。

秘密中没有包含CA的原因是您没有告诉控制器管理器包含一个。您可以为根CA提供标志--root-ca-file。

从kube-controller-manager documentation

如果设置，这根证书颁发机构将包含在服务帐户的令牌秘密。这必须是有效的PEM编码的CA bundle.`

我会强烈建议采取看看current version of the CoreOS Kubernetes Step-by-Step documentation，这是最新的最新与Kubernetes 1.4.3。已经有一些变化，它包括如何适当地生成和使用自签名的证书，以确保您的集群，并提供有效的服务帐户令牌文档

来源

2016-11-23 11:39:46 ChaosAffe

DNS插件在Kubernetes中输入crashloopbackoff 1.4.5

回答

相关问题