k8s集群部署elasticsearch集群和elasticsearch集群设置用户密码
目录
一、背景:
二、部署elasticsearch集群:
1、部署elasticsearch集群:
2、验证elasticsearch集群是否正常:
三、部署elasticsearch集群并设置用户密码
1、生产elastic集群所需的证书:
2、重新建构elasticsearch镜像:
3、部署elasticsearch集群:
4、设置elasticsearch集群的密码:
5、测试验证elasticsearch集群的认证:
总结:
一、背景:
在paas云平台运维维护过程中有遇见项目需要部署一套elasticsearch集群,用于业务系统的使用。根据业务需求,在k8s集群环境中部署一套elasticsearch集群,操作记录参考改文章!!!
二、部署elasticsearch集群:
默认的elasticsearch集群是没有密码的。
1、部署elasticsearch集群:
apiVersion: v1 kind: Service metadata: name: elasticsearch namespace: sit spec: selector: app: elasticsearch type: ClusterIP ports: - port: 9200 name: es-9200 targetPort: 9200 - port: 9300 name: es-9300 targetPort: 9300 --- apiVersion: apps/v1 kind: StatefulSet metadata: name: es7-cluster namespace: sit spec: serviceName: elasticsearch replicas: 3 selector: matchLabels: app: elasticsearch template: metadata: labels: app: elasticsearch spec: containers: - name: elasticsearch image: dockerhub.jiang.com/jiang-public/elasticsearch:7.9.3 resources: limits: cpu: 1000m requests: cpu: 100m ports: - containerPort: 9200 name: rest protocol: TCP - containerPort: 9300 name: inter-node protocol: TCP volumeMounts: - name: data mountPath: /usr/share/elasticsearch/data env: - name: cluster.name value: k8s-logs - name: node.name valueFrom: fieldRef: fieldPath: metadata.name - name: discovery.zen.minimum_master_nodes value: "2" - name: discovery.seed_hosts value: "es7-cluster-0.elasticsearch,es7-cluster-1.elasticsearch,es7-cluster-2.elasticsearch" - name: cluster.initial_master_nodes value: "es7-cluster-0,es7-cluster-1,es7-cluster-2" - name: ES_JAVA_OPTS value: "-Xms1g -Xmx1g" initContainers: - name: fix-permissions image: dockerhub.jiang.com/system_containers/busybox:latest command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"] securityContext: privileged: true volumeMounts: - name: data mountPath: /usr/share/elasticsearch/data - name: increase-vm-max-map image: dockerhub.jiang.com/system_containers/busybox:latest command: ["sysctl", "-w", "vm.max_map_count=262144"] securityContext: privileged: true - name: increase-fd-ulimit image: dockerhub.jiang.com/system_containers/busybox:latest command: ["sh", "-c", "ulimit -n 65536"] volumeClaimTemplates: - metadata: name: data spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "huawei-san" #根据实际情况而定 resources: requests: storage: 1Gi
执行yaml文件:
[root@master-01 xhj]# kubectl apply -f elasticsearch.yaml service/elasticsearch created statefulset.apps/es7-cluster created
验证结果:
[root@master-01 xhj]# k get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE elasticsearch ClusterIP 172.44.255.234 9200/TCP,9300/TCP 43s mysql ClusterIP None 3306/TCP,9125/TCP 48d [root@master-01 xhj]# k get sts NAME READY AGE es7-cluster 1/3 47s [root@master-01 xhj]# k get sts NAME READY AGE es7-cluster 3/3 2m12s
2、验证elasticsearch集群是否正常:
[root@master-01 xhj]# curl http://172.44.255.234:9200/ { "name" : "es7-cluster-2", "cluster_name" : "k8s-logs", "cluster_uuid" : "VeyRUKdwTHu5lySgV3XJVw", "version" : { "number" : "7.9.3", "build_flavor" : "default", "build_type" : "docker", "build_hash" : "c4138e51121ef06a6404866cddc601906fe5c868", "build_date" : "2020-10-16T10:36:16.141335Z", "build_snapshot" : false, "lucene_version" : "8.6.2", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" }
到此就部署好了一个elasticsearch集群服务!!!!
三、部署elasticsearch集群并设置用户密码
1、生产elastic集群所需的证书:
使用docker 运行elasticsearch容器
[root@master-01 ~]# docker run -it -d --name es-tls dockerhub.jiang.com/jxstjh-public/elasticsearch:7.9.3 /bin/bash 3dff79f1de6f7e3ba83db4b1af2d079e20589d57b028156473cba1b3346f6bbb
进入容器内生成证书
[root@master-01 ~]# docker exec -it es-tls /bin/bash [root@3dff79f1de6f elasticsearch]# pwd /usr/share/elasticsearch [root@3dff79f1de6f elasticsearch]# ./bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""
执行结果显示如下:
This tool assists you in the generation of X.509 certificates and certificate signing requests for use with SSL/TLS in the Elastic stack. The 'cert' mode generates X.509 certificate and private keys. * By default, this generates a single certificate and key for use on a single instance. * The '-multiple' option will prompt you to enter details for multiple instances and will generate a certificate and key for each one * The '-in' option allows for the certificate generation to be automated by describing the details of each instance in a YAML file * An instance is any piece of the Elastic Stack that requires an SSL certificate. Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats may all require a certificate and private key. * The minimum required value for each instance is a name. This can simply be the hostname, which will be used as the Common Name of the certificate. A full distinguished name may also be used. * A filename value may be required for each instance. This is necessary when the name would result in an invalid file or directory name. The name provided here is used as the directory name (within the zip) and the prefix for the key and certificate files. The filename is required if you are prompted and the name is not displayed in the prompt. * IP addresses and DNS names are optional. Multiple values can be specified as a comma separated string. If no IP addresses or DNS names are provided, you may disable hostname verification in your SSL configuration. * All certificates generated by this tool will be signed by a certificate authority (CA). * The tool can automatically generate a new CA for you, or you can provide your own with the -ca or -ca-cert command line options. By default the 'cert' mode produces a single PKCS#12 output file which holds: * The instance certificate * The private key for the instance certificate * The CA certificate If you specify any of the following options: * -pem (PEM formatted output) * -keep-ca-key (retain generated CA key) * -multiple (generate multiple certificates) * -in (generate certificates from an input file) then the output will be be a zip file containing individual certificate/key files Certificates written to /usr/share/elasticsearch/config/elastic-certificates.p12 This file should be properly secured as it contains the private key for your instance. This file is a self contained file and can be copied and used 'as is' For each Elastic product that you wish to configure, you should copy this '.p12' file to the relevant configuration directory and then follow the SSL configuration instructions in the product guide.
将证书文件复制到本地:
[root@master-01 ~]# docker cp es-tls:/usr/share/elasticsearch/config/elastic-certificates.p12 /xhj/elastic-certificates.p12
2、重新建构elasticsearch镜像:
使用证书通过Dockerfile重新封装镜像
FROM dockerhub.jiang.com/jxstjh-public/elasticsearch:7.9.3 MAINTAINER jiang LABEL maintainer="jiang" COPY elastic-certificates.p12 /usr/share/elasticsearch/config/ RUN chown 1000:0 /usr/share/elasticsearch/config/elastic-certificates.p12 EXPOSE 9200 9300 CMD ["eswrapper"]
编译镜像
[root@master-01 xhj]# docker build -t dockerhub.jiang.com/jxstjh-public/elasticsearch:7.9.3-p12 . Sending build context to Docker daemon 127.5MB Step 1/7 : FROM dockerhub.jiang.com/jiang-public/elasticsearch:7.9.3 ---> 1ab13f928dc8 Step 2/7 : MAINTAINER jiang ---> Running in beba0ca606a4 Removing intermediate container beba0ca606a4 ---> 4a4003bf74c9 Step 3/7 : LABEL maintainer="jiang" ---> Running in b1e0c1168982 Removing intermediate container b1e0c1168982 ---> c5eba7158904 Step 4/7 : COPY elastic-certificates.p12 /usr/share/elasticsearch/config/ ---> 7b4dab67c080 Step 5/7 : RUN chown 1000:0 /usr/share/elasticsearch/config/elastic-certificates.p12 ---> Running in cb8f9383fa70 Removing intermediate container cb8f9383fa70 ---> 071eb2ce173e Step 6/7 : EXPOSE 9200 9300 ---> Running in 5ca429e2b39e Removing intermediate container 5ca429e2b39e ---> cfc9b0b05386 Step 7/7 : CMD ["eswrapper"] ---> Running in abbf96c90588 Removing intermediate container abbf96c90588 ---> 9b6304124b9b Successfully built 9b6304124b9b Successfully tagged dockerhub.jiang.com/jiang-public/elasticsearch:7.9.3-p12
3、部署elasticsearch集群:
在k8s上部署elasticsearch集群,使用了statefulset控制器部署的。yaml文件参考如下:
apiVersion: v1 kind: Service metadata: name: elastic-svc #这里根据名称而定 namespace: sit #根据所在命名空间 labels: app: elastic-cluster spec: selector: app: elastic-cluster type: ClusterIP ports: - name: rest-api port: 9200 protocol: TCP targetPort: 9200 - name: inter-node port: 9300 protocol: TCP targetPort: 9300 --- apiVersion: apps/v1 kind: StatefulSet metadata: name: elastic-cluster #根据实际情况调整 namespace: sit #根据所在命名空间 labels: app: elastic-cluster spec: serviceName: elastic-svc #要跟上吗的service的名称对齐 replicas: 3 selector: matchLabels: app: elastic-cluster kubernetes.io/cluster-service: "true" template: metadata: labels: app: elastic-cluster kubernetes.io/cluster-service: "true" spec: initContainers: - name: fix-permissions image: dockerhub.jiang.com/system_containers/busybox:latest #根据镜像所在位置进行调整 imagePullPolicy: IfNotPresent command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"] securityContext: privileged: true volumeMounts: - name: data mountPath: /usr/share/elasticsearch/data - name: localtime readOnly: true mountPath: /etc/localtime - name: increase-vm-max-map image: dockerhub.jiang.com/system_containers/busybox:latest #根据镜像所在位置进行调整 imagePullPolicy: IfNotPresent command: ["sysctl", "-w", "vm.max_map_count=262144"] securityContext: privileged: true - name: increase-fd-ulimit image: dockerhub.jiang.com/system_containers/busybox:latest #根据镜像所在位置进行调整 imagePullPolicy: IfNotPresent command: ["sh", "-c", "ulimit -n 65536"] volumes: - name: localtime hostPath: path: /etc/localtime type: '' containers: - name: elasticsearch image: dockerhub.jiang.com/jiang-public/elasticsearch:7.9.3-p12 #根据镜像所在位置进行调整 imagePullPolicy: IfNotPresent ports: - containerPort: 9200 name: rest-api protocol: TCP - containerPort: 9300 name: inter-node protocol: TCP volumeMounts: - name: data mountPath: /usr/share/elasticsearch/data - name: localtime readOnly: true mountPath: /etc/localtime env: - name: node.name valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: discovery.zen.minimum_master_nodes value: "2" - name: discovery.seed_hosts value: "elastic-svc" #要跟上吗的service名称对齐 - name: cluster.initial_master_nodes value: "elastic-cluster-0,elastic-cluster-1,elastic-cluster-2" #根据集群名称进行调整 - name: ES_JAVA_OPTS value: "-Xms1024m -Xmx1024m" #这里根据业务需求而定,要求是要一样的配置大小 - name: xpack.security.enabled value: "true" - name: xpack.security.transport.ssl.enabled value: "true" - name: xpack.security.transport.ssl.verification_mode value: "certificate" - name: xpack.security.transport.ssl.keystore.path value: "elastic-certificates.p12" - name: xpack.security.transport.ssl.truststore.path value: "elastic-certificates.p12" volumeClaimTemplates: - metadata: name: data spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "huawei-san" #根据实际情况调整 resources: requests: storage: 2Gi
部署Elasticsearch集群
[root@master-01 xhj]# kubectl apply -f elasticsearch-p12.yaml service/elastic-svc created statefulset.apps/elastic-cluster created
验证elasticsearch集群运行情况:
[root@master-01 xhj]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE elastic-svc ClusterIP 172.41.9.197 9200/TCP,9300/TCP 36s mysql ClusterIP None 3306/TCP,9125/TCP 48d
[root@master-01 xhj]# kubectl get sts NAME READY AGE elastic-cluster 3/3 4m6s
4、设置elasticsearch集群的密码:
进入到任意pod容器操作如下:
# kubectl exec -it -n sit pods/elastic-cluster-0 -- /bin/bash
执行如下的命令
./bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user. You will be prompted to enter passwords as the process progresses. Please confirm that you would like to continue [y/N]y #输入yes Enter password for [elastic]: #输入密码,这里选择的elasticsearch Reenter password for [elastic]: #再次输入密码,这里选择的elasticsearch Enter password for [apm_system]: #输入密码,这里选择的elasticsearch Reenter password for [apm_system]: #再次输入密码,这里选择的elasticsearch Enter password for [kibana_system]: #输入密码,这里选择的elasticsearch Reenter password for [kibana_system]: #再次输入密码,这里选择的elasticsearch Enter password for [logstash_system]: #输入密码,这里选择的elasticsearch Reenter password for [logstash_system]: #再次输入密码,这里选择的elasticsearch Enter password for [beats_system]: #输入密码,这里选择的elasticsearch Reenter password for [beats_system]: #再次输入密码,这里选择的elasticsearch Enter password for [remote_monitoring_user]: #输入密码,这里选择的elasticsearch Reenter password for [remote_monitoring_user]: #再次输入密码,这里选择的elasticsearch Changed password for user [apm_system] Changed password for user [kibana_system] Changed password for user [kibana] Changed password for user [logstash_system] Changed password for user [beats_system] Changed password for user [remote_monitoring_user] Changed password for user [elastic]
5、测试验证elasticsearch集群的认证:
[root@master-01 ~]# k get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE elastic-svc ClusterIP 172.34.44.86 9200/TCP,9300/TCP 16m mysql ClusterIP None 3306/TCP,9125/TCP 48d [root@master-01 ~]# curl http://172.34.44.86:9200/ {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}[root@stcs-master-01 ~]#
发现通过无密码的方式访问提示报错,需要进行密码认证。
[root@master-01 ~]# curl --user elastic http://172.34.44.86:9200/ Enter host password for user 'elastic': { "name" : "elastic-cluster-1", "cluster_name" : "docker-cluster", "cluster_uuid" : "O-M9L7gfQBagxQxIBMjXIQ", "version" : { "number" : "7.9.3", "build_flavor" : "default", "build_type" : "docker", "build_hash" : "c4138e51121ef06a6404866cddc601906fe5c868", "build_date" : "2020-10-16T10:36:16.141335Z", "build_snapshot" : false, "lucene_version" : "8.6.2", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" }
通过密码认证的方式,就可以正常访问elasticsearch集群。
总结:
在paas云平台的运维维护中,需要经常自定义部署elasticsearch集群的,要求不高的,基本以无密码的elasticsearch集群为准,要求高的,都是以密码认证的elasticsearch集群为准的。希望可以帮助到大家!!!