实验环境
10.4.7.234 ns1.hostname.com 10.4.7.235 ns2.hostname.com 10.4.7.236 client.hostname.com
设置主机名(10.4.7.234)
[root@iZj6c1pigvpz2jl6zy1cgbZ ~]# hostnamectl set-hostname ns1.hostname.com [root@iZj6c1pigvpz2jl6zy1cgbZ ~]# exit logout [root@ns1 ~]# hostname ns1.hostname.com [root@ns1 ~]# ping -c 4 ns1.hostname.com PING ns1.hostname.com (10.4.7.234) 56(84) bytes of data. 64 bytes from iZj6c1pigvpz2jl6zy1cgbZ (10.4.7.234): icmp_seq=1 ttl=64 time=0.011 ms 64 bytes from iZj6c1pigvpz2jl6zy1cgbZ (10.4.7.234): icmp_seq=2 ttl=64 time=0.025 ms 64 bytes from iZj6c1pigvpz2jl6zy1cgbZ (10.4.7.234): icmp_seq=3 ttl=64 time=0.026 ms 64 bytes from iZj6c1pigvpz2jl6zy1cgbZ (10.4.7.234): icmp_seq=4 ttl=64 time=0.027 ms --- ns1.hostname.com ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 2999ms rtt min/avg/max/mdev = 0.011/0.022/0.027/0.007 ms [root@ns1 ~]#
设置主机名(10.4.7.235)
[root@iZj6c1pigvpz2jl6zy1cgaZ ~]# hostnamectl set-hostname ns2.hostname.com [root@iZj6c1pigvpz2jl6zy1cgaZ ~]# exit logout [root@ns2 ~]# hostname ns2.hostname.com [root@ns2 ~]# ping -c 4 ns2.hostname.com PING ns2.hostname.com (10.4.7.235) 56(84) bytes of data. 64 bytes from iZj6c1pigvpz2jl6zy1cgaZ (10.4.7.235): icmp_seq=1 ttl=64 time=0.014 ms 64 bytes from iZj6c1pigvpz2jl6zy1cgaZ (10.4.7.235): icmp_seq=2 ttl=64 time=0.028 ms 64 bytes from iZj6c1pigvpz2jl6zy1cgaZ (10.4.7.235): icmp_seq=3 ttl=64 time=0.027 ms 64 bytes from iZj6c1pigvpz2jl6zy1cgaZ (10.4.7.235): icmp_seq=4 ttl=64 time=0.028 ms --- ns2.hostname.com ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 2999ms rtt min/avg/max/mdev = 0.014/0.024/0.028/0.006 ms [root@ns2 ~]#
设置主机名(10.4.7.236)
[root@iZj6c1pigvpz2jl6zy1cgcZ ~]# hostnamectl set-hostname client.hostname.com [root@iZj6c1pigvpz2jl6zy1cgcZ ~]# exit logout [root@client ~]# hostname client.hostname.com [root@client ~]# ping -c 4 client.hostname.com PING client.hostname.com (10.4.7.236) 56(84) bytes of data. 64 bytes from iZj6c1pigvpz2jl6zy1cgcZ (10.4.7.236): icmp_seq=1 ttl=64 time=0.013 ms 64 bytes from iZj6c1pigvpz2jl6zy1cgcZ (10.4.7.236): icmp_seq=2 ttl=64 time=0.028 ms 64 bytes from iZj6c1pigvpz2jl6zy1cgcZ (10.4.7.236): icmp_seq=3 ttl=64 time=0.028 ms 64 bytes from iZj6c1pigvpz2jl6zy1cgcZ (10.4.7.236): icmp_seq=4 ttl=64 time=0.026 ms --- client.hostname.com ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3000ms rtt min/avg/max/mdev = 0.013/0.023/0.028/0.008 ms [root@client ~]#
安装bind及bind-utils软件包
[root@ns1 ~]# yum -y install bind bind-utils [root@ns2 ~]# yum -y install bind bind-utils [root@client ~]# yum -y install bind-utils
修改主DNS服务器的主配置文件
[root@ns1 ~]# vi /etc/named.conf 修改以下行内容 13 listen-on port 53 { 10.4.7.234; }; 21 allow-query { any; }; 35 dnssec-enable no; 36 dnssec-validation no; 添加以下内容 allow-transfer { 10.4.7.235; }; allow-notify { 10.4.7.235; };
新增区域配置
[root@ns1 ~]# vi /etc/named.rfc1912.zones zone "hostname.com" IN { type master; file "hostname.com.zone"; allow-update { none; }; };
创建区域配置文件
[root@ns1 ~]# vi /var/named/hostname.com.zone $TTL 600 ; 10 minutes @ IN SOA ns1.hostname.com. hostmaster.hostname.com. ( 20190001 ; serial number 10800 ; refresh time (3 hours) 900 ; retry time (15 minutes) 604800 ; expire time (1 week) 86400 ; minimum (1 day) ) NS ns1.hostname.com. NS ns2.hostname.com. $ORIGIN hostname.com. $TTL 60 ; 1 minute ns1 A 10.4.7.234 ns2 A 10.4.7.235 client A 10.4.7.236
检查配置信息
[root@ns1 ~]# named-checkconf [root@ns1 ~]# named-checkzone hostname.com. /var/named/hostname.com.zone zone hostname.com/IN: loaded serial 20190001 OK [root@ns1 ~]#
重启服务并查看监听和验证解析
配置从DNS服务器
修改从DNS服务器的主配置文件
[root@ns2 ~]# vi /etc/named.conf 修改客户机DNS配置 13 listen-on port 53 { 10.4.7.235; }; 21 allow-query { any; }; 35 dnssec-enable no; 36 dnssec-validation no; 增加 masterfile-format text;
添加区域配置
[root@ns2 ~]# vi /etc/named.rfc1912.zones zone "hostname.com" IN { type slave; masters { 10.4.7.234; }; file "slaves/hostname.com.zone"; };
检查配置启动服务
[root@ns2 ~]# named-checkconf [root@ns2 ~]# systemctl enable named Created symlink from /etc/systemd/system/multi-user.target.wants/named.service to /usr/lib/systemd/system/named.service. [root@ns2 ~]# systemctl start named [root@ns2 ~]#
查看端口监听
检查完全区域传送信息及查看本地已同步的区域配置文件
验证解析
客户机配置
Ping 客户机本机及两台DNS主机的域名,默认只有本机响应
[root@client ~]# ping -c 1 client.hostname.com PING client.hostname.com (10.4.7.236) 56(84) bytes of data. 64 bytes from iZj6c1pigvpz2jl6zy1cgcZ (10.4.7.236): icmp_seq=1 ttl=64 time=0.011 ms --- client.hostname.com ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.011/0.011/0.011/0.000 ms [root@client ~]# ping -c 1 ns1.hostname.com ping: ns1.hostname.com: Name or service not known [root@client ~]# ping -c 1 ns2.hostname.com ping: ns2.hostname.com: Name or service not known [root@client ~]#
查看当前主机DNS配置
[root@client ~]# cat /etc/resolv.conf options timeout:2 attempts:3 rotate single-request-reopen ; generated by /usr/sbin/dhclient-script nameserver 100.100.2.136 nameserver 100.100.2.138 [root@client ~]#
修改DNS
[root@client ~]# vi /etc/resolv.conf options timeout:2 attempts:3 rotate single-request-reopen ; generated by /usr/sbin/dhclient-script #nameserver 100.100.2.136 #nameserver 100.100.2.138 nameserver 10.4.7.234 nameserver 10.4.7.235
再次ping两台DNS服务器的主机名,及使用nslookup查询,主辅DNS均能够响应客户端查询
查看当前系统版本
[root@iZj6cehstgjoj3qav88fidZ ~]# cat /etc/redhat-release CentOS Linux release 7.6.1810 (Core) [root@iZj6cehstgjoj3qav88fidZ ~]#
查看bind软件包信息
[root@iZj6cehstgjoj3qav88fidZ ~]# yum info bind Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Available Packages Name : bind Arch : x86_64 Epoch : 32 Version : 9.9.4 Release : 73.el7_6 Size : 1.8 M Repo : updates/7/x86_64 Summary : The Berkeley Internet Name Domain (BIND) DNS (Domain Name System) server URL : http://www.isc.org/products/BIND/ License : ISC Description : BIND (Berkeley Internet Name Domain) is an implementation of the DNS : (Domain Name System) protocols. BIND includes a DNS server (named), : which resolves host names to IP addresses; a resolver library : (routines for applications to use when interfacing with DNS); and : tools for verifying that the DNS server is operating properly. [root@iZj6cehstgjoj3qav88fidZ ~]#
使用yum安装bind及依赖包
[root@iZj6cehstgjoj3qav88fidZ ~]# yum install bind
安装清单
Installed: bind.x86_64 32:9.9.4-73.el7_6 Dependency Installed: audit-libs-python.x86_64 0:2.8.4-4.el7 bind-libs.x86_64 32:9.9.4-73.el7_6 checkpolicy.x86_64 0:2.5-8.el7 libcgroup.x86_64 0:0.41-20.el7 libsemanage-python.x86_64 0:2.5-14.el7 policycoreutils-python.x86_64 0:2.5-29.el7_6.1 python-IPy.noarch 0:0.75-6.el7 python-ply.noarch 0:3.4-11.el7 setools-libs.x86_64 0:3.3.8-4.el7
注册服务及启动服务
[root@iZj6cehstgjoj3qav88fidZ ~]# systemctl enable named Created symlink from /etc/systemd/system/multi-user.target.wants/named.service to /usr/lib/systemd/system/named.service. [root@iZj6cehstgjoj3qav88fidZ ~]# systemctl start named [root@iZj6cehstgjoj3qav88fidZ ~]#
查看端口监听
查看主配置文件
[root@iZj6cehstgjoj3qav88fidZ ~]# vi /etc/named.conf // // named.conf // // Provided by Red Hat bind package to configure the ISC BIND named(8) DNS // server as a caching only nameserver (as a localhost DNS resolver only). // // See /usr/share/doc/bind*/sample/ for example named configuration files. // // See the BIND Administrator's Reference Manual (ARM) for details about the // configuration located in /usr/share/doc/bind-{version}/Bv9ARM.html options { listen-on port 53 { 127.0.0.1; }; listen-on-v6 port 53 { ::1; }; directory "/var/named"; dump-file "/var/named/data/cache_dump.db"; statistics-file "/var/named/data/named_stats.txt"; memstatistics-file "/var/named/data/named_mem_stats.txt"; recursing-file "/var/named/data/named.recursing"; secroots-file "/var/named/data/named.secroots"; allow-query { localhost; }; /* - If you are building an AUTHORITATIVE DNS server, do NOT enable recursion. - If you are building a RECURSIVE (caching) DNS server, you need to enable recursion. - If your recursive DNS server has a public IP address, you MUST enable access control to limit queries to your legitimate users. Failing to do so will cause your server to become part of large scale DNS amplification attacks. Implementing BCP38 within your network would greatly reduce such attack surface */ recursion yes; dnssec-enable yes; dnssec-validation yes; /* Path to ISC DLV key */ bindkeys-file "/etc/named.iscdlv.key"; managed-keys-directory "/var/named/dynamic"; pid-file "/run/named/named.pid"; session-keyfile "/run/named/session.key"; }; logging { channel default_debug { file "data/named.run"; severity dynamic; }; }; zone "." IN { type hint; file "named.ca"; }; include "/etc/named.rfc1912.zones"; include "/etc/named.root.key";
查看当前主机DNS配置
[root@iZj6cehstgjoj3qav88fidZ ~]# cat /etc/resolv.conf options timeout:2 attempts:3 rotate single-request-reopen ; generated by /usr/sbin/dhclient-script nameserver 100.100.2.136 nameserver 100.100.2.138 search localdomain [root@iZj6cehstgjoj3qav88fidZ ~]#
安装域名解析提示工具包
[root@iZj6cehstgjoj3qav88fidZ ~]# yum install bind-utils
查看软件版路径及文件
[root@iZj6cehstgjoj3qav88fidZ ~]# rpm -lq bind-utils /etc/trusted-key.key /usr/bin/dig /usr/bin/host /usr/bin/nslookup /usr/bin/nsupdate /usr/share/man/man1/dig.1.gz /usr/share/man/man1/host.1.gz /usr/share/man/man1/nslookup.1.gz /usr/share/man/man1/nsupdate.1.gz [root@iZj6cehstgjoj3qav88fidZ ~]#
修改本机DNS为当前DNS服务器并尝试域名解析操作
[root@iZj6cehstgjoj3qav88fidZ ~]# vi /etc/resolv.conf options timeout:2 attempts:3 rotate single-request-reopen ; generated by /usr/sbin/dhclient-script #nameserver 100.100.2.136 #nameserver 100.100.2.138 nameserver 127.0.0.1 search localdomain
使用nslookup命令及ping命令检查域名递归(缓存)DNS配置生效情况
为内网主机配置正向解析和反向解析
查看默认的区域配置文件
// named.rfc1912.zones: // // Provided by Red Hat caching-nameserver package // // ISC BIND named zone configuration for zones recommended by // RFC 1912 section 4.1 : localhost TLDs and address zones // and http://www.ietf.org/internet-drafts/draft-ietf-dnsop-default-local-zones-02.txt // (c)2007 R W Franks // // See /usr/share/doc/bind*/sample/ for example named configuration files. // zone "localhost.localdomain" IN { type master; file "named.localhost"; allow-update { none; }; }; zone "localhost" IN { type master; file "named.localhost"; allow-update { none; }; }; zone "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" IN { type master; file "named.loopback"; allow-update { none; }; }; zone "1.0.0.127.in-addr.arpa" IN { type master; file "named.loopback"; allow-update { none; }; }; zone "0.in-addr.arpa" IN { type master; file "named.empty"; allow-update { none; }; };
添加自定义区域配置
[root@iZj6cehstgjoj3qav88fidZ ~]# vi /etc/named.rfc1912.zones zone "ns-zone.com" IN { type master; file "ns-zone.com.zone"; allow-update { none; }; };
创建自定义区域配置文件
[root@iZj6cehstgjoj3qav88fidZ ~]# vi /var/named/ns-zone.com.zone $TTL 600 ; 10 minutes @ IN SOA ns1.ns-zone.com. hostmaster.ns-zone.com. ( 20190001 ; serial number 10800 ; refresh time (3 hours) 900 ; retry time (15 minutes) 604800 ; expire time (1 week) 86400 ; minimum (1 day) ) NS ns1.ns-zone.com. $ORIGIN ns-zone.com. $TTL 60 ; 1 minute ns1 A 10.22.105.60 www A 10.22.105.60
修改区域配置文件属性
检查主配置文件和自定义区域配置文件
[root@iZj6cehstgjoj3qav88fidZ ~]# named-checkconf [root@iZj6cehstgjoj3qav88fidZ ~]# [root@iZj6cehstgjoj3qav88fidZ ~]# vi /var/named/ns-zone.com.zone [root@iZj6cehstgjoj3qav88fidZ ~]# named-checkzone ns-zone.com. /var/named/ns-zone.com.zone zone ns-zone.com/IN: loaded serial 20190001 OK [root@iZj6cehstgjoj3qav88fidZ ~]#
重启bind服务并验证新增加的正向解析
[root@iZj6cehstgjoj3qav88fidZ ~]# systemctl restart named [root@iZj6cehstgjoj3qav88fidZ ~]# nslookup ns1.ns-zone.com Server: 127.0.0.1 Address: 127.0.0.1#53 Name: ns1.ns-zone.com Address: 10.22.105.60 [root@iZj6cehstgjoj3qav88fidZ ~]# nslookup www.ns-zone.com Server: 127.0.0.1 Address: 127.0.0.1#53 Name: www.ns-zone.com Address: 10.22.105.60 [root@iZj6cehstgjoj3qav88fidZ ~]#
使用dig命令查看NS记录
反向解析
增加区域配置
[root@iZj6cehstgjoj3qav88fidZ ~]# vi /etc/named.rfc1912.zones zone "105.22.10.in-addr.arpa" IN { type master; file "105.22.10.in-addr.arpa.zone"; allow-update { none; }; };
增加自定义区域配置文件
[root@iZj6cehstgjoj3qav88fidZ ~]# vi /var/named/105.22.10.in-addr.arpa.zone $TTL 600 ; 10 minutes @ IN SOA ns1.ns-zone.com. hostmaster.ns-zone.com. ( 20190001 ; serial number 10800 ; refresh time (3 hours) 900 ; retry time (15 minutes) 604800 ; expire time (1 week) 86400 ; minimum (1 day) ) NS ns1.ns-zone.com. $ORIGIN 105.22.10.in-addr.arpa. $TTL 60 ; 1 minute 60 PTR ns1.ns-zone.com.
检查主配置文件和自定义区域配置文件
[root@iZj6cehstgjoj3qav88fidZ ~]# named-checkconf [root@iZj6cehstgjoj3qav88fidZ ~]# [root@iZj6cehstgjoj3qav88fidZ ~]# named-checkzone 105.22.10.in-addr.arpa. /var/named/105.22.10.in-addr.arpa.zone zone 105.22.10.in-addr.arpa/IN: loaded serial 20190001 OK [root@iZj6cehstgjoj3qav88fidZ ~]#
验证反向解析
禁用SElinux设置
[root@ip-172-31-47-5 ~]# setenforce 0 [root@ip-172-31-47-5 ~]# sudo sed -i 's/^SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
上传并安装JAVA运行环境
[root@ip-172-31-47-5 ~]# yum -y install lrzsz [root@ip-172-31-47-5 ~]# rz rz waiting to receive. Starting zmodem transfer. Press Ctrl+C to cancel. Transferring jdk-8u241-linux-x64.rpm... 100% 174745 KB 604 KB/sec 00:04:49 0 Errors [root@ip-172-31-47-5 ~]# [root@ip-172-31-47-5 ~]# yum -y install jdk-8u241-linux-x64.rpm [root@ip-172-31-47-5 ~]# java -version java version "1.8.0_241" Java(TM) SE Runtime Environment (build 1.8.0_241-b07) Java HotSpot(TM) 64-Bit Server VM (build 25.241-b07, mixed mode) [root@ip-172-31-47-5 ~]#
下载Kafka二进制包并解压缩
https://downloads.apache.org/ [root@ip-172-31-47-5 ~]# curl -O https://downloads.apache.org/kafka/2.4.0/kafka_2.11-2.4.0.tgz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 66.8M 100 66.8M 0 0 5877k 0 0:00:11 0:00:11 --:--:-- 7302k [root@ip-172-31-47-5 ~]# [root@ip-172-31-47-5 ~]# tar xzf kafka_2.11-2.4.0.tgz [root@ip-172-31-47-5 ~]# mv kafka_2.11-2.4.0 /usr/local/ [root@ip-172-31-47-5 ~]#
启动zookeeper服务
[root@ip-172-31-47-5 ~]# cd /usr/local/kafka_2.11-2.4.0/ [root@ip-172-31-47-5 kafka_2.11-2.4.0]# ./bin/zookeeper-server-start.sh -daemon config/zookeeper.properties [root@ip-172-31-47-5 kafka_2.11-2.4.0]#
修改kafka服务堆内存配置
[root@ip-172-31-47-5 kafka_2.11-2.4.0]# vi bin/kafka-server-start.sh
默认 export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G 修改为 export KAFKA_HEAP_OPTS="-Xmx8G -Xms8G
启动服务
[root@ip-172-31-47-5 kafka_2.11-2.4.0]# ./bin/kafka-server-start.sh config/server.properties
成功启动服务后的终端屏幕输出最后一行
[2020-03-06 07:19:10,271] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
查看端口监听
[root@ip-172-31-47-5 ~]# netstat -lntp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 3712/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 3368/master tcp6 0 0 :::111 :::* LISTEN 1/systemd tcp6 0 0 :::22 :::* LISTEN 3712/sshd tcp6 0 0 :::45848 :::* LISTEN 13243/java tcp6 0 0 ::1:25 :::* LISTEN 3368/master tcp6 0 0 :::37435 :::* LISTEN 13406/java tcp6 0 0 :::9092 :::* LISTEN 13406/java tcp6 0 0 :::2181 :::* LISTEN 13243/java [root@ip-172-31-47-5 ~]#
创建测试主题
[centos@ip-172-31-47-5 ~]$ cd /usr/local/kafka_2.11-2.4.0/ [centos@ip-172-31-47-5 kafka_2.11-2.4.0]$ ./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test Created topic test. [centos@ip-172-31-47-5 kafka_2.11-2.4.0]$
查看主题
[centos@ip-172-31-47-5 kafka_2.11-2.4.0]$ ./bin/kafka-topics.sh --list --zookeeper localhost:2181 test [centos@ip-172-31-47-5 kafka_2.11-2.4.0]$
使用测试主题生产消息(生产者)
[centos@ip-172-31-47-5 kafka_2.11-2.4.0]$ echo "Welcome to Kafka!" | ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test > /dev/null [centos@ip-172-31-47-5 kafka_2.11-2.4.0]$
使用消息(消费者)(Ctrl+C退出)
[root@ip-172-31-47-5 kafka_2.11-2.4.0]# ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning Welcome to Kafka! ^C Processed a total of 1 messages [root@ip-172-31-47-5 kafka_2.11-2.4.0]#
Logstash 是开源的服务器端数据处理管道,能够同时从多个来源采集数据,转换数据,然后将数据发送到您最喜欢的 “存储库” 中。(我们的存储库当然是 Elasticsearch。)
https://www.elastic.co/guide/index.html
通过 Kibana,您能够对 Elasticsearch 中的数据进行可视化并在 Elastic Stack 进行操作,因此您可以在这里解开任何疑问:例如,为何会在凌晨 2:00 收到传呼,雨水会对季度数据造成怎样的影响。
https://www.elastic.co/guide/index.html
下载安装
[root@earth ~]# curl -O https://artifacts.elastic.co/downloads/kibana/kibana-7.1.1-linux-x86_64.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 160M 100 160M 0 0 13.1M 0 0:00:12 0:00:12 --:--:-- 12.2M [root@earth ~]# tar xzf kibana-7.1.1-linux-x86_64.tar.gz [root@earth ~]# mv kibana-7.1.1-linux-x86_64 /usr/local/ [root@earth ~]# cd /usr/local/kibana-7.1.1-linux-x86_64/ [root@earth kibana-7.1.1-linux-x86_64]# ls bin built_assets config data LICENSE.txt node node_modules NOTICE.txt optimize package.json plugins README.txt src target webpackShims [root@earth kibana-7.1.1-linux-x86_64]#
配置目录及配置文件
[root@earth kibana-7.1.1-linux-x86_64]# ls config/ kibana.yml [root@earth kibana-7.1.1-linux-x86_64]#
修改配置文件
[root@earth kibana-7.1.1-linux-x86_64]# vi config/kibana.yml # Kibana is served by a back end server. This setting specifies the port to use. server.port: 5601 # Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values. # The default is 'localhost', which usually means remote machines will not be able to connect. # To allow connections from remote users, set this parameter to a non-loopback address. server.host: "0.0.0.0"
启动kibana
[root@earth kibana-7.1.1-linux-x86_64]# nohup ./bin/kibana & [1] 20182 [root@earth kibana-7.1.1-linux-x86_64]# nohup: ignoring input and appending output to ‘nohup.out’ [root@earth kibana-7.1.1-linux-x86_64]#
确认端口监听
使用浏览器访问Kibana门户
http://47.75.246.214:5601
Kibana程序目录结构
Elasticsearch 是一个分布式、RESTful 风格的搜索和数据分析引擎,能够解决不断涌现出的各种用例。作为 Elastic Stack 的核心,它集中存储您的数据,帮助您发现意料之中以及意料之外的情况。
https://www.elastic.co/guide/index.html
软件版本选择
Beats 7.1 APM Server 7.1 Elasticsearch 7.1 Elasticsearch Hadoop 7.1 Kibana 7.1 Logstash 7.1
Elastic Stack系列产品的安装顺序
Install the Elastic Stack products you want to use in the following order: Elasticsearch Kibana Logstash Beats APM Server Elasticsearch Hadoop
Elasticsearch在生产环境中的重要系统配置要求
Disable swapping(禁用交换分区) Increase file descriptors(提升文件描述符到最大值(65535)) Ensure sufficient virtual memory(虚拟内存设置,临时更改sysctl -w vm.max_map_count=262144永久更改/etc/sysctl.conf) Ensure sufficient threads(用户进程数必须大于等于4096/etc/security/limits.conf) JVM DNS cache settings(详见jvm.options配置文件) Temporary directory not mounted with noexec (临时目录挂载参数不包括noexec)
阿里云ECS镜像CentOS 7.x的资源限制
ulimit命令的资源限制参数
-a All current limits are reported -b The maximum socket buffer size -c The maximum size of core files created -d The maximum size of a process's data segment -e The maximum scheduling priority ("nice") -f The maximum size of files written by the shell and its children -i The maximum number of pending signals -l The maximum size that may be locked into memory -m The maximum resident set size (many systems do not honor this limit) -n The maximum number of open file descriptors (most systems do not allow this value to be set) -p The pipe size in 512-byte blocks (this may not be set) -q The maximum number of bytes in POSIX message queues -r The maximum real-time scheduling priority -s The maximum stack size -t The maximum amount of cpu time in seconds -u The maximum number of processes available to a single user -v The maximum amount of virtual memory available to the shell and, on some systems, to its children -x The maximum number of file locks -T The maximum number of threads
以非root用户权限运行elasticsearch服务
添加用户
[root@earth ~]# useradd elastic [root@earth ~]#
下载安装并修改权限
[root@earth ~]# curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.1.1-linux-x86_64.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 330M 100 330M 0 0 12.6M 0 0:00:26 0:00:26 --:--:-- 12.2M [root@earth ~]# [root@earth ~]# tar xzf elasticsearch-7.1.1-linux-x86_64.tar.gz [root@earth ~]# mv elasticsearch-7.1.1 /usr/local/ [root@earth ~]# cd /usr/local/ [root@earth local]# chown -R elastic.elastic elasticsearch-7.1.1/ [root@earth local]#
安装目录中的主要配置文件及用途
elasticsearch.yml for configuring Elasticsearch jvm.options for configuring Elasticsearch JVM settings log4j2.properties for configuring Elasticsearch logging
修改配置文件elasticsearch.yml配置文件
[root@earth ~]# cd /usr/local/elasticsearch-7.1.1/config/ [root@earth config]# vi elasticsearch.yml # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: /usr/local/elasticsearch-7.1.1/data # # Path to log files: # path.logs: /usr/local/elasticsearch-7.1.1/logs # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 127.0.0.1 # # Set a custom port for HTTP: # http.port: 9200 # # For more information, consult the network module documentation. #
切换到elastic用户并启动ES服务
[root@earth ~]# su - elastic Last login: Wed May 29 16:56:38 CST 2019 on pts/0 [elastic@earth ~]$ cd /usr/local/elasticsearch-7.1.1/ [elastic@earth elasticsearch-7.1.1]$ ./bin/elasticsearch -d -p pid OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. [elastic@earth elasticsearch-7.1.1]$
查看启动日志
查看端口监听
停止ES服务
[elastic@earth elasticsearch-7.1.1]$ pkill -F pid
验证Elasticsearch运行状态
[elastic@earth elasticsearch-7.1.1]$ curl localhost:9200 { "name" : "earth", "cluster_name" : "elasticsearch", "cluster_uuid" : "gAkz51cpSUuuL0MK5HBGCg", "version" : { "number" : "7.1.1", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "7a013de", "build_date" : "2019-05-23T14:04:00.380842Z", "build_snapshot" : false, "lucene_version" : "8.0.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" } [elastic@earth elasticsearch-7.1.1]$
Elasticsearch目录结构说明
……
https://github.com/goharbor/harbor/releases
Support for OpenID Connect
OpenID Connect (OIDC) is an authentication layer on top of OAuth 2.0, allowing Harbor to verify the identity of users based on the authentication performed by an external authorization server or identity provider.
Robot accounts
Robot accounts can be configured to provide administrators with a token that can be granted appropriate permissions for pulling or pushing images. Harbor users can continue operating Harbor using their enterprise SSO credentials, and use robot accounts for CI/CD systems that perform Docker client commands.
Replication advancements
Harbor new version replication allows you to replicate your Harbor repository to and from non-Harbor registries. Harbor 1.8 expands on the Harbor-to-Harbor replication feature, adding the ability to replicate resources between Harbor and Docker Hub, Docker Registry, and Huawei Registry. This is enabled through both push and pull mode replication.
Additional Features
Harbor 1.8 brings numerous other capabilities for both administrators and end users:
Health check API, showing detailed status and health of all Harbor components.
Support for defining cron-based scheduled tasks in the Harbor UI. Administrators can now use cron strings to define the schedule of a job. Scan, garbage collection and replication jobs are all supported.
API explorer integration. End users can now explore and trigger Harbor’s API via the swagger UI nested inside Harbor’s UI.
Introduce a new master role to project, the role’s permissions are more than developer and less than project admin.
Introduce harbor.yml as the replacement of harbor.cfg and refactor the prepare script to provide more flexibility to the installation process based on docker-compose
Enhancement of the Job Service engine to include webhook events, additional APIs for automation, and numerous bug fixes to improve the stability of the service.
Docker Registry upgraded to v2.7.1.
查看本地镜像
[root@client ~]# docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE [root@client ~]#
从官方镜像仓库下拉一个CentOS 7.6.1810版本的镜像
[root@client ~]# docker pull centos:7.6.1810 7.6.1810: Pulling from library/centos ac9208207ada: Pull complete Digest: sha256:0baec775bbf3e01cc770d5823c285b291b44ca05bb7c31b8674db5dae1d2aea9 Status: Downloaded newer image for centos:7.6.1810 [root@client ~]#
从官方镜像仓库下拉一个Ubuntu 18.04版本的镜像
[root@client ~]# docker pull ubuntu:18.04 18.04: Pulling from library/ubuntu 6abc03819f3e: Pull complete 05731e63f211: Pull complete 0bd67c50d6be: Pull complete Digest: sha256:f08638ec7ddc90065187e7eabdfac3c96e5ff0f6b2f1762cf31a4f49b53000a5 Status: Downloaded newer image for ubuntu:18.04 [root@client ~]#
查看本地镜像信息
[root@client ~]# docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE ubuntu 18.04 7698f282e524 12 days ago 69.9MB centos 7.6.1810 f1cb7c7d58b7 2 months ago 202MB [root@client ~]#
对默认的libarary公共镜像仓库进行存取操作
在项目中标记镜像: docker tag SOURCE_IMAGE[:TAG] hub.licensebox.cn/library/IMAGE[:TAG] 推送镜像到当前项目: docker push hub.licensebox.cn/library/IMAGE[:TAG]
查看已经重新打标记的镜像
[root@client ~]# docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE ubuntu 18.04 7698f282e524 12 days ago 69.9MB centos 7.6.1810 f1cb7c7d58b7 2 months ago 202MB hub.licensebox.cn/library/centos 7.6.1810 f1cb7c7d58b7 2 months ago 202MB [root@client ~]#
登录镜像仓库未配置证书时的错误提示(自签证书)
[root@client ~]# docker login hub.licensebox.cn Username: admin Password: Error response from daemon: Get https://hub.licensebox.cn/v2/: x509: certificate signed by unknown authority [root@client ~]#
建立证书配置目录(注意:必须提供server.cert文件,server.key文件和ca.crt文件)
[root@client ~]# mkdir -p /etc/docker/certs.d/hub.licensebox.cn/ [root@client ~]# mv ca.crt hub.licensebox.cn.key hub.licensebox.cn.cert /etc/docker/certs.d/hub.licensebox.cn/ [root@client ~]# ls /etc/docker/certs.d/hub.licensebox.cn/ ca.crt hub.licensebox.cn.cert hub.licensebox.cn.key [root@client ~]#
登录自建仓库
[root@client certs.d]# docker login hub.licensebox.cn Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@client certs.d]#
上传
上传完成
[root@client ~]# docker push hub.licensebox.cn/library/centos:7.6.1810 The push refers to repository [hub.licensebox.cn/library/centos] 89169d87dbe2: Pushed 7.6.1810: digest: sha256:747b2de199b6197a26eb1a24d69740d25483995842b2d2f75824095e9d1d19eb size: 529 [root@client ~]#
在VMware Harbor中查看已推送至镜像仓库的镜像
查看审计日志
删除本地镜像
[root@client ~]# docker image rm hub.licensebox.cn/library/centos:7.6.1810 Untagged: hub.licensebox.cn/library/centos:7.6.1810 Untagged: hub.licensebox.cn/library/centos@sha256:747b2de199b6197a26eb1a24d69740d25483995842b2d2f75824095e9d1d19eb [root@client ~]# docker image rm centos:7.6.1810 Untagged: centos:7.6.1810 Untagged: centos@sha256:0baec775bbf3e01cc770d5823c285b291b44ca05bb7c31b8674db5dae1d2aea9 Deleted: sha256:f1cb7c7d58b73eac859c395882eec49d50651244e342cd6c68a5c7809785f427 Deleted: sha256:89169d87dbe2b72ba42bfbb3579c957322baca28e03a1e558076542a1c1b2b4a [root@client ~]# docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE ubuntu 18.04 7698f282e524 12 days ago 69.9MB [root@client ~]#
尝试拉取之前推送的镜像
[root@client ~]# docker pull hub.licensebox.cn/library/centos:7.6.1810 7.6.1810: Pulling from library/centos ac9208207ada: Pull complete Digest: sha256:747b2de199b6197a26eb1a24d69740d25483995842b2d2f75824095e9d1d19eb Status: Downloaded newer image for hub.licensebox.cn/library/centos:7.6.1810 [root@client ~]# docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE ubuntu 18.04 7698f282e524 12 days ago 69.9MB hub.licensebox.cn/library/centos 7.6.1810 f1cb7c7d58b7 2 months ago 202MB [root@client ~]#
查看日志
登出镜像仓库
[root@client ~]# docker logout hub.licensebox.cn Removing login credentials for hub.licensebox.cn [root@client ~]#
作为公开仓库,即使不登录仍然可以访问(拉取)仓库中的镜像
日志显示匿名用户anonymous对仓库进行了拉取镜像的操作