5 月 282019
 

Elasticsearch 是一个分布式、RESTful 风格的搜索和数据分析引擎,能够解决不断涌现出的各种用例。作为 Elastic Stack 的核心,它集中存储您的数据,帮助您发现意料之中以及意料之外的情况。

https://www.elastic.co/guide/index.html

软件版本选择

Beats 7.1
APM Server 7.1
Elasticsearch 7.1
Elasticsearch Hadoop 7.1
Kibana 7.1
Logstash 7.1

Elastic Stack系列产品的安装顺序

Install the Elastic Stack products you want to use in the following order:
Elasticsearch
Kibana
Logstash
Beats
APM Server
Elasticsearch Hadoop

Elasticsearch在生产环境中的重要系统配置要求

Disable swapping(禁用交换分区)
Increase file descriptors(提升文件描述符到最大值(65535))
Ensure sufficient virtual memory(虚拟内存设置,临时更改sysctl -w vm.max_map_count=262144永久更改/etc/sysctl.conf)
Ensure sufficient threads(用户进程数必须大于等于4096/etc/security/limits.conf)
JVM DNS cache settings(详见jvm.options配置文件)
Temporary directory not mounted with noexec (临时目录挂载参数不包括noexec)

阿里云ECS镜像CentOS 7.x的资源限制

ulimit命令的资源限制参数

-a All current limits are reported
-b The maximum socket buffer size
-c The maximum size of core files created
-d The maximum size of a process's data segment
-e The maximum scheduling priority ("nice")
-f The maximum size of files written by the shell and its children
-i The maximum number of pending signals
-l The maximum size that may be locked into memory
-m The maximum resident set size (many systems do not honor this limit)
-n The maximum number of open file descriptors (most systems do not allow this value to be set)
-p The pipe size in 512-byte blocks (this may not be set)
-q The maximum number of bytes in POSIX message queues
-r The maximum real-time scheduling priority
-s The maximum stack size
-t The maximum amount of cpu time in seconds
-u The maximum number of processes available to a single user
-v The maximum amount of virtual memory available to the shell and, on some systems, to its children
-x The maximum number of file locks
-T The maximum number of threads

以非root用户权限运行elasticsearch服务
添加用户

[root@earth ~]# useradd elastic
[root@earth ~]# 

下载安装并修改权限

[root@earth ~]# curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.1.1-linux-x86_64.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 330M 100 330M 0 0 12.6M 0 0:00:26 0:00:26 --:--:-- 12.2M
[root@earth ~]#

[root@earth ~]# tar xzf elasticsearch-7.1.1-linux-x86_64.tar.gz 
[root@earth ~]# mv elasticsearch-7.1.1 /usr/local/
[root@earth ~]# cd /usr/local/
[root@earth local]# chown -R elastic.elastic elasticsearch-7.1.1/
[root@earth local]#

安装目录中的主要配置文件及用途

elasticsearch.yml for configuring Elasticsearch
jvm.options for configuring Elasticsearch JVM settings
log4j2.properties for configuring Elasticsearch logging

修改配置文件elasticsearch.yml配置文件

[root@earth ~]# cd /usr/local/elasticsearch-7.1.1/config/
[root@earth config]# vi elasticsearch.yml 
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /usr/local/elasticsearch-7.1.1/data
#
# Path to log files:
#
path.logs: /usr/local/elasticsearch-7.1.1/logs
#

# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 127.0.0.1
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#

切换到elastic用户并启动ES服务

[root@earth ~]# su - elastic
Last login: Wed May 29 16:56:38 CST 2019 on pts/0
[elastic@earth ~]$ cd /usr/local/elasticsearch-7.1.1/
[elastic@earth elasticsearch-7.1.1]$ ./bin/elasticsearch -d -p pid
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[elastic@earth elasticsearch-7.1.1]$

查看启动日志

查看端口监听

停止ES服务

[elastic@earth elasticsearch-7.1.1]$ pkill -F pid

验证Elasticsearch运行状态

[elastic@earth elasticsearch-7.1.1]$ curl localhost:9200
{
"name" : "earth",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "gAkz51cpSUuuL0MK5HBGCg",
"version" : {
"number" : "7.1.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "7a013de",
"build_date" : "2019-05-23T14:04:00.380842Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
[elastic@earth elasticsearch-7.1.1]$

Elasticsearch目录结构说明

5 月 282019
 

https://github.com/goharbor/harbor/releases

Support for OpenID Connect
OpenID Connect (OIDC) is an authentication layer on top of OAuth 2.0, allowing Harbor to verify the identity of users based on the authentication performed by an external authorization server or identity provider.

Robot accounts
Robot accounts can be configured to provide administrators with a token that can be granted appropriate permissions for pulling or pushing images. Harbor users can continue operating Harbor using their enterprise SSO credentials, and use robot accounts for CI/CD systems that perform Docker client commands.

Replication advancements
Harbor new version replication allows you to replicate your Harbor repository to and from non-Harbor registries. Harbor 1.8 expands on the Harbor-to-Harbor replication feature, adding the ability to replicate resources between Harbor and Docker Hub, Docker Registry, and Huawei Registry. This is enabled through both push and pull mode replication.

Additional Features
Harbor 1.8 brings numerous other capabilities for both administrators and end users:

Health check API, showing detailed status and health of all Harbor components.
Support for defining cron-based scheduled tasks in the Harbor UI. Administrators can now use cron strings to define the schedule of a job. Scan, garbage collection and replication jobs are all supported.
API explorer integration. End users can now explore and trigger Harbor’s API via the swagger UI nested inside Harbor’s UI.
Introduce a new master role to project, the role’s permissions are more than developer and less than project admin.
Introduce harbor.yml as the replacement of harbor.cfg and refactor the prepare script to provide more flexibility to the installation process based on docker-compose
Enhancement of the Job Service engine to include webhook events, additional APIs for automation, and numerous bug fixes to improve the stability of the service.
Docker Registry upgraded to v2.7.1.