368 lines
17 KiB
Markdown
368 lines
17 KiB
Markdown
# OpenSearch
|
|
## QuicStart
|
|
## Install OpenSearch from an APT repository
|
|
### Install the necessary packages.
|
|
```bash
|
|
sudo apt-get update && sudo apt-get -y install lsb-release ca-certificates curl gnupg2
|
|
```
|
|
### Import the public GPG key. This key is used to verify that the APT repository is signed.
|
|
```bash
|
|
curl -o- https://artifacts.opensearch.org/publickeys/opensearch.pgp | sudo gpg --dearmor --batch --yes -o /usr/share/keyrings/opensearch-keyring
|
|
```
|
|
### Create an APT repository for OpenSearch
|
|
```bash
|
|
echo "deb [signed-by=/usr/share/keyrings/opensearch-keyring] https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-2.x.list
|
|
```
|
|
|
|
### Create an APT repository for OpenSearch-Dashboard
|
|
```bash
|
|
echo "deb [signed-by=/usr/share/keyrings/opensearch-keyring] https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-dashboards-2.x.list
|
|
```
|
|
|
|
### Verify that the repository was created successfully.
|
|
```bash
|
|
sudo apt-get update
|
|
```
|
|
설치 가능 패키지 확인
|
|
```bash
|
|
apt-cache madison opensearch
|
|
```
|
|
확인결과
|
|
```bash
|
|
opensearch | 2.13.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
|
|
opensearch | 2.12.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
|
|
opensearch | 2.11.1 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
|
|
opensearch | 2.11.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
|
|
opensearch | 2.10.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
|
|
opensearch | 2.9.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
|
|
opensearch | 2.8.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
|
|
opensearch | 2.7.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
|
|
opensearch | 2.6.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
|
|
opensearch | 2.5.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
|
|
```
|
|
|
|
### Choose the version of OpenSearch you want to install:
|
|
> Unless otherwise indicated, the latest available version of OpenSearch is installed.
|
|
|
|
항상 최신 버전 설치
|
|
```bash
|
|
# For new installations of OpenSearch 2.12 and later, you must define a custom admin password in order to set up a demo security configuration.
|
|
# Use one of the following commands to define a custom admin password:
|
|
sudo env OPENSEARCH_INITIAL_ADMIN_PASSWORD=<custom-admin-password> apt-get install opensearch
|
|
|
|
# Use the following command for OpenSearch versions 2.11 and earlier:
|
|
sudo apt-get install opensearch
|
|
```
|
|
|
|
버전 지정 설치방법
|
|
```bash
|
|
# Specify the version manually using opensearch=<version>
|
|
|
|
# For new installations of OpenSearch 2.12 and later, you must define a custom admin password in order to set up a demo security configuration.
|
|
# Use one of the following commands to define a custom admin password:
|
|
sudo env OPENSEARCH_INITIAL_ADMIN_PASSWORD=<custom-admin-password> apt-get install opensearch=2.13.0
|
|
|
|
# Use the following command for OpenSearch versions 2.11 and earlier:
|
|
sudo apt-get install opensearch=2.13.0
|
|
```
|
|
|
|
### MultiMaster Setting
|
|
```bash
|
|
/etc/opensearch/opensearch.yml
|
|
```
|
|
|
|
```yaml
|
|
cluster.name: os_cluster_name
|
|
node.name: os1 # os2 , os3
|
|
node.roles: ["cluster_manager", "data"]
|
|
path.data: /var/lib/opensearch
|
|
path.logs: /var/log/opensearch
|
|
network.host: 0.0.0.0
|
|
http.port: 9200
|
|
transport.port: 9300
|
|
discovery.seed_hosts: ["10.10.2.30", "10.10.2.89", "10.10.2.97"]
|
|
cluster.initial_cluster_manager_nodes: ["10.10.2.30", "10.10.2.89", "10.10.2.97"]
|
|
node.max_local_storage_nodes: 3
|
|
plugins.security.disabled: true
|
|
```
|
|
### Specify initial and maximum JVM heap sizes.
|
|
Open jvm.options
|
|
```bash
|
|
vi /etc/opensearch/jvm.options
|
|
```
|
|
Modify the values for initial and maximum heap sizes. As a starting point, you should set these values to half of the available system memory. For dedicated hosts this value can be increased based on your workflow requirements.
|
|
As an example, if the host machine has 8 GB of memory, then you might want to set the initial and maximum heap sizes to 4 GB:
|
|
```bash
|
|
-Xms4g
|
|
-Xmx4g
|
|
```
|
|
Save your changes and close the file.
|
|
### Naver Cloud LB 설정
|
|
멀티마스터 구성시 사설인증서로 Offloading 설정하여 진행.
|
|
Network Proxy LoadBalancer에서 동작 검증완료.
|
|
> 내부 인증서 처리 방식이 좀 까다로워 테스트 하다 오프로딩으로 전환.
|
|
|
|
## Reference
|
|
[link1](https://opster.com/guides/opensearch/opensearch-data-architecture/how-to-configure-opensearch-node-roles/) 국외자료
|
|
[link2](https://developer-jp.tistory.com/33) 국내자료
|
|
[link3](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/debian/) 공식자료
|
|
|
|
---
|
|
## SingeNode 적용 예시
|
|
아래 내용참조. SingleNode는 SSL 설정도 까다롭지 않음.
|
|
```yaml
|
|
vi /etc/opensearch/opensearch.yml
|
|
```
|
|
```yaml
|
|
# ======================== OpenSearch Configuration =========================
|
|
#
|
|
# NOTE: OpenSearch comes with reasonable defaults for most settings.
|
|
# Before you set out to tweak and tune the configuration, make sure you
|
|
# understand what are you trying to accomplish and the consequences.
|
|
#
|
|
# The primary way of configuring a node is via this file. This template lists
|
|
# the most important settings you may want to configure for a production cluster.
|
|
#
|
|
# Please consult the documentation for further information on configuration options:
|
|
# https://www.opensearch.org
|
|
#
|
|
# ---------------------------------- Cluster -----------------------------------
|
|
#
|
|
# Use a descriptive name for your cluster:
|
|
#
|
|
#cluster.name: my-application
|
|
#
|
|
# ------------------------------------ Node ------------------------------------
|
|
#
|
|
# Use a descriptive name for the node:
|
|
#
|
|
#node.name: node-1
|
|
#
|
|
# Add custom attributes to the node:
|
|
#
|
|
#node.attr.rack: r1
|
|
#
|
|
# ----------------------------------- Paths ------------------------------------
|
|
#
|
|
# Path to directory where to store the data (separate multiple locations by comma):
|
|
#
|
|
path.data: /var/lib/opensearch
|
|
#
|
|
# Path to log files:
|
|
#
|
|
path.logs: /var/log/opensearch
|
|
#
|
|
# ----------------------------------- Memory -----------------------------------
|
|
#
|
|
# Lock the memory on startup:
|
|
#
|
|
#bootstrap.memory_lock: true
|
|
#
|
|
# Make sure that the heap size is set to about half the memory available
|
|
# on the system and that the owner of the process is allowed to use this
|
|
# limit.
|
|
#
|
|
# OpenSearch performs poorly when the system is swapping the memory.
|
|
#
|
|
# ---------------------------------- Network -----------------------------------
|
|
#
|
|
# Set the bind address to a specific IP (IPv4 or IPv6):
|
|
#
|
|
#network.host: 192.168.0.1
|
|
#
|
|
# Set a custom port for HTTP:
|
|
#
|
|
#http.port: 9200
|
|
#
|
|
# For more information, consult the network module documentation.
|
|
#
|
|
# --------------------------------- Discovery ----------------------------------
|
|
#
|
|
# Pass an initial list of hosts to perform discovery when this node is started:
|
|
# The default list of hosts is ["127.0.0.1", "[::1]"]
|
|
#
|
|
#discovery.seed_hosts: ["host1", "host2"]
|
|
#
|
|
# Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:
|
|
#
|
|
#cluster.initial_cluster_manager_nodes: ["node-1", "node-2"]
|
|
#
|
|
# For more information, consult the discovery and cluster formation module documentation.
|
|
#
|
|
# ---------------------------------- Gateway -----------------------------------
|
|
#
|
|
# Block initial recovery after a full cluster restart until N nodes are started:
|
|
#
|
|
#gateway.recover_after_nodes: 3
|
|
#
|
|
# For more information, consult the gateway module documentation.
|
|
#
|
|
# ---------------------------------- Various -----------------------------------
|
|
#
|
|
# Require explicit names when deleting indices:
|
|
#
|
|
#action.destructive_requires_name: true
|
|
#
|
|
# ---------------------------------- Remote Store -----------------------------------
|
|
# Controls whether cluster imposes index creation only with remote store enabled
|
|
# cluster.remote_store.enabled: true
|
|
#
|
|
# Repository to use for segment upload while enforcing remote store for an index
|
|
# node.attr.remote_store.segment.repository: my-repo-1
|
|
#
|
|
# Repository to use for translog upload while enforcing remote store for an index
|
|
# node.attr.remote_store.translog.repository: my-repo-1
|
|
#
|
|
# ---------------------------------- Experimental Features -----------------------------------
|
|
# Gates the visibility of the experimental segment replication features until they are production ready.
|
|
#
|
|
#opensearch.experimental.feature.segment_replication_experimental.enabled: false
|
|
#
|
|
# Gates the functionality of a new parameter to the snapshot restore API
|
|
# that allows for creation of a new index type that searches a snapshot
|
|
# directly in a remote repository without restoring all index data to disk
|
|
# ahead of time.
|
|
#
|
|
#opensearch.experimental.feature.searchable_snapshot.enabled: false
|
|
#
|
|
#
|
|
# Gates the functionality of enabling extensions to work with OpenSearch.
|
|
# This feature enables applications to extend features of OpenSearch outside of
|
|
# the core.
|
|
#
|
|
#opensearch.experimental.feature.extensions.enabled: false
|
|
#
|
|
#
|
|
# Gates the concurrent segment search feature. This feature enables concurrent segment search in a separate
|
|
# index searcher threadpool.
|
|
#
|
|
#opensearch.experimental.feature.concurrent_segment_search.enabled: false
|
|
|
|
######## Start OpenSearch Security Demo Configuration ########
|
|
# 키정보는 openssl로 생성해서 교체해도 됨.
|
|
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
|
|
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
|
|
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
|
|
plugins.security.ssl.transport.enforce_hostname_verification: false
|
|
plugins.security.ssl.http.enabled: true
|
|
plugins.security.ssl.http.pemcert_filepath: esnode.pem
|
|
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
|
|
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
|
|
plugins.security.allow_unsafe_democertificates: true # 데모아니면 불필요
|
|
plugins.security.allow_default_init_securityindex: true #이건 모르겠음
|
|
plugins.security.authcz.admin_dn:
|
|
- CN=kirk,OU=client,O=client,L=test, C=de
|
|
|
|
plugins.security.audit.type: internal_opensearch
|
|
plugins.security.enable_snapshot_restore_privilege: true
|
|
plugins.security.check_snapshot_restore_write_privileges: true
|
|
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
|
|
plugins.security.system_indices.enabled: true
|
|
plugins.security.system_indices.indices: [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*"]
|
|
node.max_local_storage_nodes: 3
|
|
######## End OpenSearch Security Demo Configuration ########
|
|
# Bind OpenSearch to the correct network interface. Use 0.0.0.0
|
|
# to include all available interfaces or specify an IP address
|
|
# assigned to a specific interface.
|
|
network.host: 0.0.0.0
|
|
|
|
# Unless you have already configured a cluster, you should set
|
|
# discovery.type to single-node, or the bootstrap checks will
|
|
# fail when you try to start the service.
|
|
discovery.type: single-node
|
|
|
|
# If you previously disabled the Security plugin in opensearch.yml,
|
|
# be sure to re-enable it. Otherwise you can skip this setting.
|
|
plugins.security.disabled: false # false가 기능활성임.
|
|
```
|
|
|
|
---
|
|
|
|
## Password Change
|
|
multimaster 설정 과정에 대하여 기술함.
|
|
|
|
> 기존에 생성했던 인증서를 노드별로 적용하는 과정을 거쳐서 테스트를 진행하였음.
|
|
|
|
[참고자료](https://opensearch.org/docs/latest/security/configuration/generate-certificates/)
|
|
|
|
사용했던 실제 스크립트
|
|
```sh
|
|
#!/bin/sh
|
|
# Root CA
|
|
openssl genrsa -out root-ca-key.pem 2048
|
|
openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=root.dns.a-record" -out root-ca.pem -days 730
|
|
# Admin cert
|
|
openssl genrsa -out admin-key-temp.pem 2048
|
|
openssl pkcs8 -inform PEM -outform PEM -in admin-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out admin-key.pem
|
|
openssl req -new -key admin-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=A" -out admin.csr
|
|
openssl x509 -req -in admin.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out admin.pem -days 730
|
|
# Node cert 1
|
|
openssl genrsa -out os1-key-temp.pem 2048
|
|
openssl pkcs8 -inform PEM -outform PEM -in os1-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out os1-key.pem
|
|
openssl req -new -key os1-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=os1.dns.a-record" -out os1.csr
|
|
echo 'subjectAltName=DNS:os1.dns.a-record' > os1.ext
|
|
openssl x509 -req -in os1.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out os1.pem -days 730 -extfile os1.ext
|
|
# Node cert 2
|
|
openssl genrsa -out os2-key-temp.pem 2048
|
|
openssl pkcs8 -inform PEM -outform PEM -in os2-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out os2-key.pem
|
|
openssl req -new -key os2-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=os2.dns.a-record" -out os2.csr
|
|
echo 'subjectAltName=DNS:os2.dns.a-record' > os2.ext
|
|
openssl x509 -req -in os2.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out os2.pem -days 730 -extfile os2.ext
|
|
# Node cert 3
|
|
openssl genrsa -out os3-key-temp.pem 2048
|
|
openssl pkcs8 -inform PEM -outform PEM -in os3-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out os3-key.pem
|
|
openssl req -new -key os3-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=os3.dns.a-record" -out os3.csr
|
|
echo 'subjectAltName=DNS:os3.dns.a-record' > os3.ext
|
|
openssl x509 -req -in os3.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out os3.pem -days 730 -extfile os3.ext
|
|
# Cleanup
|
|
rm admin-key-temp.pem
|
|
rm admin.csr
|
|
rm os1-key-temp.pem
|
|
rm os1.csr
|
|
rm os1.ext
|
|
rm os2-key-temp.pem
|
|
rm os2.csr
|
|
rm os2.ext
|
|
rm os3-key-temp.pem
|
|
rm os3.csr
|
|
rm os3.ext
|
|
```
|
|
|
|
### 설정했던 opensearch.yml
|
|
```bash
|
|
cluster.name: my-application
|
|
node.name: os1
|
|
node.roles: ["cluster_manager", "data"]
|
|
path.data: /var/lib/opensearch
|
|
path.logs: /var/log/opensearch
|
|
network.host: 0.0.0.0
|
|
http.port: 9200
|
|
transport.port: 9300
|
|
discovery.seed_hosts: ["10.151.2.82", "10.151.2.69", "10.151.2.8"]
|
|
cluster.initial_cluster_manager_nodes: ["10.151.2.82", "10.151.2.69", "10.151.2.8"]
|
|
plugins.security.ssl.transport.pemcert_filepath: /etc/opensearch/ssh-files/os1.pem
|
|
plugins.security.ssl.transport.pemkey_filepath: /etc/opensearch/ssh-files/os1-key.pem
|
|
plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/opensearch/ssh-files/root-ca.pem
|
|
plugins.security.ssl.transport.enforce_hostname_verification: false
|
|
plugins.security.ssl.http.enabled: true
|
|
plugins.security.ssl.http.pemcert_filepath: /etc/opensearch/ssh-files/os1.pem
|
|
plugins.security.ssl.http.pemkey_filepath: /etc/opensearch/ssh-files/os1-key.pem
|
|
plugins.security.ssl.http.pemtrustedcas_filepath: /etc/opensearch/ssh-files/root-ca.pem
|
|
plugins.security.allow_unsafe_democertificates: true
|
|
plugins.security.allow_default_init_securityindex: true
|
|
plugins.security.authcz.admin_dn:
|
|
- 'CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
|
|
plugins.security.nodes_dn:
|
|
- 'CN=os1.dns.a-record,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
|
|
- 'CN=os2.dns.a-record,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
|
|
- 'CN=os3.dns.a-record,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
|
|
plugins.security.audit.type: internal_opensearch
|
|
plugins.security.enable_snapshot_restore_privilege: true
|
|
plugins.security.check_snapshot_restore_write_privileges: true
|
|
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
|
|
plugins.security.system_indices.enabled: true
|
|
plugins.security.system_indices.indices: [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*"]
|
|
node.max_local_storage_nodes: 3
|
|
plugins.security.disabled: false
|
|
```
|