17 KiB

Raw Blame History

OpenSearch

QuicStart

Install OpenSearch from an APT repository

Install the necessary packages.

sudo apt-get update && sudo apt-get -y install lsb-release ca-certificates curl gnupg2

Import the public GPG key. This key is used to verify that the APT repository is signed.

 curl -o- https://artifacts.opensearch.org/publickeys/opensearch.pgp | sudo gpg --dearmor --batch --yes -o /usr/share/keyrings/opensearch-keyring

Create an APT repository for OpenSearch

echo "deb [signed-by=/usr/share/keyrings/opensearch-keyring] https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-2.x.list

Create an APT repository for OpenSearch-Dashboard

echo "deb [signed-by=/usr/share/keyrings/opensearch-keyring] https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.x/apt stable main" | sudo tee /etc/apt/sources.list.d/opensearch-dashboards-2.x.list

Verify that the repository was created successfully.

sudo apt-get update

설치 가능 패키지 확인

apt-cache madison opensearch

확인결과

opensearch |     2.13.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
opensearch |     2.12.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
opensearch |     2.11.1 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
opensearch |     2.11.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
opensearch |     2.10.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
opensearch |      2.9.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
opensearch |      2.8.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
opensearch |      2.7.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
opensearch |      2.6.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages
opensearch |      2.5.0 | https://artifacts.opensearch.org/releases/bundle/opensearch/2.x/apt stable/main amd64 Packages

Choose the version of OpenSearch you want to install:

Unless otherwise indicated, the latest available version of OpenSearch is installed.

항상 최신 버전 설치

# For new installations of OpenSearch 2.12 and later, you must define a custom admin password in order to set up a demo security configuration.
# Use one of the following commands to define a custom admin password:
sudo env OPENSEARCH_INITIAL_ADMIN_PASSWORD=<custom-admin-password> apt-get install opensearch

# Use the following command for OpenSearch versions 2.11 and earlier:
sudo apt-get install opensearch

버전 지정 설치방법

# Specify the version manually using opensearch=<version>

# For new installations of OpenSearch 2.12 and later, you must define a custom admin password in order to set up a demo security configuration.
# Use one of the following commands to define a custom admin password:
sudo env OPENSEARCH_INITIAL_ADMIN_PASSWORD=<custom-admin-password> apt-get install opensearch=2.13.0

# Use the following command for OpenSearch versions 2.11 and earlier:
sudo apt-get install opensearch=2.13.0

MultiMaster Setting

/etc/opensearch/opensearch.yml

cluster.name: os_cluster_name
node.name: os1 # os2 , os3
node.roles: ["cluster_manager", "data"]
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
network.host: 0.0.0.0
http.port: 9200
transport.port: 9300
discovery.seed_hosts: ["10.10.2.30", "10.10.2.89", "10.10.2.97"]
cluster.initial_cluster_manager_nodes: ["10.10.2.30", "10.10.2.89", "10.10.2.97"]
node.max_local_storage_nodes: 3
plugins.security.disabled: true

Specify initial and maximum JVM heap sizes.

Open jvm.options

vi /etc/opensearch/jvm.options

Modify the values for initial and maximum heap sizes. As a starting point, you should set these values to half of the available system memory. For dedicated hosts this value can be increased based on your workflow requirements. As an example, if the host machine has 8 GB of memory, then you might want to set the initial and maximum heap sizes to 4 GB:

-Xms4g
-Xmx4g

Save your changes and close the file.

Naver Cloud LB 설정

멀티마스터 구성시 사설인증서로 Offloading 설정하여 진행. Network Proxy LoadBalancer에서 동작 검증완료.

내부 인증서 처리 방식이 좀 까다로워 테스트 하다 오프로딩으로 전환.

Reference

link1 국외자료 link2 국내자료 link3 공식자료

SingeNode 적용 예시

아래 내용참조. SingleNode는 SSL 설정도 까다롭지 않음.

vi /etc/opensearch/opensearch.yml

# ======================== OpenSearch Configuration =========================
#
# NOTE: OpenSearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.opensearch.org
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/opensearch
#
# Path to log files:
#
path.logs: /var/log/opensearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# OpenSearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of cluster-manager-eligible nodes:
#
#cluster.initial_cluster_manager_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Remote Store -----------------------------------
# Controls whether cluster imposes index creation only with remote store enabled
# cluster.remote_store.enabled: true
#
# Repository to use for segment upload while enforcing remote store for an index
# node.attr.remote_store.segment.repository: my-repo-1
#
# Repository to use for translog upload while enforcing remote store for an index
# node.attr.remote_store.translog.repository: my-repo-1
#
# ---------------------------------- Experimental Features -----------------------------------
# Gates the visibility of the experimental segment replication features until they are production ready.
#
#opensearch.experimental.feature.segment_replication_experimental.enabled: false
#
# Gates the functionality of a new parameter to the snapshot restore API
# that allows for creation of a new index type that searches a snapshot
# directly in a remote repository without restoring all index data to disk
# ahead of time.
#
#opensearch.experimental.feature.searchable_snapshot.enabled: false
#
#
# Gates the functionality of enabling extensions to work with OpenSearch.
# This feature enables applications to extend features of OpenSearch outside of
# the core.
#
#opensearch.experimental.feature.extensions.enabled: false
#
#
# Gates the concurrent segment search feature. This feature enables concurrent segment search in a separate
# index searcher threadpool.
#
#opensearch.experimental.feature.concurrent_segment_search.enabled: false

######## Start OpenSearch Security Demo Configuration ########
# 키정보는 openssl로 생성해서 교체해도 됨.
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true # 데모아니면 불필요
plugins.security.allow_default_init_securityindex: true #이건 모르겠음
plugins.security.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*"]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########
# Bind OpenSearch to the correct network interface. Use 0.0.0.0
# to include all available interfaces or specify an IP address
# assigned to a specific interface.
network.host: 0.0.0.0

# Unless you have already configured a cluster, you should set
# discovery.type to single-node, or the bootstrap checks will
# fail when you try to start the service.
discovery.type: single-node

# If you previously disabled the Security plugin in opensearch.yml,
# be sure to re-enable it. Otherwise you can skip this setting.
plugins.security.disabled: false # false가 기능활성임.

Password Change

multimaster 설정 과정에 대하여 기술함.

기존에 생성했던 인증서를 노드별로 적용하는 과정을 거쳐서 테스트를 진행하였음.

참고자료

사용했던 실제 스크립트

#!/bin/sh
# Root CA
openssl genrsa -out root-ca-key.pem 2048
openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=root.dns.a-record" -out root-ca.pem -days 730
# Admin cert
openssl genrsa -out admin-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in admin-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out admin-key.pem
openssl req -new -key admin-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=A" -out admin.csr
openssl x509 -req -in admin.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out admin.pem -days 730
# Node cert 1
openssl genrsa -out os1-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in os1-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out os1-key.pem
openssl req -new -key os1-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=os1.dns.a-record" -out os1.csr
echo 'subjectAltName=DNS:os1.dns.a-record' > os1.ext
openssl x509 -req -in os1.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out os1.pem -days 730 -extfile os1.ext
# Node cert 2
openssl genrsa -out os2-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in os2-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out os2-key.pem
openssl req -new -key os2-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=os2.dns.a-record" -out os2.csr
echo 'subjectAltName=DNS:os2.dns.a-record' > os2.ext
openssl x509 -req -in os2.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out os2.pem -days 730 -extfile os2.ext
# Node cert 3
openssl genrsa -out os3-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in os3-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out os3-key.pem
openssl req -new -key os3-key.pem -subj "/C=CA/ST=ONTARIO/L=TORONTO/O=ORG/OU=UNIT/CN=os3.dns.a-record" -out os3.csr
echo 'subjectAltName=DNS:os3.dns.a-record' > os3.ext
openssl x509 -req -in os3.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out os3.pem -days 730 -extfile os3.ext
# Cleanup
rm admin-key-temp.pem
rm admin.csr
rm os1-key-temp.pem
rm os1.csr
rm os1.ext
rm os2-key-temp.pem
rm os2.csr
rm os2.ext
rm os3-key-temp.pem
rm os3.csr
rm os3.ext

설정했던 opensearch.yml

cluster.name: my-application
node.name: os1
node.roles: ["cluster_manager", "data"]
path.data: /var/lib/opensearch
path.logs: /var/log/opensearch
network.host: 0.0.0.0
http.port: 9200
transport.port: 9300
discovery.seed_hosts: ["10.151.2.82", "10.151.2.69", "10.151.2.8"]
cluster.initial_cluster_manager_nodes: ["10.151.2.82", "10.151.2.69", "10.151.2.8"]
plugins.security.ssl.transport.pemcert_filepath: /etc/opensearch/ssh-files/os1.pem
plugins.security.ssl.transport.pemkey_filepath: /etc/opensearch/ssh-files/os1-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/opensearch/ssh-files/root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: /etc/opensearch/ssh-files/os1.pem
plugins.security.ssl.http.pemkey_filepath: /etc/opensearch/ssh-files/os1-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: /etc/opensearch/ssh-files/root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - 'CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.nodes_dn:
  - 'CN=os1.dns.a-record,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
  - 'CN=os2.dns.a-record,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
  - 'CN=os3.dns.a-record,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*"]
node.max_local_storage_nodes: 3
plugins.security.disabled: false

17 KiB Raw Blame History