Elasticsearch Service (AES) Cheat Sheet

Components

Domain = Elasticsearch cluster + options (instance type, counts, storage options, etc.)
Nodes = Instances
- Dedicated master node (optional) = cluster management nodes, do not process data
  - Required for cluster with 10+ instances
- Data node = stores data using EBS, processes data
- UltraWarm node = stores data in S3, processes data

At rest
- Can be enabled, uses KMS to manage key
- Indices, logs, swap, automatic snapshots, application directory are encrypted
  - NOT encrypted: manual snapshots, slow / error logs
- Only symmetric CMK is supported
- Packages (custom dictionaries) are always encrypted using AES-managed keys
Node-to-node
- Each AES domain always resides in its own VPC to prevent traffic interception, regardless of whether it uses VPC access
- Traffic within the AES VPC is not encrypted
- Can be enabled, so in-VPC traffic uses TLS
- Can only be enabled during creation, cannot be disabled after creation, create new domain and migrate if needed

Uses resource-based, identity-based or IP-based access control
Access can be controlled at cluster, index, document and field level
All requests to AES configuration API must be signed even if a domain is made to allow completely open access

Can be launched within a selected VPC and subnets
Places a private endpoint into one subnet, or multiple subnets in different AZ if multi-AZ is enabled
Also places ENI for each of the data nodes in the subnets
Kibana can only be accessed from within the VPC, using VPN or through a proxy
Need to reserve a lot of IPs for the data nodes and blue/green deployments

Preloaded for every domain at domain-endpoint/_plugin/kibana/
Access
- Public access = Use Cognito or IP based policy
- VPC access = Use Cognito or security group, to grant Internet access you need a proxy
Self-hosted Kibana can connect to AES domains

Use dedicated master nodes to offload management tasks and increase availability
- 1 master node = no backup when failed, 2 master nodes = necessary quorum of elect new master node, 3 master nodes = minimum
- Master nodes do not need to be as powerful as data processing nodes

Multi-AZ configuration can be enabled
- You also need to create replicas then AES will try to distribute the replica to another AZ

Configuration changes are deployed using blue/green strategy, a new environment is created to apply changes and traffic is switched after changes are applied
- During configuration change the nodes may be doubled
- You do not pay for extra resources provisioned during configuration change, unless you change to a new instance type, where you will pay for both instances

Uses S3 for read-only, “warm” data
Billed by time for the nodes, and by actual size of storage used
Performance is lower than “hot” nodes
- Hot nodes use EBS and instance store, while UltraWarm nodes uses S3 + sophisticated cache mechanism
UltraWarm and hot data can migrate to each other, and this process can be automated using Index State Management at no extra cost
UltraWarm nodes cannot be stopped and resumed; to disable UltraWarm, user must delete or migrate all indices to hot nodes

AES service upgrade
- Some are required and automatically applied, some are optional
- Official document recommends requesting service upgrade at a low traffic time which suggests service upgrade do not interrupt domain operation
Elasticsearch upgrade
- Manually triggered, AES will take snapshot before upgrade
- 15 minutes to several hours down time

By index snapshot
- Create a snapshot for self-hosted Elasticsearch cluster, upload to S3, grant S3 permissions for newly created AES domain, restore

You may only deploy a list of officially supported plugins, i.e. you cannot install your own plugins