Elasticsearch Service with Cognito -- how to access via Python HTTP request

Question

I recently setup an Elasticsearch service and configured it to use Cognito for identify management. I followed this guide and so far everything is working exactly as expected. I'm able to add new users as expected, and they can access Kibana as expected.

However, I would also like to use Python to interact with the Elasticsearch service. I've following this guide, but I get permission errors about not having the right access.

from elasticsearch import Elasticsearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth
import boto3

host = 'hostname.us-east-2.es.amazonaws.com/'
region = 'us-east-2'
service = 'es'
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)

es = Elasticsearch(
    hosts = [{'host': host, 'port': 443}],
    http_auth = awsauth,
    use_ssl = True,
    verify_certs = True,
    connection_class = RequestsHttpConnection
)

print(es.info())

AuthorizationException: AuthorizationException(403, 'security_exception', 'no permissions for [indices:admin/get] and User [name=arn:aws:iam::12345678:user/username, backend_roles=[], requestedTenant=null]')

I'm not sure if the issue is related to the way that I have Conginto configured, or if it's related to the way that I'm submitting this request. Any help on figuring out where my issue might be would be greatly appreciated.

question from:https://stackoverflow.com/questions/65623509/elasticsearch-service-with-cognito-how-to-access-via-python-http-request

Answer 1

It happened to me too. There are two ways I found, you have to create an IAM user for both of them.

• Open IAM console and give necessary policies to IAM user for reaching your elasticsearch domain (or I attached "AdministratorAccess" policy only (It gives all access of AWS)), then you should use credentials of this IAM account.
• Or open kibana -> security -> roles -> click all_access role -> mapped_users -> manage_mapping. Then you should add arn of your IAM user to backend roles. You have to use this user's credentials not "boto3.Session().get_credentials()" on your code to connect ES. I suggest the second one if you are not familiar with IAM roles and policies.