Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.omni.co/llms.txt

Use this file to discover all available pages before exploring further.

Omni’s Athena integration supports two types of authentication:
  • Access key - Uses an IAM user to authenticate to Amazon Web Services (AWS) by providing Omni with the user’s Access key ID and Secret access key
  • Cross-account role - Enables Omni services to assume an IAM role in your account

​
Requirements

To follow the steps in this guide, you’ll need:
  • An existing Athena database. Additionally, this database should have an S3 bucket configured to store query results. Refer to the AWS documentation to get started.
  • Omni Organization Admin permissions. This is required to create the Athena connection in Omni.
Depending on the type of authentication you want to use, you’ll also need permissions in AWS that:
  • Allow you to create access keys
  • Allow you to create IAM permission policies and roles

​
Option 1: Access key authentication

To use this method of authentication, you’ll need to create an access key in the AWS console and then add it to the Athena configuration in Omni.
1

Create the access key in AWS

To complete this step, you need permissions in AWS that allow you to create access keys.
In the AWS console, follow these instructions to create an access key. Note: The key will only display once after it’s created - make sure to copy it immediately.
2

Create the connection in Omni

  1. In Omni, click Settings > Connections.
  2. Click the Athena option.
  3. On the connection setup page, fill in the connection details:
    ​
    S3 Output Bucket
    required
    The Amazon S3 bucket used to store query results for the Athena database
    ​
    Data Catalog
    required
    The lowercase name of the data catalog to connect to
    ​
    Include Other Data Catalogs
    A comma-delimited list of data catalogs you want to generate models for, ex: catalog1, catalog2
    ​
    Authentication Type
    required
    Select AWS Access Key
    ​
    AWS Key ID
    required
    The ID of the AWS access key, which you created in the previous step
    ​
    AWS Secret Access Key
    required
    The secret access key for AWS authentication, which you created in the previous step
    ​
    Region
    default:"us-east-1"
    The AWS region the database is in. Defaults to us-east-1.
  4. When finished, click Create connection.

​
Option 2: Cross-account role authentication

Cross-account role authentication allows you to specify an IAM role for Omni to assume when connecting to Athena.
1

Open the Athena setup page in Omni

Leave this page open for now - you’ll need it in the next step.
2

Create an IAM policy

To complete this step, you need permissions in AWS that allow you to create IAM policies.
In this step, you’ll create an IAM policy that grants the required permissions to the IAM role Omni will assume.
  1. Sign into your AWS console.
  2. Navigate to IAM, then click Roles in the left navigation.
  3. On the page that displays, click Create policy.
  4. In the Policy editor section, click the toggle to JSON: AWS IAM policy editor
  5. In the policy editor, paste the following policy. Replace <NAME-OF-ATHENA-QUERY-RESULTS-BUCKET> with the name of the S3 bucket used to store Athena query results: 
    {
 "Version": "2012-10-17",
 "Statement": [
  {
   "Sid": "BasePermissionsForAllS3Buckets",
   "Effect": "Allow",
   "Action": [
    "s3:ListAllMyBuckets",
    "s3:ListBucket",
    "s3:GetBucketLocation"
   ],
   "Resource": "*"
  },
  {
   "Sid": "BaseAndQueryPermissionsForQueryResultBucket",
   "Effect": "Allow",
   "Action": [
    "s3:GetObject",
    "s3:ListBucket",
    "s3:GetBucketLocation",
    "s3:ListBucket",
    "s3:ListBucketMultipartUploads",
    "s3:ListMultipartUploadParts",
    "s3:AbortMultipartUpload",
    "s3:PutObject"
   ],
   "Resource": [
    "arn:aws:s3:::<NAME-OF-ATHENA-QUERY-RESULTS-BUCKET>",
    "arn:aws:s3:::*/*"
   ]
  },
  {
   "Sid": "GluePermissions",
   "Effect": "Allow",
   "Action": [
    "glue:GetTables",
    "glue:GetDatabases",
    "glue:GetDatabase",
    "glue:GetTable"
   ],
   "Resource": "*"
  },
  {
   "Sid": "AthenaPermissions",
   "Effect": "Allow",
   "Action": [
    "athena:StartQueryExecution",
    "athena:GetQueryResults",
    "athena:GetQueryResultsStream",
    "athena:ListDatabases",
    "athena:GetQueryExecution",
    "athena:ListTableMetadata"
   ],
   "Resource": "*"
  }
 ]
}
    If your data is stored in a separate bucket than the query results, you’ll need to add an additional permissions block to the policy.Replace <NAME-OF-BUCKET-CONTAINING-DATA> with the name of the bucket that contains the data:
    Additional permissions for different data & query buckets
    {
  "Sid": "S3AccessForDataBuckets",
  "Effect": "Allow",
  "Action": [
    "s3:GetObject",
    "s3:ListBucket"
  ],
  "Resource": [
    "arn:aws:s3:::<NAME-OF-BUCKET-CONTAINING-DATA>",
    "arn:aws:s3:::<NAME-OF-BUCKET-CONTAINING-DATA>/*"
  ]
}
  6. Click Next.
  7. On the Policy details page, enter a name and description for the policy.
  8. When finished, click Create policy.
3

Create the IAM role

To complete this step, you need permissions in AWS that allow you to create IAM roles.
In this step, you’ll create the IAM role and attach the permissions policy you created in step 2 to it.
  1. You should still be in the IAM section of the AWS console. If not, navigate there.
  2. Click Roles in the left navigation.
  3. On the page that displays, click Create role.
  4. Define the trusted entity for the role as follows:
    • Trusted entity type - Select AWS account
    • In An AWS account, click Another AWS account
    • Account ID field - Paste the AWS Account ID value from the Omni setup page
    • In Options, check Require external ID
    • External ID - Enter a placeholder value - we used we-will-add-this-later in the below image. You will update this after you save the connection in Omni!
    At this point, the page should look like this: Athena role entity When finished, click Next.
  5. On the Add permissions page, locate and select the policy you created in the previous section.
  6. Click Next.
  7. In the Role details section, enter a name for the role.
  8. Review the selections and click Create role when finished.
After the role is created, open its details page in AWS and locate the ARN near the top of the page:Athena role ARNLeave this page open - you’ll need it to complete the setup.
4

Create the connection in Omni

Navigate back to the Athena setup page in Omni and fill in the connection details:
​
S3 Output Bucket
required
The Amazon S3 bucket used to store query results for the Athena database
​
Data Catalog
required
The lowercase name of the data catalog to connect to
​
Include Other Data Catalogs
A comma-delimited list of data catalogs you want to generate models for, ex: catalog1, catalog2
​
Authentication Type
required
Select AWS Cross Account Role
​
AWS Role ARN
required
The ARN of the IAM role for Omni to assume, which you created in step 3
​
Region
default:"us-east-1"
The AWS region the database is in. Defaults to us-east-1.
When finished, click Create connection.
After you save the connection, you’ll likely see an error similar to System message: AWS Role must be configured to check for an External ID - don’t worry! You’re going to address this in the next step.
5

Add the Omni external ID to the IAM role

  1. In AWS, navigate back to the IAM role page.
  2. On the role’s details page, click the Trust relationships tab in the middle of the page: Athena role trust relationships
  3. Click Edit trust policy.
  4. Locate the sts:ExternalId property. Replace its value with the AWS External ID value from the Omni setup page:
    Add AWS external ID from Omni
    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::277577138649:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<REPLACE-WITH-OMNI-EXTERNAL-ID>"
        }
      }
    }
  ]
}
  5. Click Update policy when finished.
Lastly, navigate back to the Omni setup page and click Update and test connection. At this point, the connection should return a Success! message.

​
What’s next?

Now that your database is set up, you can:

​
Troubleshooting

Applicable if using cross-account role authentication. If the S3 bucket containing your data is different than the bucket Athena is configured to use for query results, you may encounter an error in Omni.Adding a block like the following to the IAM permissions policy attached to the IAM role you created can help resolve the issue.Replace <NAME-OF-BUCKET-CONTAINING-DATA> with the name of the bucket that contains the data:
Additional permissions for different data & query buckets
{
  "Sid": "S3AccessForDataBuckets",
  "Effect": "Allow",
  "Action": [
    "s3:GetObject",
    "s3:ListBucket"
  ],
  "Resource": [
    "arn:aws:s3:::<NAME-OF-BUCKET-CONTAINING-DATA>",
    "arn:aws:s3:::<NAME-OF-BUCKET-CONTAINING-DATA>/*"
  ]
}