Big Data

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Amazon QuickSight is a scalable, serverless, embeddable, machine learning-powered business intelligence (BI) service built for the cloud. QuickSight lets you easily create and publish interactive BI dashboards that include Machine Learning-powered insights. QuickSight dashboards can be accessed from any device, and seamlessly embedded into your applications, portals, and websites.

QuickSight offers several sources for data, including but not limited to Amazon Athena, Amazon Redshift, Amazon Simple Storage Service (Amazon S3), and Snowflake. This post presents solutions to enable you to set up Snowflake as a data source for QuickSight regardless of your network configuration requirements.

We cover the following configurations of Snowflake as a data source for QuickSight:

  • QuickSight connection to Snowflake via AWS PrivateLink
  • QuickSight connection to Snowflake via AWS PrivateLink and virtual private cloud (VPC) peering (same Region)
  • QuickSight connection to Snowflake via AWS PrivateLink and VPC peering (cross-Region)
  • QuickSight connection to Snowflake (public network)

Prerequisites

To complete this solution, you need the following:

QuickSight connection to Snowflake via AWS PrivateLink

First, we show you how to connect to Snowflake with QuickSight over AWS PrivateLink. The following diagram illustrates the solution architecture.

Set up the Snowflake AWS PrivateLink integration

To start, we walk through enabling AWS PrivateLink for your Snowflake account. This includes locating resources in your AWS account, access to the Snowflake UI, and creating a support case with Snowflake.

  1. Identify the VPC you want to use to set up the AWS PrivateLink integration. To do so, retrieve a list of VPCs from the command line, then retrieve the VpcId element from the resulting JSON object for the desired VPC. See the following code:
aws ec2 describe-vpcs --output json

  1. Retrieve your AWS account ID. This post assumes that the account you’re targeting is your default account on your AWS CLI configuration.
aws sts get-caller-identity --output json

  1. If you’re setting up multiple accounts, repeat these steps for all accounts and VPCs (this post assumes you’re setting up a single account and VPC and will use this as the context moving forward).
  2. Contact Snowflake Support with your AWS account ID, VPC ID, and the corresponding account URL you use to access Snowflake (for example, <account id>.snowflakecomputing.com).

Enabling AWS PrivateLink for your Snowflake account can take up to two business days.

  1. After AWS PrivateLink is enabled, retrieve the AWS PrivateLink configuration for your Region by running the following command in a Snowflake worksheet, then retrieve the values for privatelink-account-url and privatelink_ocsp-url from the resulting JSON object. Examples of each value are as follows:
select SYSTEM$GET_PRIVATELINK_CONFIG();

privatelink-vpce-id: com.amazonaws.vpce.<region_id>.vpce-svc-xxxxxxxxxxxxxxxxx
privatelink-account-url: xxxxxxxx.<region>.privatelink.snowflakecomputing.com
privatelink_ocsp-url: ocsp.xxxxxxxx.<region>.privatelink.snowflakecomputing.com

  1. Store these values in a text editor for later use.

Next, we configure the VPC endpoint on the Amazon Virtual Private Cloud (Amazon VPC) console create all the required security groups.

  1. On the Amazon VPC console, choose Endpoints in the navigation menu.
  2. Choose Create endpoint.
  3. Select Find AWS Service by Name.
  4. For Service Name, enter the value for privatelink-vpce-id that we retrieved earlier.
  5. Choose Verify.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

A green alert with “Service Name Found” appears and the VPC and subnet options automatically expand.

Depending on your targeted Region, your resulting screen may show another Region name.

  1. Choose the same VPC ID that you sent to Snowflake.
  2. Select the subnets where you want to create endpoints.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

AWS recommends using more than one subnet for high availability.

  1. For Security group, choose Create a new security group.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

This opens the Security groups page on the Amazon VPC console in a new tab.

  1. Choose Create security group.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Give your new security group a name (for example, quicksight-doc-snowflake-privatelink-connection) and description.
  2. Choose the VPC ID you used in previous steps.

Next, you add two rules that allow traffic from within your VPC to this VPC endpoint.

  1. Retrieve the CIDR block for your targeted VPC:
aws ec2 describe-vpcs --vpc-ids vpc-xxxxxxxxxxxxxxxxx | jq -r '.Vpcs[].CidrBlock'

  1. Choose Add rule in the Inbound rules
  2. Choose HTTPS for the type, leave the source as Custom, and enter the value retrieved from the preceding describe-vpcs call (for example, 10.0.0.0/16).
  3. Choose Add rule in the Inbound rules
  4. Choose HTTP for the type, leave the source as Custom, and enter the value retrieved from the preceding describe-vpcs
  5. Choose Create security group.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Retrieve the security group ID from the newly created security group.
  2. On the VPC endpoint configuration page, remove the default security group.
  3. Search for and select the new security group ID.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Create endpoint.

You’re redirected to a page that has a link to your VPC endpoint configuration, specified by the VPC ID. The next page has a link to view the configuration in full.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Retrieve the topmost record in the DNS names list.

This can be differentiated from other DNS names because it only includes the Region name (such as us-west-2), and no Availability Zone letter notation (such as us-west-2a).

  1. Store this record in a text editor for later use.

Configure DNS for Snowflake endpoints in your VPC

To configure your Snowflake endpoints, complete the following steps:

  1. On the Route 53 console, choose Hosted Zones in the navigation pane.
  2. Choose Create hosted zone.
  3. For Domain name, enter the value you stored for privatelink-account-url from the previous steps.

In this field, we remove the Snowflake account ID from the DNS name and only use the value starting with the Region identifier (for example, <region>.privatelink.snowflakecomputing.com). We create a resource record set later for the subdomain.

  1. For Type, select Private hosted zone.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Your Region code may not be us-west-2; reference the DNS name returned to you by Snowflake.

  1. In the VPCs to associate with the hosted zone section, choose the Region in which your VPC is located and the VPC ID used in previous steps.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Create hosted zone.

Next. we create two records: one for privatelink-account-url and one for privatelink_ocsp-url.

  1. On the Hosted zones page, choose Create record set.
  2. For Record name, enter your Snowflake account ID (the first eight characters in privatelink-account-url).
  3. For Record type, choose CNAME.
  4. For Value, enter the DNS name for the Regional VPC endpoint we retrieved in the previous section.
  5. Choose Create records.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Repeat these steps for the OCSP record we notated as privatelink-ocsp-url earlier, starting with ocsp through the eight-character Snowflake ID for the record name (for example, ocsp.xxxxxxxx).

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Configure a Route 53 resolver inbound endpoint for your VPC

QuickSight doesn’t use the standard AWS resolver (the VPC’s .2 resolver). To resolve private DNS from QuickSight, you need to set up Route 53 resolver endpoints.

First, we create a security group for the Route 53 resolver inbound endpoint.

  1. On the Security groups page of the Amazon VPC console, choose Create security group.
  2. Enter a name for your security group (for example, quicksight-doc-route53-resolver-sg) and a description.
  3. Choose the VPC ID used in previous steps.
  4. Create rules that allow for DNS (Port 53) over UDP and TCP from within the VPC CIDR block.
  5. Choose Create security group.
  6. Note the security group ID, because we now add a rule to allow traffic to the VPC endpoint security group.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Now we create the Route 53 resolver inbound endpoint for our VPC.

  1. On the Route 53 console, choose Inbound endpoint in the navigation pane.
  2. Choose Create inbound endpoint.
  3. For Endpoint name, enter a name (for example, quicksight-inbound-resolver).
  4. For VPC in the Region, choose the VPC ID used in previous steps.
  5. For Security group for the endpoint, choose the security group ID you saved earlier.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. In the IP address section, choose two Availability Zones and subnets, and leave Use an IP address that is selected automatically selected.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Submit.
  2. Choose the inbound endpoint after it’s created and take note of the two IP addresses for the resolvers.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Connect a VPC to QuickSight

To connect a VPC to QuickSight, complete the following steps:

  1. On the Security groups page of the Amazon VPC console, choose Create security group.
  2. Enter a name (for example, quicksight-snowflake-privatelink-sg) and a description.
  3. Choose the VPC ID used in previous steps.

Security groups for QuickSight are different from other security groups in that they are stateless, rather than stateful. This means you must explicitly allow return traffic from the targeted security group. The inbound rule in your security group must allow traffic on all ports. It needs to do this because the destination port number of any inbound return packets is set to a randomly allocated port number. For more information, see Inbound Rules.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Create security group.
  2. Take note of the security group ID, because we now add a rule to allow traffic to the VPC endpoint security group.
  3. On the Security groups page, search for the security group ID that is used for the VPC endpoint.
  4. Choose Edit inbound rules.
  5. Add rules for both HTTPS and HTTP traffic, using the security group ID for the security group you created as the source.
  6. Choose Save rules.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Next, we move to the QuickSight console to configure the VPC connection.

  1. Navigate to the QuickSight console.
  2. Choose the user name and choose Manage QuickSight.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. In the navigation pane, choose Manage VPC connections.
  2. Choose Add a VPC connection.
  3. For VPC connection name, enter a name (for example, snowflake-privatelink).
  4. For VPC ID, choose the VPC used in previous steps.
  5. For Subnet ID, choose one of the subnets that has a VPC endpoint, as specified when you created the endpoint earlier.
  6. For Security group ID, enter the ID of the security group you created.
  7. For DNS resolver endpoints, enter the two IPs for the inbound resolver endpoint you created earlier.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Create.

Set up a Snowflake data source through the VPC

To set up a Snowflake data source, complete the following steps.

  1. On the QuickSight console, choose Datasets in the navigation page.
  2. Choose New dataset.
  3. Choose the Snowflake option.
  4. For Data source name, enter a name (for example, snowflake).
  5. For Connection type¸ choose the VPC connection you created earlier (snowflake-privatelink).
  6. For Database server, enter privatelink-account-url.
  7. For Database name, enter the name of your database.
  8. For Warehouse, enter the name of a running Snowflake warehouse.
  9. For Username, enter your Snowflake username.
  10. For Password, enter your Snowflake password.
  11. Choose Validate.
  12. Upon successful validation, choose Create data source.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Create your first QuickSight dashboard

In this section, we cover creating a dataset in QuickSight, then using this data in a visualization. We’re using a dummy dataset that has information about fictional employees.

  1. For Schema, choose your schema.
  2. For Tables, select your tables.
  3. Choose Select.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

In the Finish dataset creation section, you can determine if QuickSight imports your dataset into SPICE to improve query performance or directly queries your data each time a dashboard is loaded. For more information about SPICE, see Importing Data into SPICE.

  1. For this post, we select Import to SPICE for quicker analytics.
  2. Choose Visualize.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Now that we have the schema, table, and SPICE configuration for the dataset, we can create our first visualization.

  1. Choose a field from the available fields list. For this post, we choose City.
  2. Choose a visualization in the Visual types

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

This only scratches the surface of the visualization capabilities of QuickSight. For more information, see Working with Amazon QuickSight Visuals.

Next, we cover a network configuration that allows for QuickSight to be connected to one VPC with AWS PrivateLink in another VPC, and use VPC peering to allow QuickSight to use the AWS PrivateLink connection.

QuickSight connection to Snowflake via AWS PrivateLink and VPC peering within the same Region

In this section, we show you how to connect to Snowflake with QuickSight with two VPCs peered and AWS PrivateLink. The following diagram illustrates the solution architecture.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Set up VPC peering

First, we create the VPC peering connection from the requesting VPC.

  1. On the Peering connections page of the Amazon VPC console, choose Create peering connection.
  2. For Select a local VPC to peer with, choose the VPC in which you configured your Snowflake AWS PrivateLink connection.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. In the Select another VPC to peer with section, leave the default options for Account and Region (My account and This Region, respectively).
  2. For VPC (Accepter), choose the VPC where your QuickSight is connected to.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Create peering connection.

Next, we accept the VPC connection from the accepting VPC.

  1. On the Peering connections page, select the connection you created.
  2. On the Actions menu, choose Accept.
  3. Review the information about the request. If everything looks correct, choose Yes, Accept.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Next, we configure DNS to resolve between the two VPCs.

  1. On the Peering connections page, choose your new peering connection.
  2. On the DNS tab, check if the two options show as Disabled.

If they’re enabled, you can skip to the steps on creating route tables.

  1. On the Actions menu, choose Edit DNS Settings.

This requires your VPC to have DNS host name and resolution enabled.

  1. Select both check boxes to allow DNS to resolve from both the acceptor and requestor VPCs.
  2. Choose Save.

Next, create the route table entry to allow for routes to propagate between the two VPCs.

  1. On the Route tables page, choose the route tables in your requesting VPC.
  2. On the Route tab, choose Edit routes.
  3. Add a route for the CIDR block that your peered VPC uses (for this post, 172.31.0.0/16).
  4. Choose Save routes.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Repeat for the route tables in your accepter VPC.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Configure DNS in the accepter VPC

In this section, we associate the accepter VPC that with the same private hosted zone as the requester VPC (<region>.privatelink.snowflakecomputing.com).

  1. On the Route 53 console, choose Hosted zones in the navigation pane.
  2. Select the hosted zone <region>.privatelink.snowflakecomputing.com and choose Edit.
  3. In the VPCs to associate with the hosted zone section, choose Add VPC.
  4. Choose the Region and VPC ID associated with the accepter VPC.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Save changes.

Configure Route 53 resolver inbound endpoints in the accepter VPC

To configure your Route 53 resolver inbound endpoints, complete the following steps:

  1. On the Security groups page of the Amazon VPC console, choose Create security group.
  2. Enter a name (for example, quicksight-doc-route53-resolver-sg) and a description.
  3. Choose the VPC ID used in previous steps.
  4. Create rules that allow for DNS (port 53) over UDP and TCP from within the VPC CIDR block (for this post, 172.31.0.0/16).
  5. Choose Create security group.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Take note of the security group ID, because we now add a rule to allow traffic to the VPC endpoint security group.

Next, we set up the Route 53 inbound endpoint for this VPC.

  1. On the Route 53 console, choose Inbound endpoint in the navigation pane.
  2. Choose Create inbound endpoint.
  3. Enter a name for the endpoint (for example, quicksight-inbound-resolver).
  4. For VPC in the Region, choose the VPC ID for the accepter VPC.
  5. For Security group, choose the security group ID you saved earlier.
  6. In the IP Address section, select two Availability Zones and subnets, and leave Use an IP address that is selected automatically
  7. Choose Submit.
  8. Choose the inbound endpoint after it’s created.
  9. After the inbound endpoint has provisioned, note the two IP addresses for the resolvers.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Connect the accepter VPC to QuickSight

To start, we need to create a security group for QuickSight to allow traffic to the Route 53 resolver inbound endpoints, the VPC endpoint for AWS PrivateLink, and traffic within the local network.

  1. On the Security groups page of the Amazon VPC console, choose Create security group.
  2. Enter a name (for example, quicksight-snowflake-privatelink-vpc-peering-sg) and a description.
  3. Choose the VPC ID for the accepter VPC.
  4. Create the following ingress rules:
    1. One rule for the local network for all TCP ports (e.g., 172.31.0.0/16).
    2. One rule allowing DNS traffic from the security group for the Route 53 resolver inbound endpoint for all TCP ports.
    3. One rule allowing DNS traffic from the security group for the Route 53 resolver inbound endpoint for all UDP ports.
    4. One rule allowing traffic to the security group for the VPC endpoint (located in the peered VPC).

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

As discussed earlier, security groups for QuickSight are different from other security groups. You must explicitly allow return traffic from the targeted security group, and the inbound rule in your security group must allow traffic on all ports. For more information, see Inbound Rules.

Next, we modify the security group for the Route 53 resolver inbound endpoint to allow traffic from the security group we created.

  1. On the Security groups page, search for the security group ID used for the Route 53 resolver inbound endpoint.
  2. Choose Edit inbound rules.
  3. Add rules for both DNS over UDP and DNS over TCP, using the security group ID for the security group we created for QuickSight as the source.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Save rules.

Next, modify the security group that was created for the VPC endpoint for the AWS PrivateLink connection.

  1. On the Security groups page, search for the security group ID used for the VPC endpoint for the AWS PrivateLink connection.
  2. Choose Edit inbound rules.
  3. Add rules for both HTTPS and HTTP, using the security group ID for the security group created for QuickSight as the source.
  4. Choose Save rules.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Next, we set up the VPC connection in QuickSight.

  1. On the QuickSight console, choose the user name and choose Manage QuickSight.
  2. In the navigation pane, choose Manage VPC connections.
  3. Choose Add a VPC connection.
  4. For VPC connection name¸ enter a name (for example, snowflake-privatelink-vpc-peering).
  5. For Subnet, choose a subnet ID that has a route table with a peering connection to the requester VPC where the AWS PrivateLink connection resides.
  6. For Security group ID, enter the ID of the security group created earlier.
  7. For DNS resolver endpoints, enter the two IPs for the inbound resolver endpoint you created.
  8. Choose Create.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Set up a Snowflake data source in QuickSight through the VPC

To set up a Snowflake data source in QuickSight, complete the following steps:

  1. On the QuickSight console, choose Datasets in the navigation pane.
  2. Choose New dataset.
  3. Choose the Snowflake option.
  4. Enter a data source name (for example, snowflake-dataset).
  5. Choose the VPC connection you created (snowflake-privatelink).
  6. For Database server, enter the privatelink-account-url.
  7. For Database name, enter the name of your database.
  8. For Warehouse, enter the name of a running Snowflake warehouse.
  9. For Username, enter your Snowflake username.
  10. For Password, enter your Snowflake password.
  11. Choose Validate.
  12. Upon successful validation, choose Create data source.

For steps to create a dashboard, see the earlier section, Create your first QuickSight dashboard.

In the next section, we cover a similar network configuration, with the difference being that we use cross-Region VPC peering.

QuickSight connection to Snowflake via AWS PrivateLink and VPC peering across Regions

In this section, we show you how to connect to Snowflake with QuickSight over AWS PrivateLink with two VPCs peered across Regions.

We refer to Regions generically throughout this post, denoting the Region that has the Snowflake AWS PrivateLink connection as Region A and the Region in which QuickSight is set up as Region B.

The following diagram illustrates our solution architecture.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Set up VPC peering between two Regions

First, we create the VPC peering connection from the requesting VPC.

  1. Navigate to the Peering connections page on the Amazon VPC console in Region B (the Region in which you plan to use QuickSight to deploy dashboards).
  2. Choose Create peering connection.
  3. In the Select a local VPC to peer with section, for VPC (Requester), choose the VPC in which you have connected or intend to connect QuickSight.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. For Select another VPC to peer with, select My account and Another Region.
  2. Choose the Region in which your Snowflake AWS PrivateLink connection exists.
  3. For VPC ID (Accepter), enter the VPC ID for the VPC in which your Snowflake AWS PrivateLink exists.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Create peering connection.
  2. Copy the VPC peering connection ID so we can easily locate it in the next steps (it looks like pcx-xxxxxxxxxxxx).

Next, we accept the VPC peering connection from the Region in which you created your AWS PrivateLink connection.

  1. Navigate to the Amazon VPC console in Region A (where your Snowflake AWS PrivateLink connection exists).
  2. Search for and select the peering connection you created.
  3. On the Actions menu, choose Accept Request.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Review the information about the request. If everything looks correct, choose Yes, Accept.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Next, we configure DNS to resolve between the two VPCs.

  1. On the Peering connections page of the Amazon VPC console, choose your newly created VPC peering connection.
  2. On the DNS tab, check if the two options show Disabled.

If they’re enabled, skip to the steps on creating route tables.

  1. On the Actions menu, choose Edit DNS settings.

This requires your VPC to have DNS host name and resolution enabled.

  1. Select both check boxes to allow DNS to resolve from both the accepter and requestor VPCs.
  2. Choose Save.

Next, we create the route table entry to allow for routes to propagate between the two VPCs for Region B.

  1. Navigate to the Amazon VPC console in Region B (the Region in which you plan to use QuickSight to deploy dashboards).
  2. In the navigation pane, choose Route tables.
  3. Select the route tables in your requesting VPC.
  4. On the Route tab, choose Edit routes.
  5. Add a route for the CIDR block that your peered VPC uses (for this post, 10.0.0.0/16 is the CIDR block for the VPC in which the Snowflake AWS PrivateLink connection resides).
  6. Choose Save routes.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Next, create the route table entry to allow for routes to propagate between the two VPCs for Region A.

  1. Navigate to the Amazon VPC console in Region A (where your Snowflake AWS PrivateLink connection exists).
  2. Repeat the previous steps, using the CIDR block for the peered VPC (in this post, 172.16.0.0/16).

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Configure DNS in the VPC in Region B

First, we need to associate the VPC in Region B (where you deploy QuickSight) with the same private hosted zone as the VPC in Region A where your Snowflake AWS PrivateLink connection exists (<region>.privatelink.snowflakecomputing.com).

  1. On the Route 53 console, choose Hosted zones in the navigation pane.
  2. Select the private hosted zone <region>.privatelink.snowflakecomputing.com and choose Edit.
  3. In the VPCs to associate with the hosted zone section, choose Add VPC.
  4. Choose the Region and VPC ID associated with the accepter VPC.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Save changes.

Configure the Route 53 resolver inbound endpoint for your VPC in Region B

To configure the resolver inbound endpoint in Region B, complete the following steps:

  1. On the Security groups page on the Amazon VPC console, choose Create security group.
  2. Enter a name (for example, quicksight-doc-route53-resolver-sg) and a description.
  3. Choose the VPC ID used in previous steps.
  4. Create rules that allow for DNS (port 53) over UDP and TCP from within the VPC CIDR block (for this post, 172.16.0.0/16).

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Create security group.
  2. Take note the security group ID, because we now add a rule to allow traffic to the VPC endpoint security group.

Next, we set up the Route 53 inbound endpoint for this VPC.

  1. On the Route 53 console, choose Inbound endpoint in the navigation pane.
  2. Choose Create inbound endpoint.
  3. Enter a name for the endpoint (for example, quicksight-inbound-resolver).
  4. For VPC in the Region, choose the VPC ID used in previous steps.
  5. For Security group, choose the security group ID from the previous step.
  6. In the IP Address section, select two Availability Zones and subnets, and leave Use an IP address that is selected automatically
  7. Choose Submit.
  8. Choose the inbound endpoint after it’s created.
  9. After the inbound endpoint has provisioned, note the two IP addresses for the resolvers.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

Connect the VPC to QuickSight in Region B

To start, we need to create a security group for QuickSight to allow traffic to the Route 53 resolver inbound endpoints, the VPC endpoint for AWS PrivateLink, and traffic within the local network.

  1. On the Security groups page of the Amazon VPC console in Region B, choose Create security group.
  2. Enter a name (for example, quicksight-snowflake-sg) and a description.
  3. Choose the VPC ID for the VPC where you previously created the VPC peering connection.
  4. Create the following ingress rules:
    1. One for the local network all TCP ports (for example, 172.16.0.0/16).
    2. One rule allowing DNS traffic from the security group for the Route 53 resolver inbound endpoint for all TCP ports.
    3. One rule allowing DNS traffic from the security group for the Route 53 resolver inbound endpoint for all UDP ports.
    4. One allowing traffic for all TCP ports to the CIDR block for the VPC located in Region A, where your Snowflake AWS PrivateLink connection exists (for this post, 10.0.0.0/16).

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

As discussed earlier, security groups for QuickSight are different from other security groups. You must explicitly allow return traffic from the targeted security group, and the inbound rule in your security group must allow traffic on all ports. For more information, see Inbound Rules.

Next, we modify the security group for the Route 53 resolver inbound endpoint in Region B to allow traffic from the security group we created.

  1. On the Security groups page, search for the security group ID used for the Route 53 resolver inbound endpoint.
  2. Choose Edit inbound rules.
  3. Add rules for both DNS over UDP and DNS over TCP, using the CIDR block for the VPC in Region B (for this post, 172.16.0.0/16).

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Save rules.

Next, we need to modify the security group we’re using for the AWS PrivateLink connection.

  1. Navigate to the Security groups page on the Amazon VPC console in Region A.
  2. Search for the security group ID that is used for the VPC endpoint for the AWS PrivateLink connection.
  3. Choose Edit inbound rules.
  4. Add rules for both HTTPS and HTTP, using the CIDR Block for the VPC in Region B as the source (for this post, 172.16.0.0/16).

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Save rules.

Finally, we set up the QuickSight VPC connection.

  1. Navigate to the QuickSight console in Region B.
  2. Choose the user name and choose Manage QuickSight.
  3. In the navigation pane, choose Manage VPC connection.
  4. Choose Add a VPC connection.
  5. For VPC connection name, enter a connection name (for example, snowflake-privatelink-cross-region).
  6. For VPC ID, choose the VPC ID of the VPC in Region B.
  7. For Subnet, choose a subnet ID from the VPC in Region B that has a route table with a peering connection to the VPC where the AWS PrivateLink connection resides.
  8. For Security group ID, enter the ID of the security group you created.
  9. For DNS resolver endpoints, enter the two IPs for the inbound resolver endpoint created earlier.

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web Services

  1. Choose Create.

Set up a Snowflake data source in QuickSight through the VPC

To set up a Snowflake data source in QuickSight, complete the following steps:

  1. On the QuickSight console, choose Datasets in the navigation pane.
  2. Choose New dataset.
  3. Choose the Snowflake option.
  4. Enter a name for your data source (for example, snowflake-dataset).
  5. Choose the VPC connection you created (snowflake-privatelink).
  6. For Database server, enter the privatelink-account-url.
  7. For Database name, enter the name of your database.
  8. For Warehouse, enter the name of a running Snowflake warehouse.
  9. For Username, enter your Snowflake username.
  10. For Password, enter your Snowflake password.
  11. Choose Validate.
  12. Upon successful validation, choose Create data source.

For steps to create a dashboard, see the earlier section, Create your first QuickSight dashboard.

For our last configuration, we cover how to set up a QuickSight connection to Snowflake without AWS PrivateLink.

QuickSight connection to Snowflake without AWS PrivateLink

In this section, we show you how to connect to Snowflake with QuickSight without using AWS PrivateLink.

  1. On the QuickSight console, choose Datasets in the navigation pane.
  2. Choose New dataset.
  3. Choose the Snowflake option.
  4. Enter a data source name (for example, snowflake-dataset).
  5. Leave the connection type as Public network.
  6. For Database name, enter the name of your database.
  7. For your database server, enter the URL you use to log in to your Snowflake (xxxxxxxx.snowflakecomputing.com).
  8. For Warehouse, enter the name of a running Snowflake warehouse.
  9. For Username, enter your Snowflake username.
  10. For Password, enter your Snowflake password.
  11. Choose Validate.
  12. Choose Create data source.

For steps to create a dashboard, see the earlier section, Create your first QuickSight dashboard.

Clean up

If your work with QuickSight, Snowflake, and PrivateLink is complete, remove your Route53 resolver inbound endpoint, Route 53 private host zone, and the VPC endpoint for Snowflake in order to avoid incurring additional fees.

Conclusion

In this post, we covered four scenarios for connecting QuickSight to Snowflake as a data source using AWS PrivateLink for connectivity in three different scenarios: the same VPC, with VPC peering in the same Region, and with VPC peering across Regions. We also covered how to connect QuickSight to Snowflake without AWS PrivateLink.

After you set up the data source, you can gain further insights from your data by setting up ML Insights in QuickSight, set up graphical representations of your data using QuickSight visuals, or join data from multiple datasets, as well as all other QuickSight features.


About the Author

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web ServicesMaxwell Moon is a Senior Solutions Architect at AWS working with Independent Software Vendors (ISVs) to design and scale their applications on AWS. Outside of work, Maxwell is a dad to two cats, is an avid supporter of the Wolverhampton Wanderers Football Club, and is patiently waiting for a new wave of ska music.

 

 

Establish private connectivity between Amazon QuickSight and Snowflake using AWS PrivateLink | Amazon Web ServicesBosco Albuquerque is a Sr Partner Solution Architect at AWS and has over 20 years of experience in working with database and analytics products, from enterprise database vendors, and cloud providers and has helped large technology companies in designing data analytics solutions as well as led engineering teams is designing and implementing data analytics platforms and data products.

 Source link

Back to top button
SoundCloud To Mp3