Snowflake connector

reference Facts, options, APIs

Prerequisites

Before configuring the connector, make sure that you have:

A Snowflake account accessible from your CluedIn environment.
A destination database and schema, and a warehouse for executing export operations.
A Snowflake user with key-pair authentication configured.
A role granted to that user with access to the destination and the permissions needed by the export.
OpenSSL installed on the computer where you will generate the keys.
The Snowflake connector available in Consume > Export Targets in CluedIn.

Use a dedicated integration user and role. A Snowflake administrator should create or approve the user and grant the required access.

Find your Snowflake connection details

Sign in to Snowflake, open your user menu, and select Connect a tool to Snowflake. The Account Details dialog provides connection settings. For the account identifier, use the organisation and account format, for example, myorg-myaccount. Enter the identifier only, without https:// or .snowflakecomputing.com.

Alternatively, run the following in a Snowflake SQL worksheet:

SELECT
    CURRENT_ORGANIZATION_NAME() || '-' || CURRENT_ACCOUNT_NAME()
        AS ACCOUNT_IDENTIFIER,
    CURRENT_USER()      AS USER_NAME,
    CURRENT_ROLE()      AS ROLE_NAME,
    CURRENT_WAREHOUSE() AS WAREHOUSE_NAME,
    CURRENT_DATABASE() AS DATABASE_NAME,
    CURRENT_SCHEMA()   AS SCHEMA_NAME;

The results describe your current session. If you plan to use a separate integration user or role, enter its details in CluedIn instead. A null warehouse, database, or schema means that object has not been selected in the session.

Use the following commands to list objects visible to your current role. Replace the example database and schema names with your destination:

SHOW WAREHOUSES;
SHOW DATABASES;
SHOW SCHEMAS IN DATABASE CLUEDIN_DB;
SHOW TABLES IN SCHEMA CLUEDIN_DB.EXPORTS;

You can also locate the destination by expanding the database, schema, and table in Snowsight’s database explorer.

For more information, see Snowflake’s connection settings guide.

Configure key-pair authentication

CluedIn holds the private key; Snowflake holds the matching public key assigned to the user. The private key is generated outside Snowflake and cannot be downloaded from Snowflake.

Generate the keys

On a Mac, open Terminal. On Windows or Linux, use a terminal with OpenSSL installed. Run:

openssl genrsa 2048 | openssl pkcs8 -topk8 -v2 aes-256-cbc -inform PEM -out cluedin_snowflake_key.p8

Choose and retain the encryption passphrase when prompted. Then generate the public key:

openssl rsa -in cluedin_snowflake_key.p8 -pubout -out cluedin_snowflake_key.pub

The .p8 file contains the encrypted PKCS#8 private key. The .pub file contains the public key.

Keep the private key and passphrase secure. Do not paste them into tickets or commit them to source control.

Assign the public key to the user

Open cluedin_snowflake_key.pub. Copy the key body without the -----BEGIN PUBLIC KEY----- and -----END PUBLIC KEY----- lines, and remove line breaks.

Have an administrator with permission to modify the user’s authentication methods run:

ALTER USER CLUEDIN_EXPORT
SET RSA_PUBLIC_KEY = 'PASTE_PUBLIC_KEY_BODY_HERE';

Replace CLUEDIN_EXPORT with the actual Snowflake user object name. Register the key against the user whose login name you will enter in CluedIn. Check for an existing registered key before replacing it, because other integrations may depend on it.

For current registration and rotation options, see Snowflake’s key-pair authentication guide.

Prepare the private key for CluedIn

Open cluedin_snowflake_key.p8 and copy the entire file, including:

-----BEGIN ENCRYPTED PRIVATE KEY-----
...
-----END ENCRYPTED PRIVATE KEY-----

Paste the complete contents into Private Key (PEM) in CluedIn. Enter the encryption passphrase in Private Key Passphrase. Do not enter a file path or your Snowflake password.

If your organisation supplies an unencrypted private key, include its BEGIN PRIVATE KEY and END PRIVATE KEY lines and leave the passphrase field blank.

Configure the Snowflake connector

On the navigation pane, go to Consume > Export Targets, and select Add Export Target.
On the Choose Target tab, select the Snowflake target, and select Next.
On the Configure tab, complete the following fields.

Field	What to enter
Name	A user-friendly label, for example, `Snowflake Customer Export`. This is a CluedIn label, not a Snowflake object.
Account Identifier	Your Snowflake account identifier, for example, `myorg-myaccount`.
User	The integration user’s Snowflake login name. An email address is valid only if it is the user’s login name.
Private Key (PEM)	The complete PEM private key, including its header, footer, and line breaks.
Private Key Passphrase	The passphrase used to encrypt the private key. Required only for an encrypted key.
Database	The destination database name, for example, `CLUEDIN_DB`.
Schema	The schema within that database, for example, `EXPORTS`.
Warehouse	The compute warehouse used for export operations, for example, `CLUEDIN_WH`.
Role	A role granted to the integration user that can perform the export. When blank, the user’s default role is used.
Table Name	The target table in Snowflake that rows are merged into, for example, `CUSTOMERS`, within the configured database and schema.
Export Schedule	Select when CluedIn should export. The schedule uses UTC. Choose Custom Cron to enter your own cron expression instead of the preset hourly, daily, or weekly options.

There is no separate output format field for this connector: Snowflake export always streams rows through the Snowpipe Streaming REST API as JSON, regardless of whether stream cache is enabled.

For example, the values CLUEDIN_DB, EXPORTS, and CUSTOMERS identify the destination CLUEDIN_DB.EXPORTS.CUSTOMERS.

Configure tab of the Add Export Target pane with Name, Account Identifier, User, Private Key, Database, Schema, Warehouse, Role, Table Name, and Export Schedule filled in

Use the exact object names. Snowflake normally stores unquoted identifiers in uppercase; quoted identifiers can be case-sensitive.

Table Name can also be a pattern, using the same variable syntax as other connectors’ file name patterns: {StreamId}, {ContainerName}, and {OutputFormat} (each can be formatted with a formatString modifier, for example {StreamId:N}). For example, {ContainerName}_Table resolves to a different target table per stream. See File name patterns for the variable syntax.

Do not use {DataTime} in Table Name. The resolved name is reused as the name of the connector’s internal landing table and Snowpipe Streaming pipe, both of which must stay the same across every export run for a given stream. Including {DataTime} makes connection verification fail.

Select Test connection.
When the connection test succeeds, select Add.

The export target is now available when configuring a stream. Configure the stream’s records and output fields, then run a small sample export.

If Table Name does not already exist in Snowflake, the connector creates it on the first export, with one VARCHAR column per exported property (and a NUMBER column for the internal persist-version field). As the stream’s properties change over time, the connector adds missing columns automatically on later exports; it never removes a column.

Snowflake permissions

The connector creates everything it writes to itself: an internal landing table, the target table named in Table Name (if it does not already exist), and a Snowpipe Streaming pipe. In Snowflake, the role that creates an object automatically becomes its owner, which already carries every privilege the connector’s day-to-day operations need against that object. So beyond the key-pair setup above, the integration role only needs the following granted explicitly:

GRANT USAGE ON WAREHOUSE CLUEDIN_WH TO ROLE CLUEDIN_EXPORT_ROLE;
GRANT USAGE ON DATABASE CLUEDIN_DB TO ROLE CLUEDIN_EXPORT_ROLE;
GRANT USAGE, CREATE TABLE, CREATE PIPE ON SCHEMA CLUEDIN_DB.EXPORTS TO ROLE CLUEDIN_EXPORT_ROLE;

GRANT ROLE CLUEDIN_EXPORT_ROLE TO USER CLUEDIN_EXPORT;

Have your Snowflake administrator adapt these names to your warehouse, database, schema, and role.

See Snowflake’s privilege reference.

Verify the export

A successful connection test does not prove that the complete export can write to the destination.

Export a small number of records through the configured stream.
Review the stream logs for errors.

In Snowflake, query the destination:

SELECT *
FROM CLUEDIN_DB.EXPORTS.CUSTOMERS
LIMIT 10;

Confirm that the expected records and columns are present before exporting the full dataset.

Troubleshooting

Issue	What to check
Authentication fails	Check the account identifier and login name. Confirm that the public key registered on that user matches the private key supplied to CluedIn.
Private key cannot be read	Paste the entire PEM file with actual line breaks. Confirm the passphrase for an encrypted key.
Role cannot be used	Confirm that the role is granted to the integration user. If the field is blank, check the user’s default role.
Warehouse, database, schema, or table is unavailable	Check the object name and the selected role’s privileges. An object may be hidden from a role that cannot access it.
Connection test fails, mentioning `DataTime` or `Table Name`	Remove `{DataTime}` from Table Name. Only `{StreamId}`, `{ContainerName}`, and `{OutputFormat}` are supported there.
Connection succeeds but export fails	Review stream logs. A common cause is the role missing `CREATE TABLE` or `CREATE PIPE` on the schema — see Snowflake permissions.
Connection is blocked	Check Snowflake network policies and connectivity from the CluedIn environment, including its outbound IP addresses. CluedIn’s outbound IP address(es) and CIDR are listed under Organization Settings > General Settings > Cluster Details.
Export runs at the wrong local time	Check the UTC schedule. For example, 00:00 UTC is 10:00 in Brisbane.