Online Store Spec

class databricks.ml_features.online_store_spec.AmazonDynamoDBSpec(*, region: Optional[str], access_key_id: Optional[str] = None, secret_access_key: Optional[str] = None, session_token: Optional[str] = None, table_name: Optional[str] = None, read_secret_prefix: Optional[str] = None, write_secret_prefix: Optional[str] = None, ttl: Optional[timedelta] = None, endpoint_url: Optional[str] = None)

Bases: OnlineStoreSpec

Note

Aliases: databricks.feature_engineering.online_store_spec.AmazonDynamoDBSpec, databricks.feature_store.online_store_spec.AmazonDynamoDBSpec

This OnlineStoreSpec implementation is intended for publishing features to Amazon DynamoDB.

If table_name is not provided, publish_table() will use the offline store’s database and table name combined as the online table name.

To use a different table name in the online store, provide a value for the table_name argument.

The expected read or write secrets for DynamoDB for a given {prefix} string are ${prefix}-access-key-id, ${prefix}-secret-access-key, and ${prefix}-session-token.

If none of the access_key_id, secret_access_key, and write_secret_prefix are passed, the instance profile attached to the cluster will be used to write to DynamoDB.

Parameters

  • region – Region to access online store.

  • access_key_id – Access key ID that has access to the online store. Deprecated. Use write_secret_prefix instead.

  • secret_access_key – Secret access key to access the online store. Deprecated. Use write_secret_prefix instead.

  • session_token – Session token to access the online store. Deprecated. Use write_secret_prefix instead.

  • table_name – Table name.

  • read_secret_prefix – Prefix for read secret.

  • write_secret_prefix – Prefix for write secret.

  • ttl – The time to live for data published to the online store. This attribute is only applicable when publishing time series feature tables. If the time to live is specified for a time series table, publish_table() will publish a window of data instead of the latest snapshot.

property access_key_id

Warning

databricks.ml_features.online_store_spec.amazon_dynamodb_online_store_spec.AmazonDynamoDBSpec.access_key_id is deprecated. This method will be removed in a future release. Use write_secret_prefix instead.

Access key ID that has access to the online store. Property will be empty if write_secret_prefix or the instance profile attached to the cluster are intended to be used.

property secret_access_key

Warning

databricks.ml_features.online_store_spec.amazon_dynamodb_online_store_spec.AmazonDynamoDBSpec.secret_access_key is deprecated. This method will be removed in a future release. Use write_secret_prefix instead.

Secret access key to access the online store. Property will be empty if write_secret_prefix or the instance profile attached to the cluster are intended to be used.

property session_token

Warning

databricks.ml_features.online_store_spec.amazon_dynamodb_online_store_spec.AmazonDynamoDBSpec.session_token is deprecated. This method will be removed in a future release. Use write_secret_prefix instead.

Session token to access the online store. Property will be empty if write_secret_prefix or the instance profile attached to the cluster are intended to be used.

property endpoint_url

Endpoint url of DynamoDB online store, mainly used for testing with LocalStack

property cloud

Define the cloud property for the data store.

property store_type

Define the data store type.

property region

Region to access the online store.

property ttl: Optional[timedelta]

Time to live attribute for the online store.

auth_type()

Publish Auth type.

class databricks.ml_features.online_store_spec.AmazonRdsMySqlSpec(hostname: str, port: int, user: Optional[str] = None, password: Optional[str] = None, database_name: Optional[str] = None, table_name: Optional[str] = None, driver_name: Optional[str] = None, read_secret_prefix: Optional[str] = None, write_secret_prefix: Optional[str] = None)

Bases: OnlineStoreSpec

Note

Aliases: databricks.feature_engineering.online_store_spec.AmazonRdsMySqlSpec, databricks.feature_store.online_store_spec.AmazonRdsMySqlSpec

Class that defines and creates AmazonRdsMySqlSpec objects.

This OnlineStoreSpec implementation is intended for publishing features to Amazon RDS MySQL and Aurora (MySQL-compatible edition).

See OnlineStoreSpec documentation for more usage information, including parameter descriptions.

Parameters

  • hostname – Hostname to access online store.

  • port – Port number to access online store.

  • user – Username that has access to the online store. Deprecated. Use write_secret_prefix instead.

  • password – Password to access the online store. Deprecated. Use write_secret_prefix instead.

  • database_name – Database name.

  • table_name – Table name.

  • driver_name – Name of custom JDBC driver to access the online store.

  • read_secret_prefix – Prefix for read secret.

  • write_secret_prefix – Prefix for write secret.

property hostname

Hostname to access the online store.

property port

Port number to access the online store.

property database_name

Database name.

property cloud

Define the cloud propert for the data store.

property store_type

Define the data store type property.

auth_type()

Publish Auth type.

class databricks.ml_features.online_store_spec.AzureCosmosDBSpec(*, account_uri: str, database_name: Optional[str] = None, container_name: Optional[str] = None, read_secret_prefix: Optional[str] = None, write_secret_prefix: str, **kwargs)

Bases: OnlineStoreSpec

Note

Aliases: databricks.feature_engineering.online_store_spec.AzureCosmosDBSpec, databricks.feature_store.online_store_spec.AzureCosmosDBSpec

This OnlineStoreSpec implementation is intended for publishing features to Azure Cosmos DB.

If database_name and container_name are not provided, publish_table() will use the offline store’s database and table name as the Cosmos DB database and container name.

The expected read or write secret for Cosmos DB for a given {prefix} string is ${prefix}-authorization-key.

The authorization key can be either the Cosmos DB account primary or secondary key.

Parameters

  • account_uri – URI of the Cosmos DB account.

  • database_name – Database name.

  • container_name – Container name.

  • read_secret_prefix – Prefix for read secret.

  • write_secret_prefix – Prefix for write secret.

property account_uri

Account URI of the online store.

property database_name

Database name.

property container_name

Container name.

property target_throughput_threshold_for_provisioned

Threshold for handling CosmosDB Requests Units. Note that this is for CosmosDB Provisioned Throughput, so you need to specify a number between 0 and 1 indicating the percentage.

property target_throughput_for_serverless

Threshold for handling CosmosDB Requests Units. Note that this is for CosmosDB Serverless account, so you need to specify an absolute number, which is the threshold for the Spark job to write to the account.

property cloud

Define the cloud property for the data store.

property store_type

Define the data store type.

auth_type()

Publish Auth type.

class databricks.ml_features.online_store_spec.AzureMySqlSpec(hostname: str, port: int, user: Optional[str] = None, password: Optional[str] = None, database_name: Optional[str] = None, table_name: Optional[str] = None, driver_name: Optional[str] = None, read_secret_prefix: Optional[str] = None,