Lakehouse Monitoring for GenAI

WARNING: This API is deprecated. Please use the new production monitoring API in MLflow 3.

databricks.agents.monitoring.create_external_monitor(*, catalog_name: str, schema_name: str, assessments_config: AssessmentsSuiteConfig | dict, experiment_id: str | None = None, experiment_name: str | None = None) → ExternalMonitor

Create a monitor for a GenAI application served outside Databricks.

Parameters:

  • catalog_name (str) – The name of the catalog in UC to create the trace archive table in.

  • schema_name (str) – The name of the schema in UC to create the trace archive table in.

  • assessments_config (AssessmentsSuiteConfig | dict) – The configuration for the suite of assessments to be run on traces from the GenAI application.

  • experiment_id (str | None, optional) – ID of Mlflow experiment that the monitor should be associated with. Defaults to the currently active experiment.

  • experiment_name (str | None, optional) – The name of the Mlflow experiment that the monitor should be associated with. Defaults to the currently active experiment.

Returns:

The created monitor.

Return type:

ExternalMonitor

databricks.agents.monitoring.delete_external_monitor(*, experiment_id: str | None = None, experiment_name: str | None = None) → None

Deletes the monitor for a GenAI application served outside Databricks.

Parameters:

  • experiment_id (str | None, optional) – ID of the Mlflow experiment that the monitor is associated with. Defaults to None.

  • experiment_name (str | None, optional) – Name of the Mlflow experiment that the monitor is associated with. Defaults to None.

databricks.agents.monitoring.get_external_monitor(*, experiment_id: str | None = None, experiment_name: str | None = None) → ExternalMonitor

Gets the monitor for a GenAI application served outside Databricks.

Parameters:

  • experiment_id (str | None, optional) – ID of the Mlflow experiment that the monitor is associated with. Defaults to None.

  • experiment_name (str | None, optional) – Name of the Mlflow experiment that the monitor is associated with. Defaults to None.

Raises:

  • ValueError – When neither experiment_id nor experiment_name is provided.

  • ValueError – When no monitor is found for the given experiment_id or experiment_name.

Returns:

The retrieved external monitor.

Return type:

entities.ExternalMonitor

databricks.agents.monitoring.update_external_monitor(*, experiment_id: str | None = None, experiment_name: str | None = None, assessments_config: AssessmentsSuiteConfig | dict) → ExternalMonitor

Updates the monitor for an GenAI application served outside Databricks.

Parameters:

  • assessments_config (assessments.AssessmentsSuiteConfig) – The updated configuration for the suite of assessments to be run on traces from the AI system. Partial updates of arrays is not supported, so assessments specified here will override your monitor’s assessments. If unspecified, non-nested fields like sample will not be updated.

  • experiment_id (str | None, optional) – ID of the Mlflow experiment that the monitor is associated with. Defaults to None.

  • experiment_name (str | None, optional) – Name of the Mlflow experiment that the monitor is associated with. Defaults to None.

Raises:

ValueError – When assessments_config is not provided.

Returns:

The updated external monitor.

Return type:

entities.ExternalMonitor

databricks.agents.monitoring.create_monitor(endpoint_name: str, *, assessments_config: dict | AssessmentsSuiteConfig | None = None, experiment_id: str | None = None, monitoring_config: dict | MonitoringConfig | None = None) → Monitor

Create a monitor for a Databricks serving endpoint.

Parameters:

  • endpoint_name – The name of the serving endpoint.

  • assessments_config – The configuration for the suite of assessments to be run on traces.

  • experiment_id – The experiment ID to log the monitoring results. Defaults to the currently active MLflow experiment.

  • monitoring_config – Deprecated. The monitoring configuration.

Returns:

The monitor for the serving endpoint.

databricks.agents.monitoring.delete_monitor(*, endpoint_name: str | None = None) → None

Deletes a monitor for a Databricks serving endpoint.

Parameters:

endpoint_name (str, optional) – The name of the agent’s serving endpoint.

databricks.agents.monitoring.get_monitor(*, endpoint_name: str) → Monitor

Retrieves a monitor for a Databricks serving endpoint.

Parameters:

endpoint_name (str, optional) – The name of the agent’s serving endpoint.

Returns:

Monitor | ExternalMonitor metadata. For external monitors, this will include the status of the ingestion endpoint.

databricks.agents.monitoring.update_monitor(*, endpoint_name: str, assessments_config: dict | AssessmentsSuiteConfig | None = None, monitoring_config: dict | MonitoringConfig | None = None) → Monitor

Partially update a monitor for a serving endpoint.

Parameters:

  • endpoint_name (str, optional) – The name of the agent’s serving endpoint. Only supported for agents served on Databricks.

  • assessments_config – The updated configuration for the suite of assessments to be run on traces. Partial updates of arrays is not supported, so assessments specified here will override your monitor’s assessments. If unspecified, non-nested fields like sample will not be updated.

  • monitoring_config – Deprecated. The configuration change, using upsert semantics.

Returns:

The updated monitor for the serving endpoint.

Return type:

Monitor

class databricks.agents.monitoring.AssessmentsSuiteConfig(sample