diff --git a/NEXT_CHANGELOG.md b/NEXT_CHANGELOG.md index a774c1991..0a662081d 100644 --- a/NEXT_CHANGELOG.md +++ b/NEXT_CHANGELOG.md @@ -5,6 +5,9 @@ ### New Features and Improvements * Added automatic detection of AI coding agents (Amp, Antigravity, Augment, Claude Code, Cline, Codex, Copilot CLI, Copilot VS Code, Cursor, Gemini CLI, Goose, Kiro, OpenClaw, OpenCode, Windsurf) in the user-agent string. The SDK now appends `agent/` to HTTP request headers when running inside a known AI agent environment. Also honors the `AGENT=` standard: when `AGENT` is set to a known product name the SDK reports that product, and when set to an unrecognized non-empty value the SDK reports `agent/unknown`. Environment variables set to the empty string (e.g. `CLAUDECODE=""`) now count as "set" for presence-only matchers, matching `databricks-sdk-go` semantics; previously they were treated as unset. Explicit agent env vars (e.g. `CLAUDECODE`, `GOOSE_TERMINAL`) always take precedence over the generic `AGENT=` signal. When multiple agent env vars are present (e.g. a Cursor CLI subagent invoked from Claude Code), the user-agent reports `agent/multiple`. +### Breaking Changes +* Remove the `experimentalIsUnifiedHost` field (and the `DATABRICKS_EXPERIMENTAL_IS_UNIFIED_HOST` environment variable) from `DatabricksConfig`. The flag was unused — `getHostType()` never read it — so unified host detection is now determined purely by URL pattern and the automatic `/.well-known/databricks-config` metadata resolution. Callers that set the flag should remove those calls. Mirrors [databricks/databricks-sdk-go#1641](https://github.com/databricks/databricks-sdk-go/pull/1641) and [databricks/databricks-sdk-py#1358](https://github.com/databricks/databricks-sdk-py/pull/1358). + ### Bug Fixes * Add `X-Databricks-Org-Id` header to `SharesExtImpl.list()` for SPOG host compatibility. Without this header, calls to the hand-written extension were rejected by the SPOG proxy with `Unable to load OAuth Config (400 UNKNOWN)`. Mirrors [databricks/databricks-sdk-go#1635](https://github.com/databricks/databricks-sdk-go/pull/1635). diff --git a/README.md b/README.md index 54182b945..328d2a665 100644 --- a/README.md +++ b/README.md @@ -87,8 +87,10 @@ workspace. // press for autocompletion ### In this section - [Default authentication flow](#default-authentication-flow) +- [Unified host support](#unified-host-support) - [Databricks native authentication](#databricks-native-authentication) - [Azure native authentication](#azure-native-authentication) +- [Google Cloud Platform native authentication](#google-cloud-platform-native-authentication) - [Overriding .databrickscfg](#overriding-databrickscfg) - [Additional authentication configuration options](#additional-authentication-configuration-options) @@ -98,9 +100,25 @@ If you run the [Databricks Terraform Provider](https://registry.terraform.io/pro 1. [Databricks native authentication](#databricks-native-authentication) 2. [Azure native authentication](#azure-native-authentication) -3. If the SDK is unsuccessful at this point, it returns an authentication error and stops running. +3. [Google Cloud Platform native authentication](#google-cloud-platform-native-authentication) +4. If the SDK is unsuccessful at this point, it returns an authentication error and stops running. -You can instruct the Databricks SDK for Java to use a specific authentication method by instantiating the `DatabricksConfig` class and setting the `auth_type` as described in the following sections. +Each authentication method requires specific configuration attributes (e.g., `token` for PAT auth, `azureClientId` for Azure service principal auth). The SDK automatically detects the cloud provider and skips authentication methods whose required configuration attributes are not present. This means that Azure-specific methods like `azure-cli` are automatically skipped when connecting to an AWS or GCP workspace, and vice versa for GCP-specific methods. + +To force a specific authentication method instead of relying on auto-detection, set the `authType` on `DatabricksConfig`: + +```java +import com.databricks.sdk.WorkspaceClient; +import com.databricks.sdk.core.DatabricksConfig; +... +// Force Azure CLI authentication — skip all other methods +DatabricksConfig config = new DatabricksConfig() + .setHost("https://mycompany.databricks.com") + .setAuthType("azure-cli"); +WorkspaceClient workspace = new WorkspaceClient(config); +``` + +This is useful when your environment has credentials for multiple authentication methods and you want to ensure a specific one is used, or when auto detection is not accurate. For each authentication method, the SDK searches for compatible authentication credentials in the following locations, in the following order. Once the SDK finds a compatible set of credentials that it can use, it stops searching: @@ -115,6 +133,41 @@ For each authentication method, the SDK searches for compatible authentication c Depending on the Databricks authentication method, the SDK uses the following information. Presented are the `WorkspaceClient` and `AccountClient` arguments (which have corresponding `.databrickscfg` file fields), their descriptions, and any corresponding environment variables. +### Unified host support + +Certain Databricks host types support both account-level and workspace-level API operations from a single endpoint. When using such a unified host, a single configuration profile can be used to create both `WorkspaceClient` and `AccountClient` instances without changing the `host`. + +For this to work, the following conditions must be met: + +1. The host must support unified operations. +2. Both `account_id` and `workspace_id` must be available — either set explicitly in the configuration or auto-discovered. + +When both values are present, the SDK uses `workspace_id` to route workspace-level requests and `account_id` to route account-level requests, all through the same host. + +```ini +# .databrickscfg +[unified] +host = https://mycompany.databricks.com +account_id = 00000000-0000-0000-0000-000000000000 +workspace_id = 1234567890 +``` + +```java +import com.databricks.sdk.AccountClient; +import com.databricks.sdk.WorkspaceClient; +import com.databricks.sdk.core.DatabricksConfig; +... +// Both clients share the same host and profile +WorkspaceClient workspace = new WorkspaceClient(new DatabricksConfig().setProfile("unified")); +AccountClient account = new AccountClient(new DatabricksConfig().setProfile("unified")); + +// A WorkspaceClient for a different workspace under the same host and account +WorkspaceClient otherWorkspace = new WorkspaceClient( + new DatabricksConfig().setProfile("unified").setWorkspaceId("2345678901")); +``` + +If the host supports it, `account_id` and `workspace_id` may be auto-discovered, reducing the required explicit configuration. + ### Databricks native authentication By default, the Databricks SDK for Java initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Workload Identity Federation (WIF). See [Supported WIF](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-federation-provider) for the supported JWT token providers. @@ -123,13 +176,17 @@ By default, the Databricks SDK for Java initially tries [Databricks token authen - For Databricks OIDC authentication, you must provide the `host`, `client_id` and `token_audience` _(optional)_ either directly, through the corresponding environment variables, or in your `.databrickscfg` configuration file. - For Azure DevOps OIDC authentication, the `token_audience` is irrelevant as the audience is always set to `api://AzureADTokenExchange`. Also, the `System.AccessToken` pipeline variable required for OIDC request must be exposed as the `SYSTEM_ACCESSTOKEN` environment variable, following [Pipeline variables](https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#systemaccesstoken) -| Argument | Description | Environment variable | -|--------------|-------------|-------------------| -| `host` | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint. | `DATABRICKS_HOST` | -| `account_id` | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Only has effect when `Host` is either `https://accounts.cloud.databricks.com/` _(AWS)_, `https://accounts.azuredatabricks.net/` _(Azure)_, or `https://accounts.gcp.databricks.com/` _(GCP)_. | `DATABRICKS_ACCOUNT_ID` | -| `token` | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_. | `DATABRICKS_TOKEN` | -| `client_id` | _(String)_ The Databricks Service Principal Application ID. | `DATABRICKS_CLIENT_ID` | -| `token_audience` | _(String)_ When using Workload Identity Federation, the audience to specify when fetching an ID token from the ID token supplier. | `TOKEN_AUDIENCE` | +During initialization, the SDK automatically resolves missing configuration fields (`account_id`, `workspace_id`, and `discovery_url`) from the host metadata endpoint. Any explicitly provided values take precedence and are never overwritten. If the auto-discovery fails, the SDK falls back to the explicit configuration. It is recommended to always set explicit configuration. + +| Argument | Description | Environment variable | +|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------| +| `host` | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint. | `DATABRICKS_HOST` | +| `account_id` | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Auto-discovered if not provided. Has effect on hosts that serve account-level APIs. | `DATABRICKS_ACCOUNT_ID` | +| `workspace_id` | _(String)_ The Databricks workspace ID for the Databricks workspace endpoint. Auto-discovered if not provided. | `DATABRICKS_WORKSPACE_ID` | +| `discovery_url` | _(String)_ The OpenID Connect discovery URL. Auto-discovered if not provided. When set, OIDC endpoints are fetched directly from this URL instead of using the default host-based well-known endpoint logic. | `DATABRICKS_DISCOVERY_URL`| +| `token` | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_. | `DATABRICKS_TOKEN` | +| `client_id` | _(String)_ The Databricks Service Principal Application ID. | `DATABRICKS_CLIENT_ID` | +| `token_audience` | _(String)_ When using Workload Identity Federation, the audience to specify when fetching an ID token from the ID token supplier. | `TOKEN_AUDIENCE` | For example, to use Databricks token authentication: diff --git a/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksConfig.java b/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksConfig.java index 40b212e52..cad8b747f 100644 --- a/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksConfig.java +++ b/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksConfig.java @@ -37,13 +37,6 @@ public class DatabricksConfig { @ConfigAttribute(env = "DATABRICKS_WORKSPACE_ID") private String workspaceId; - /** - * Flag to explicitly mark a host as a unified host. Note: This API is experimental and may change - * or be removed in future releases without notice. - */ - @ConfigAttribute(env = "DATABRICKS_EXPERIMENTAL_IS_UNIFIED_HOST") - private Boolean experimentalIsUnifiedHost; - @ConfigAttribute(env = "DATABRICKS_TOKEN", auth = "pat", sensitive = true) private String token; @@ -342,15 +335,6 @@ public DatabricksConfig setWorkspaceId(String workspaceId) { return this; } - public Boolean getExperimentalIsUnifiedHost() { - return experimentalIsUnifiedHost; - } - - public DatabricksConfig setExperimentalIsUnifiedHost(Boolean experimentalIsUnifiedHost) { - this.experimentalIsUnifiedHost = experimentalIsUnifiedHost; - return this; - } - public String getDatabricksCliPath() { return this.databricksCliPath; }