From 3e81d21c7595626b253de8eb1d3a7445b5539087 Mon Sep 17 00:00:00 2001 From: Hector Castejon Diaz Date: Wed, 22 Apr 2026 07:25:51 +0000 Subject: [PATCH 1/2] Remove experimentalIsUnifiedHost flag from DatabricksConfig MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Drops the experimentalIsUnifiedHost field and the DATABRICKS_EXPERIMENTAL_IS_UNIFIED_HOST env var. The flag was never read — getHostType() derives the host type from the URL pattern, and unified-host metadata fields (accountId, workspaceId, discoveryUrl) are auto-resolved via /.well-known/databricks-config during config.resolve(). No production path referenced the flag, and no tests set it. Ports databricks/databricks-sdk-go#1641 and databricks/databricks-sdk-py#1358. This is a compile-time breaking change. Downstream callers that were setting the flag should drop those calls. Co-authored-by: Isaac --- NEXT_CHANGELOG.md | 3 +++ .../databricks/sdk/core/DatabricksConfig.java | 16 ---------------- 2 files changed, 3 insertions(+), 16 deletions(-) diff --git a/NEXT_CHANGELOG.md b/NEXT_CHANGELOG.md index a774c1991..0a662081d 100644 --- a/NEXT_CHANGELOG.md +++ b/NEXT_CHANGELOG.md @@ -5,6 +5,9 @@ ### New Features and Improvements * Added automatic detection of AI coding agents (Amp, Antigravity, Augment, Claude Code, Cline, Codex, Copilot CLI, Copilot VS Code, Cursor, Gemini CLI, Goose, Kiro, OpenClaw, OpenCode, Windsurf) in the user-agent string. The SDK now appends `agent/` to HTTP request headers when running inside a known AI agent environment. Also honors the `AGENT=` standard: when `AGENT` is set to a known product name the SDK reports that product, and when set to an unrecognized non-empty value the SDK reports `agent/unknown`. Environment variables set to the empty string (e.g. `CLAUDECODE=""`) now count as "set" for presence-only matchers, matching `databricks-sdk-go` semantics; previously they were treated as unset. Explicit agent env vars (e.g. `CLAUDECODE`, `GOOSE_TERMINAL`) always take precedence over the generic `AGENT=` signal. When multiple agent env vars are present (e.g. a Cursor CLI subagent invoked from Claude Code), the user-agent reports `agent/multiple`. +### Breaking Changes +* Remove the `experimentalIsUnifiedHost` field (and the `DATABRICKS_EXPERIMENTAL_IS_UNIFIED_HOST` environment variable) from `DatabricksConfig`. The flag was unused — `getHostType()` never read it — so unified host detection is now determined purely by URL pattern and the automatic `/.well-known/databricks-config` metadata resolution. Callers that set the flag should remove those calls. Mirrors [databricks/databricks-sdk-go#1641](https://github.com/databricks/databricks-sdk-go/pull/1641) and [databricks/databricks-sdk-py#1358](https://github.com/databricks/databricks-sdk-py/pull/1358). + ### Bug Fixes * Add `X-Databricks-Org-Id` header to `SharesExtImpl.list()` for SPOG host compatibility. Without this header, calls to the hand-written extension were rejected by the SPOG proxy with `Unable to load OAuth Config (400 UNKNOWN)`. Mirrors [databricks/databricks-sdk-go#1635](https://github.com/databricks/databricks-sdk-go/pull/1635). diff --git a/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksConfig.java b/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksConfig.java index 40b212e52..cad8b747f 100644 --- a/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksConfig.java +++ b/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksConfig.java @@ -37,13 +37,6 @@ public class DatabricksConfig { @ConfigAttribute(env = "DATABRICKS_WORKSPACE_ID") private String workspaceId; - /** - * Flag to explicitly mark a host as a unified host. Note: This API is experimental and may change - * or be removed in future releases without notice. - */ - @ConfigAttribute(env = "DATABRICKS_EXPERIMENTAL_IS_UNIFIED_HOST") - private Boolean experimentalIsUnifiedHost; - @ConfigAttribute(env = "DATABRICKS_TOKEN", auth = "pat", sensitive = true) private String token; @@ -342,15 +335,6 @@ public DatabricksConfig setWorkspaceId(String workspaceId) { return this; } - public Boolean getExperimentalIsUnifiedHost() { - return experimentalIsUnifiedHost; - } - - public DatabricksConfig setExperimentalIsUnifiedHost(Boolean experimentalIsUnifiedHost) { - this.experimentalIsUnifiedHost = experimentalIsUnifiedHost; - return this; - } - public String getDatabricksCliPath() { return this.databricksCliPath; } From 1ffda2c7cd35648cffa45784893bc74040b07d8d Mon Sep 17 00:00:00 2001 From: Hector Castejon Diaz Date: Wed, 22 Apr 2026 08:57:53 +0000 Subject: [PATCH 2/2] Document unified host support and auto-discovery in README Completes the port of databricks/databricks-sdk-go#1641 and databricks/databricks-sdk-py#1358 by aligning README with what Go and Python published. Changes: - Add "Unified host support" entry + "GCP native authentication" entry to the auth "In this section" TOC. - List GCP as the third step of the default auth order, and explain that auth methods auto-skip when their required config is absent. Show how to force a method via DatabricksConfig.setAuthType. - Add a "Unified host support" section with a .databrickscfg example and equivalent Java code (WorkspaceClient + AccountClient sharing one profile, plus overriding workspace_id). - Extend the native-auth attribute table with workspace_id and discovery_url, mark account_id / workspace_id / discovery_url as auto-discoverable, and note that auto-discovery never overwrites explicit values. The Go/Python PRs also document a "cloud" config field; Java does not (yet) expose one on DatabricksConfig, so it is omitted here. Co-authored-by: Isaac --- README.md | 75 ++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 66 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 54182b945..328d2a665 100644 --- a/README.md +++ b/README.md @@ -87,8 +87,10 @@ workspace. // press for autocompletion ### In this section - [Default authentication flow](#default-authentication-flow) +- [Unified host support](#unified-host-support) - [Databricks native authentication](#databricks-native-authentication) - [Azure native authentication](#azure-native-authentication) +- [Google Cloud Platform native authentication](#google-cloud-platform-native-authentication) - [Overriding .databrickscfg](#overriding-databrickscfg) - [Additional authentication configuration options](#additional-authentication-configuration-options) @@ -98,9 +100,25 @@ If you run the [Databricks Terraform Provider](https://registry.terraform.io/pro 1. [Databricks native authentication](#databricks-native-authentication) 2. [Azure native authentication](#azure-native-authentication) -3. If the SDK is unsuccessful at this point, it returns an authentication error and stops running. +3. [Google Cloud Platform native authentication](#google-cloud-platform-native-authentication) +4. If the SDK is unsuccessful at this point, it returns an authentication error and stops running. -You can instruct the Databricks SDK for Java to use a specific authentication method by instantiating the `DatabricksConfig` class and setting the `auth_type` as described in the following sections. +Each authentication method requires specific configuration attributes (e.g., `token` for PAT auth, `azureClientId` for Azure service principal auth). The SDK automatically detects the cloud provider and skips authentication methods whose required configuration attributes are not present. This means that Azure-specific methods like `azure-cli` are automatically skipped when connecting to an AWS or GCP workspace, and vice versa for GCP-specific methods. + +To force a specific authentication method instead of relying on auto-detection, set the `authType` on `DatabricksConfig`: + +```java +import com.databricks.sdk.WorkspaceClient; +import com.databricks.sdk.core.DatabricksConfig; +... +// Force Azure CLI authentication — skip all other methods +DatabricksConfig config = new DatabricksConfig() + .setHost("https://mycompany.databricks.com") + .setAuthType("azure-cli"); +WorkspaceClient workspace = new WorkspaceClient(config); +``` + +This is useful when your environment has credentials for multiple authentication methods and you want to ensure a specific one is used, or when auto detection is not accurate. For each authentication method, the SDK searches for compatible authentication credentials in the following locations, in the following order. Once the SDK finds a compatible set of credentials that it can use, it stops searching: @@ -115,6 +133,41 @@ For each authentication method, the SDK searches for compatible authentication c Depending on the Databricks authentication method, the SDK uses the following information. Presented are the `WorkspaceClient` and `AccountClient` arguments (which have corresponding `.databrickscfg` file fields), their descriptions, and any corresponding environment variables. +### Unified host support + +Certain Databricks host types support both account-level and workspace-level API operations from a single endpoint. When using such a unified host, a single configuration profile can be used to create both `WorkspaceClient` and `AccountClient` instances without changing the `host`. + +For this to work, the following conditions must be met: + +1. The host must support unified operations. +2. Both `account_id` and `workspace_id` must be available — either set explicitly in the configuration or auto-discovered. + +When both values are present, the SDK uses `workspace_id` to route workspace-level requests and `account_id` to route account-level requests, all through the same host. + +```ini +# .databrickscfg +[unified] +host = https://mycompany.databricks.com +account_id = 00000000-0000-0000-0000-000000000000 +workspace_id = 1234567890 +``` + +```java +import com.databricks.sdk.AccountClient; +import com.databricks.sdk.WorkspaceClient; +import com.databricks.sdk.core.DatabricksConfig; +... +// Both clients share the same host and profile +WorkspaceClient workspace = new WorkspaceClient(new DatabricksConfig().setProfile("unified")); +AccountClient account = new AccountClient(new DatabricksConfig().setProfile("unified")); + +// A WorkspaceClient for a different workspace under the same host and account +WorkspaceClient otherWorkspace = new WorkspaceClient( + new DatabricksConfig().setProfile("unified").setWorkspaceId("2345678901")); +``` + +If the host supports it, `account_id` and `workspace_id` may be auto-discovered, reducing the required explicit configuration. + ### Databricks native authentication By default, the Databricks SDK for Java initially tries [Databricks token authentication](https://docs.databricks.com/dev-tools/api/latest/authentication.html) (`auth_type='pat'` argument). If the SDK is unsuccessful, it then tries Workload Identity Federation (WIF). See [Supported WIF](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-federation-provider) for the supported JWT token providers. @@ -123,13 +176,17 @@ By default, the Databricks SDK for Java initially tries [Databricks token authen - For Databricks OIDC authentication, you must provide the `host`, `client_id` and `token_audience` _(optional)_ either directly, through the corresponding environment variables, or in your `.databrickscfg` configuration file. - For Azure DevOps OIDC authentication, the `token_audience` is irrelevant as the audience is always set to `api://AzureADTokenExchange`. Also, the `System.AccessToken` pipeline variable required for OIDC request must be exposed as the `SYSTEM_ACCESSTOKEN` environment variable, following [Pipeline variables](https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#systemaccesstoken) -| Argument | Description | Environment variable | -|--------------|-------------|-------------------| -| `host` | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint. | `DATABRICKS_HOST` | -| `account_id` | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Only has effect when `Host` is either `https://accounts.cloud.databricks.com/` _(AWS)_, `https://accounts.azuredatabricks.net/` _(Azure)_, or `https://accounts.gcp.databricks.com/` _(GCP)_. | `DATABRICKS_ACCOUNT_ID` | -| `token` | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_. | `DATABRICKS_TOKEN` | -| `client_id` | _(String)_ The Databricks Service Principal Application ID. | `DATABRICKS_CLIENT_ID` | -| `token_audience` | _(String)_ When using Workload Identity Federation, the audience to specify when fetching an ID token from the ID token supplier. | `TOKEN_AUDIENCE` | +During initialization, the SDK automatically resolves missing configuration fields (`account_id`, `workspace_id`, and `discovery_url`) from the host metadata endpoint. Any explicitly provided values take precedence and are never overwritten. If the auto-discovery fails, the SDK falls back to the explicit configuration. It is recommended to always set explicit configuration. + +| Argument | Description | Environment variable | +|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------| +| `host` | _(String)_ The Databricks host URL for either the Databricks workspace endpoint or the Databricks accounts endpoint. | `DATABRICKS_HOST` | +| `account_id` | _(String)_ The Databricks account ID for the Databricks accounts endpoint. Auto-discovered if not provided. Has effect on hosts that serve account-level APIs. | `DATABRICKS_ACCOUNT_ID` | +| `workspace_id` | _(String)_ The Databricks workspace ID for the Databricks workspace endpoint. Auto-discovered if not provided. | `DATABRICKS_WORKSPACE_ID` | +| `discovery_url` | _(String)_ The OpenID Connect discovery URL. Auto-discovered if not provided. When set, OIDC endpoints are fetched directly from this URL instead of using the default host-based well-known endpoint logic. | `DATABRICKS_DISCOVERY_URL`| +| `token` | _(String)_ The Databricks personal access token (PAT) _(AWS, Azure, and GCP)_ or Azure Active Directory (Azure AD) token _(Azure)_. | `DATABRICKS_TOKEN` | +| `client_id` | _(String)_ The Databricks Service Principal Application ID. | `DATABRICKS_CLIENT_ID` | +| `token_audience` | _(String)_ When using Workload Identity Federation, the audience to specify when fetching an ID token from the ID token supplier. | `TOKEN_AUDIENCE` | For example, to use Databricks token authentication: