Which Interface Endpoint Need to Be Enabled in AWS for Working of Private Link in Databricks

The interface endpoint you must enable for AWS PrivateLink to work with Databricks is the one for the Databricks workspace service, identified by the service name com.amazonaws..databricks for classic workspaces, or a VPC endpoint service name provided by Databricks for serverless and account-level access. Enabling the wrong endpoint, such as one for S3 or STS only, leaves workspace traffic on the public internet. The correct VPC endpoint is the gateway to the Databricks control plane, and it must be paired with the workspace's PrivateLink configuration in the account console.

PrivateLink for Databricks is not a single switch. It is a chain of AWS VPC endpoints, security group rules, DNS settings, and workspace configuration that must all align. The interface endpoint question is the entry point to that chain, but getting it right requires understanding how Databricks splits its data plane from its control plane, how AWS resolves endpoint service names, and why some architectures need more than one endpoint. This explainer walks through the mechanism at a level deeper than a checklist, quantifies the traffic paths, isolates the edge cases that break deployments, and gives concrete verification steps.

How a Databricks workspace rides an AWS interface endpoint

AWS PrivateLink exposes a service to your VPC through an elastic network interface, not through an internet gateway or a NAT device. When you create an interface endpoint, AWS provisions one ENI per subnet in the specified availability zones, each with a private IP from your subnet CIDR. Traffic to the endpoint service stays on the AWS backbone and never traverses the public internet. The endpoint has a security group that must allow inbound TCP 443 from your client subnets, and the endpoint policy must permit the actions your clients need.

Databricks operates as a managed service with a split architecture. The classic workspace data plane runs inside a Databricks-managed VPC that you do not control, while the control plane, which handles the web UI, REST API, cluster management, and notebook metadata, is the front door. PrivateLink for Databricks connects your VPC to that control plane front door, so that browser sessions, API calls, and ODBC or JDBC connections from your own network resolve to private IPs instead of public Databricks hostnames.

The endpoint service name follows the standard AWS pattern. For classic workspaces in most commercial regions, the service is published as com.amazonaws..databricks. When you create the endpoint, AWS shows you the service name from the Databricks documentation or from the account console during PrivateLink setup. The endpoint type is Interface, because Databricks publishes its control plane through a Network Load Balancer that PrivateLink fronts. Gateway endpoints, such as those for S3 or DynamoDB, cannot serve this purpose because they only handle traffic to specific AWS services through route table entries.

The DNS layer is where most failures hide. When you enable private DNS on the interface endpoint, AWS creates a private hosted zone that overrides the public resolution of the Databricks hostnames within your VPC. Without private DNS, the endpoint exists but your clients still resolve public IPs, so traffic exits through the internet gateway. The endpoint policy and security group then govern whether that private traffic is allowed. In practice, three objects must agree: the interface endpoint, the security group attached to it, and the workspace's PrivateLink registration in the Databricks account console.

A useful mental model is a private door into a shared hallway. The hallway is the Databricks control plane. The door is the interface endpoint. Your VPC is the room on the other side. AWS gives you a private IP for the door, and private DNS makes the room's map point to that door instead of the public street address. The security group is the lock, and the endpoint policy is the guest list.

Quantified context: latency, throughput, and cost tradeoffs

PrivateLink changes the path, not the protocol. Each interface endpoint ENI supports the full bandwidth of the underlying network interface, and AWS does not impose a fixed throughput ceiling per endpoint beyond the limits of the instance types and subnet capacity. In practice, the bottleneck moves to the NAT gateway or internet gateway you no longer use, or to the security group and endpoint policy inspection overhead, which is negligible for HTTPS.

Latency gains depend on where your clients sit. A client in the same AWS region as the Databricks control plane avoids the public internet hop, which for cross-region or on-premises paths can cut round-trip time from tens of milliseconds to single digits. A client already in the same region sees a smaller improvement, because the public path was short to begin with. The most dramatic gains appear for hybrid architectures, where an on-premises data center reaches Databricks through Direct Connect or Site-to-Site VPN into a VPC that hosts the interface endpoint.

Cost has two components. AWS bills interface endpoint hours per AZ, and data processing per GB in each direction. The endpoint hour and per-GB rates vary by region and are published in the AWS PrivateLink pricing page, so check the current schedule rather than relying on a cached figure. The offset is that you stop paying NAT gateway data processing charges for Databricks-bound traffic, and NAT gateway charges are typically higher per GB than interface endpoint charges. For a workload pushing hundreds of gigabytes per month through a NAT gateway, replacing that path with a PrivateLink endpoint usually reduces spend.

Availability zone placement matters for both latency and cost. Deploy the endpoint in at least two AZs so that a single AZ failure does not sever control plane access. Databricks control plane endpoints are regional, so AWS balances across the AZs where you created ENIs. A single-AZ endpoint configuration is a common cost optimization that quietly becomes an availability risk.

Timeline: what happens as you enable the endpoint and register PrivateLink

Each step has a failure mode that looks identical at the surface: the workspace loads slowly, the API times out, or the UI returns a public IP. Step 5 is the one teams skip. Creating the AWS endpoint without registering it in the Databricks account console leaves the control plane unaware of the private path, and the workspace either rejects the connection or silently falls back.

Registration also determines which workspace subnets and which VPC endpoint IDs Databricks accepts. The account console requires the VPC endpoint ID, the region, and the subnets where ENIs exist. A mismatch in region between the endpoint and the workspace is a hard failure, because PrivateLink is regional and Databricks does not proxy across regions.

Edge cases: serverless, multi-workspace, and non-classic paths

The classic workspace answer does not cover every Databricks deployment model. Serverless compute and some account-level features connect through different PrivateLink services. Databricks documents separate endpoint service names for serverless SQL and serverless compute paths, and those services must be enabled in addition to the classic workspace endpoint when you use those features. Treat the endpoint list as a set, not a single item, and check the Databricks PrivateLink documentation for the exact service name strings in your region, because they are not interchangeable.

Multi-workspace accounts compound the problem. Each workspace can have its own PrivateLink configuration, and a VPC can host endpoints for several workspaces only if the endpoint policies and security groups permit traffic to each. A single endpoint cannot serve two workspaces with conflicting endpoint policies. The safe pattern is one endpoint per workspace per VPC, or a shared endpoint with a policy that explicitly allows each workspace's control plane.

On-premises and cross-VPC clients add a DNS wrinkle. Private DNS on an interface endpoint only affects resolution inside the VPC that owns the endpoint. Clients in a peered VPC or on-premises network need DNS forwarding rules that point Databricks hostnames at the endpoint ENIs' private IPs or at a Route 53 Resolver inbound endpoint. Skip this and the clients resolve public IPs while the endpoint sits idle, which is one of the most common misdiagnoses of "PrivateLink is not working."

The counterintuitive caveat: a correctly enabled interface endpoint can still leave traffic on the public internet if private DNS is disabled or if the client bypasses the VPC resolver. PrivateLink is a routing mechanism, not a firewall. The endpoint does not intercept traffic that resolves elsewhere.

Another edge case is endpoint policy versus IAM. The endpoint policy is evaluated before IAM for traffic through the endpoint, and a restrictive policy can block legitimate Databricks control plane calls even when IAM roles are correct. Audit the policy alongside the security group whenever a new workspace or feature is onboarded.

Misconceptions that break PrivateLink deployments

The S3 gateway endpoint is enough. It is not. S3 gateway endpoints serve data plane storage traffic and do nothing for the Databricks control plane. One endpoint covers all features. Serverless and account-level paths may need separate service names, so enumerate every feature you use. Private DNS is optional. Without it, clients resolve public hostnames and the endpoint is bypassed entirely. The security group on the endpoint is the only gate. The endpoint policy and the Databricks account console registration are equally authoritative. Interface endpoints work across regions. PrivateLink is regional; a cross-region workspace requires the endpoint in the workspace's region. Enabling PrivateLink removes all public exposure. The workspace may still expose public endpoints unless you disable public access in the workspace configuration.

Each bullet maps to a distinct failure signature. The S3-only mistake produces a workspace that loads over the internet while data plane storage looks private. The missing private DNS produces timeouts that vanish when a client outside the VPC tries the same request. The registration gap produces intermittent UI failures that correlate with control plane calls but not with data reads.

Practical actions: verify the endpoint chain in order

Start with the service name. Open the Databricks account console PrivateLink setup page and copy the exact service name, then open the AWS VPC console and create an Interface endpoint with that string. Do not type the name from memory, because a single character difference produces an endpoint that never leaves the pending state.

Next, scope the security group. Allow inbound TCP 443 only from the CIDR ranges of your client subnets or from a security group reference. A 0.0.0.0/0 inbound rule defeats much of the purpose, because any host that reaches the ENI can use the private path.

Then enable private DNS and confirm resolution. From an instance in the VPC, resolve the workspace hostname and check that the answer is a private IP inside your subnet range. If it is a public address, the private hosted zone is not associated or the endpoint is not in the resolver's path.

Register the endpoint in the Databricks account console with the endpoint ID, region, and subnets, then wait for the workspace to report PrivateLink as configured. A registration that stays pending usually indicates a service name mismatch or an endpoint in the wrong region.

Finally, validate the path. Use connectivity tests from each client subnet, not just one, because an endpoint deployed in two AZs can still be unreachable from a third subnet if the route table or network ACL blocks it. Re-test after any workspace or account-level feature change, since new features can require new service names.

Set a review threshold: any workspace onboarding should include a checklist that names every endpoint service, the security group, the private DNS setting, and the console registration. Treat a missing item as a release blocker, not a follow-up ticket.

Limitations and what this cannot settle

PrivateLink behavior depends on the Databricks deployment model, the region, and the AWS account layout, and the exact service name strings change as Databricks adds features. This article explains the stable mechanism and the verification sequence, but it cannot substitute for the current Databricks PrivateLink documentation or your AWS account's endpoint catalog. Where pricing or service availability is dated, confirm against the AWS PrivateLink pricing page and the Databricks release notes for your region. Any architecture that mixes serverless, classic, and account-level access should be validated in a non-production workspace before rollout. The security posture also depends on disabling public workspace access separately, so PrivateLink alone does not guarantee a private-only deployment.