Introduction
The Databricks SDK for Python simplifies interaction with Databricks services, enabling developers to programmatically manage resources, execute workflows, and handle data securely. Setting up the SDK on a Windows environment involves straightforward steps to install, configure, and utilize its features effectively.
This guide provides an overview of how to install the Databricks SDK for Python, followed by an example to retrieve the list of secret scopes and keys. These secret scopes are crucial for securely managing credentials and sensitive data within your Databricks environment.
By the end of this guide, you'll be equipped with the foundational steps to interact with Databricks securely and efficiently in a Windows environment.
Install Databricks Cli
To install fastest version of Databricks-CLI, you need to download from https://github.com/databricks/cli/releases
After download, you need to setup the enviroment path in Windows to able to use command databricks
- Choose edit Path in Environment Variables
- Add your zip file downloaded and extracted in Databricks GitHub
After complete download, you need to do login to authentication
databricks auth login --host <workspace-url>
When you get the information auth is successfully saved, you can check the config at %USERPROFILE%\.databrickscfg
Now you can go to next part to be able to use Databricks python SDK
Install Databricks SDK Python
pip install databricks-sdk --upgrade
Get the list of scopes
from databricks.sdk.runtime import dbutils
for secret_scope in dbutils.secrets.listScopes():
for secret_metadata in dbutils.secrets.list(secret_scope.name):
print(f'found {secret_metadata.key} secret in {secret_scope.name} scope')
Usage Databricks Python SDK
Create a scope
from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
scope_name = 'aps-dev'
w.secrets.create_scope(scope=scope_name)
Put a secret key and value
w.secrets.put_secret("aps-dev","<key>",string_value="<value>")
Get a secret by key and value
dbutils.secrets.get(scope = "<scope name>", key = "<key>")
Delete a secret by key
w.secrets.delete_secret(scope_name, "<key name>")
Get all scopes
dbutils.secrets.listScopes()
Get all keys from scope name
dbutils.secrets.list("<key name>")
How to use in Databricks
To use in Databricks notebook, you don't need to import the library to use
Get List Scopes
dbutils.secrets.listScopes()
Get a secret by key and value
dbutils.secrets.get(scope = "<scope name>", key = "<key>")
When you try to see the value secret of key, you will see it REDACTED
Done, now you use your scope secret in Databricks notebook !
Reference :
- https://docs.databricks.com/en/dev-tools/sdk-python.html
- https://docs.databricks.com/en/dev-tools/auth/config-profiles.html
- https://databricks-sdk-py.readthedocs.io/en/latest/dbutils.html
- https://docs.databricks.com/en/dev-tools/auth/pat.html
- https://docs.databricks.com/api/workspace/secrets/listscopes
- https://docs.databricks.com/api/workspace/secrets
Top comments (0)