DEV Community

Cover image for Config Secret Value Use SDK Python Databricks Windows
chuongmep
chuongmep

Posted on

Config Secret Value Use SDK Python Databricks Windows

Introduction

The Databricks SDK for Python simplifies interaction with Databricks services, enabling developers to programmatically manage resources, execute workflows, and handle data securely. Setting up the SDK on a Windows environment involves straightforward steps to install, configure, and utilize its features effectively.

This guide provides an overview of how to install the Databricks SDK for Python, followed by an example to retrieve the list of secret scopes and keys. These secret scopes are crucial for securely managing credentials and sensitive data within your Databricks environment.

By the end of this guide, you'll be equipped with the foundational steps to interact with Databricks securely and efficiently in a Windows environment.

Install Databricks Cli

To install fastest version of Databricks-CLI, you need to download from https://github.com/databricks/cli/releases

Image description

After download, you need to setup the enviroment path in Windows to able to use command databricks

  1. Choose edit Path in Environment Variables
  2. Add your zip file downloaded and extracted in Databricks GitHub

After complete download, you need to do login to authentication

databricks auth login --host <workspace-url>
Enter fullscreen mode Exit fullscreen mode

Image description

When you get the information auth is successfully saved, you can check the config at %USERPROFILE%\.databrickscfg

Now you can go to next part to be able to use Databricks python SDK

Install Databricks SDK Python

pip install databricks-sdk --upgrade
Enter fullscreen mode Exit fullscreen mode

Get the list of scopes

from databricks.sdk.runtime import dbutils
for secret_scope in dbutils.secrets.listScopes():
    for secret_metadata in dbutils.secrets.list(secret_scope.name):
        print(f'found {secret_metadata.key} secret in {secret_scope.name} scope')
Enter fullscreen mode Exit fullscreen mode

Usage Databricks Python SDK

Create a scope

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
scope_name = 'aps-dev'
w.secrets.create_scope(scope=scope_name)
Enter fullscreen mode Exit fullscreen mode

Put a secret key and value

w.secrets.put_secret("aps-dev","<key>",string_value="<value>")
Enter fullscreen mode Exit fullscreen mode

Get a secret by key and value

dbutils.secrets.get(scope = "<scope name>", key = "<key>")
Enter fullscreen mode Exit fullscreen mode

Delete a secret by key

w.secrets.delete_secret(scope_name, "<key name>")
Enter fullscreen mode Exit fullscreen mode

Get all scopes

dbutils.secrets.listScopes()
Enter fullscreen mode Exit fullscreen mode

Get all keys from scope name

dbutils.secrets.list("<key name>")
Enter fullscreen mode Exit fullscreen mode

How to use in Databricks

To use in Databricks notebook, you don't need to import the library to use

Get List Scopes

dbutils.secrets.listScopes()
Enter fullscreen mode Exit fullscreen mode

Get a secret by key and value

dbutils.secrets.get(scope = "<scope name>", key = "<key>")
Enter fullscreen mode Exit fullscreen mode

When you try to see the value secret of key, you will see it REDACTED
Image description

Done, now you use your scope secret in Databricks notebook !

Reference :

Top comments (0)