Compute

Access modes

Available access modes vary by whether Unity Catalog is enabled or not

With Unity Catalog enabled, standard and dedicated modes are recommended.
- standard access mode
  - Libraries and init scripts must be added to the allowlist to use them
  - allowlist cannot be disabled
  - Instance profiles are unavailable, specifically IMDS is unavailable
    - Service credentials can be used as an alternative of instance profiles.
- dedicated access mode
- Standard compute runs user code in full isolation with no access to lower-level resources.
  - See Lakeguard for the details
The legacy access mode no isolation shared is also available
- Unity Catalog is disabled on the compute

Unity Catalog is account-level resource
- Only one Unity Catalog can be created per region
The default catalog varies by whether Unity Catalog is enabled or not (doc)
- Unity Catalog enabled -> the workspace catalog
- Unity Catalog disabled -> hive_metastore

Use a role registered in service credentials for aws client

boto3_session = boto3.Session(
  botocore_session=dbutils.credentials.getServiceCredentialsProvider(credential_name)
  ...
)

dbutils provides utility commands for notebooks.

Top