Access Iceberg tables managed in AWS Glue Iceberg REST catalog and S3 Tables

Memo to access Iceberg tables managed in AWS Glue Iceberg REST catalog and S3 Tables.

I used DuckDB 1.4.3.

Setup

  1. Install and load extensions
    INSTALL aws;
    INSTALL httpfs;
    INSTALL iceberg;
    LOAD iceberg;
    
  2. Create a secret that is used to access the Iceberg REST catalog and tables
    CREATE SECRET (
     TYPE s3,
     PROVIDER credential_chain,
     CHAIN sts,
     ASSUME_ROLE_ARN 'arn:aws:iam::account_id:role/role',
     REGION 'us-east-1'
    );
    
  3. Connect to the catalog
    ATTACH 'account_id' AS glue_catalog (
     TYPE iceberg,
     ENDPOINT 'glue.REGION.amazonaws.com/iceberg',
     AUTHORIZATION_TYPE 'sigv4'
    );
    -- for federated catalogs of s3 table buckets
    ATTACH 'account_id:s3tablescatalog/table_bucket_name' AS glue_catalog (
     TYPE iceberg,
     ENDPOINT_TYPE 'glue'
    );
    

Note: To create tables in Glue Iceberg REST catalog, you need to specify ENDPOINT_TYPE instead of ENDPOINT or disable support_stage_create explicitly. support_stage_create is not supported in Glue Iceberg REST catalog and it is enabled by default when ENDPOINT is specified.

Check

The document suggests to use show all tables to see attached tables, but it didn’t return attached tables.

Query

To access attached Iceberg tables, you need to specify a table by catalog.namespace.table format.

select count(*) from glue_catalog.tpch.nation;
D select count(*) from glue_catalog.test_db.test_tbl;
+--------------+
| count_star() |
+--------------+
| 100          |
+--------------+

Links