AWS Glue Data Catalog¶

The AWS Glue Data Catalog is a centralized metadata repository for data assets. Use the operators below to manage Glue Data Catalog resources.

Create a Catalog Database¶

To create a database in the AWS Glue Data Catalog, use GlueCatalogCreateDatabaseOperator.

tests/system/amazon/aws/example_glue_catalog.py[source]

create_database = GlueCatalogCreateDatabaseOperator(
    task_id="create_database",
    database_name=db_name,
    description="Test database for Glue Catalog",
)

Reference¶

AWS boto3 Library Documentation for Glue

Create a Table¶

To create a table in an AWS Glue Data Catalog database, use GlueCatalogCreateTableOperator.

tests/system/amazon/aws/example_glue_catalog.py[source]

create_table = GlueCatalogCreateTableOperator(
    task_id="create_table",
    database_name=db_name,
    table_name=table_name,
    table_input=table_input,
)

Delete a Catalog Database¶

To delete a database from the AWS Glue Data Catalog, use GlueCatalogDeleteDatabaseOperator.

tests/system/amazon/aws/example_glue_catalog.py[source]

delete_database = GlueCatalogDeleteDatabaseOperator(
    task_id="delete_database",
    database_name=db_name,
    trigger_rule=TriggerRule.ALL_DONE,
)

Delete a Table¶

To delete a table from an AWS Glue Data Catalog database, use GlueCatalogDeleteTableOperator.

tests/system/amazon/aws/example_glue_catalog.py[source]

delete_table = GlueCatalogDeleteTableOperator(
    task_id="delete_table",
    database_name=db_name,
    table_name=table_name,
    trigger_rule=TriggerRule.ALL_DONE,
)