airflow.providers.google.cloud.hooks.datastore

This module contains Google Datastore hook.

Classes

DatastoreHook

Interact with Google Cloud Datastore. This hook uses the Google Cloud connection.

Module Contents

class airflow.providers.google.cloud.hooks.datastore.DatastoreHook(gcp_conn_id='google_cloud_default', api_version='v1', impersonation_chain=None, **kwargs)[source]

Bases: airflow.providers.google.common.hooks.base_google.GoogleBaseHook

Interact with Google Cloud Datastore. This hook uses the Google Cloud connection.

This object is not threads safe. If you want to make multiple requests simultaneously, you will need to create a hook per thread.

Parameters:

api_version (str) – The version of the API it is going to connect to.

connection = None[source]
api_version = 'v1'[source]
get_conn()[source]

Establish a connection to the Google API.

Returns:

a Google Cloud Datastore service object.

Return type:

googleapiclient.discovery.Resource

allocate_ids(partial_keys, project_id)[source]

Allocate IDs for incomplete keys.

See also

https://cloud.google.com/datastore/docs/reference/rest/v1/projects/allocateIds

Parameters:

  • partial_keys (list) – a list of partial keys.

  • project_id (str) – Google Cloud project ID against which to make the request.

Returns:

a list of full keys.

Return type:

list

begin_transaction(project_id, transaction_options)[source]

Begins a new transaction.

See also

https://cloud.google.com/datastore/docs/reference/rest/v1/projects/beginTransaction

Parameters:

  • project_id (str) – Google Cloud project ID against which to make the request.

  • transaction_options (dict[str, Any]) – Options for a new transaction.

Returns:

a transaction handle.

Return type:

str

commit(body, project_id)[source]

Commit a transaction, optionally creating, deleting or modifying some entities.

See also

https://cloud.google.com/datastore/docs/reference/rest/v1/projects/commit

Parameters:

  • body (dict) – the body of the commit request.

  • project_id (str) – Google Cloud project ID against which to make the request.

Returns:

the response body of the commit request.

Return type:

dict

lookup(keys, project_id, read_consistency=None, transaction=None)[source]

Lookup some entities by key.

See also

https://cloud.google.com/datastore/docs/reference/rest/v1/projects/lookup

Parameters:

  • keys (list) – the keys to lookup.

  • read_consistency (str | None) – the read consistency to use. default, strong or eventual. Cannot be used with a transaction.

  • transaction (str | None) – the transaction to use, if any.

  • project_id (str) – Google Cloud project ID against which to make the request.

Returns:

the response body of the lookup request.

Return type:

dict

rollback(transaction, project_id)[source]

Roll back a transaction.

See also

https://cloud.google.com/datastore/docs/reference/rest/v1/projects/rollback

Parameters:

  • transaction (str) – the transaction to roll back.

  • project_id (str) – Google Cloud project ID against which to make the request.

run_query(body, project_id)[source]

Run a query for entities.

See also

https://cloud.google.com/datastore/docs/reference/rest/v1/projects/runQuery

Parameters:

  • body (dict) – the body of the query request.

  • project_id (str) – Google Cloud project ID against which to make the request.

Returns:

the batch of query results.

Return type:

dict

get_operation(name)[source]

Get the latest state of a long-running operation.

See also

https://cloud.google.com/datastore/docs/reference/data/rest/v1/projects.operations/get

Parameters:

name (str) – the name of the operation resource.

Returns:

a resource operation instance.

Return type:

dict

delete_operation(name)[source]

Delete the long-running operation.

See also

https://cloud.google.com/datastore/docs/reference/data/rest/v1/projects.operations/delete

Parameters:

name (str) – the name of the operation resource.

Returns:

none if successful.

Return type:

dict

poll_operation_until_done(name, polling_interval_in_seconds)[source]

Poll backup operation state until it’s completed.

Parameters:

  • name (str) – the name of the operation resource

  • polling_interval_in_seconds (float) – The number of seconds to wait before calling another request.

Returns:

a resource operation instance.

Return type:

dict

export_to_storage_bucket(bucket, project_id, namespace=None, entity_filter=None, labels=None)[source]

Export entities from Cloud Datastore to Cloud Storage for backup.

Note

Keep in mind that this requests the Admin API not the Data API.

See also

https://cloud.google.com/datastore/docs/reference/admin/rest/v1/projects/export

Parameters:

  • bucket (str) – The name of the Cloud Storage bucket.

  • namespace (str | None) – The Cloud Storage namespace path.

  • entity_filter (dict | None) – Description of what data from the project is included in the export.

  • labels (dict[str, str] | None) – Client-assigned labels.

  • project_id (str) – Google Cloud project ID against which to make the request.

Returns:

a resource operation instance.

Return type:

dict

import_from_storage_bucket(bucket, file, project_id, namespace=None, entity_filter=None, labels=None)[source]

Import a backup from Cloud Storage to Cloud Datastore.

Note

Keep in mind that this requests the Admin API not the Data API.

See also

https://cloud.google.com/datastore/docs/reference/admin/rest/v1/projects/import

Parameters:

  • bucket (str) – The name of the Cloud Storage bucket.

  • file (str) – the metadata file written by the projects.export operation.

  • namespace (str | None) – The Cloud Storage namespace path.

  • entity_filter (dict | None) – specify which kinds/namespaces are to be imported.

  • labels (dict | str | None) – Client-assigned labels.

  • project_id (str) – Google Cloud project ID against which to make the request.

Returns:

a resource operation instance.

Return type:

dict

Was this entry helpful?