academic_observatory_workflows.orcid_telescope.release

Classes

OrcidRelease

Functions

orcid_batch_names(→ List[str])

Create a list of all the possible ORCID directories

Module Contents

class academic_observatory_workflows.orcid_telescope.release.OrcidRelease(*, dag_id: str, run_id: str, cloud_workspace: observatory_platform.airflow.workflow.CloudWorkspace, bq_dataset_id: str, bq_main_table_name: str, bq_upsert_table_name: str, bq_delete_table_name: str, start_date: pendulum.DateTime, end_date: pendulum.DateTime, prev_release_end: pendulum.DateTime, prev_latest_modified_record: pendulum.DateTime, is_first_run: bool)[source]

Bases: observatory_platform.airflow.release.ChangefileRelease

cloud_workspace[source]

bq_dataset_id[source]

bq_main_table_name[source]

bq_upsert_table_name[source]

bq_delete_table_name[source]

prev_release_end[source]

prev_latest_modified_record[source]

is_first_run[source]

master_manifest_file[source]

upsert_blob_glob = 'Uninferable/*_upsert.jsonl.gz'[source]

upsert_table_uri[source]

delete_blob_glob = 'Uninferable/*_delete.jsonl.gz'[source]

delete_table_uri[source]

bq_main_table_id[source]

bq_upsert_table_id[source]

bq_delete_table_id[source]

bq_snapshot_table_id[source]

property upsert_files[source]

property delete_files[source]

property downloaded_records[source]

property orcid_directory_paths: List[str][source]: Generates the paths to the orcid directories in the download folder

orcid_batches() → List[academic_observatory_workflows.orcid_telescope.batch.OrcidBatch][source]: Creates the orcid directories in the download folder if they don’t exist and returns them

to_dict() → Dict[source]

static from_dict(dict_: Dict)[source]

academic_observatory_workflows.orcid_telescope.release.orcid_batch_names() → List[str][source]

Create a list of all the possible ORCID directories

Returns:: A list of all the possible ORCID directories