D-Net

updating job properties

setting export_action_set_id_entity_dataset to $UNDEFINED$ by default, this should not be required because dataset reference extraction module might be deactivated. Check will be performed at dataset entity exporter module and when value is not set - exception will be raised.

#757 temporarily disabling PMC ingestion until fixing openaire identifiers building process

#568, #577 enabling proper citations export by introducing PMC citation ingestion and citation matching outcome merging and grouping for exporting purposes. Introducing union instead of collapser which should be introduced in near future.

removing output_citation_pmc port duplicate

removing common import input parameters which are not required in this context

introducing trust_level_threshold support in statistics workflow

introducing trust_level_threshold support in common import workflow

providing default value for action_set_id_entity_dataset set to $UNDEFINED$. This change is required when exporting in statistics export mode where no entities are exported and such parameter should not be required.

introducing dedicated statistics mainworkflow encapsulating importing, processing and exporting phases. This workflow was introduced explicitly for statistics purposes because we want to operate over InformationSpace imported data in contrary to primary workflow where some of the statistics input was inferred and it wasn't clear whether it will become part of InformationSpace.

allowing overriding inference_provenance_blacklist default 'iis' value which will be required in mainworkflows/statistcs where inferenced document to project relations should be taken into account

setting default $undefined$ value for 'input_aux_dataset_existing_id'

setting default undefined values for 'mdstore_service_location' and 'dataset_mdstore_ids_csv'

#9059 reverting #717 change: shortening app_path for primary workflow due to the fix applied by Paweł on WF_JOBS MODIFY mysql table: canging varchar(255) to mediumtext.

updating job.properties

fixing placeholder name

#717 shortening app_path for primary workflow

updating job.properties

fixing output port names: removing default values for citation_pmc and dataset, setting proper output_citation_pmc in both preprocessing and primary workflows

updating job.properties

#717 shortening app_path for preprocessing workflow and subworkflows

#712 introducing plaintext caching

shortening node names

updating workingDir for generating empty outputs: removing import_dataset part

skipping PMC citations ingestion when citationmatching algorithm is not enabled

shortening transformer_export_documentto* action names to be less than 50 characters

#354 hooking up primary/main workflow with documenttodataset and documenttoproject transformers skipping export of already existing relations in HBase

updating default job.properties

#486 introducing last piece missing: text collapser in front of referenceextraction_researchinitiatives joining text contents coming from already existing document_text input port and newly introduced document_text_wos input port providing WoS contents

#486 bugfix: reordering existence filter with id relacer: we need to update identifiers first, then update existence filter

intregrating pmc citations ingestion with primary workflow, adjust port names, deduplicating dependencies

updating default job.properties

renaming input ports from input_citation to input_citations to be aligned with exporter subworkflow

skipping exporting citation matching outcome

renaming input ports from input_citation to input_citations to be aligned with exporter subworkflow

updating default job.properties

setting excluded_ids to undefined value

setting excluded_ids to undefined value

setting excluded_ids to undefined value

setting excluded_ids to undefined value