wf to harvest datacite sets
added objectStoreBlacklistCSV to wf definition
refactored submit job nodes, introducing AdminJobNodes
added doaj as entity registry wf
added waitAll node
added copytable workflow
added workflow to export datasets directly on HBASE (zenodo)
added workflow shortcut to avoid exporting all the md record on hdfs, allows to reuse the intermediate data stored during the previous run
links to external editors in wf details pages
Removed dedup meta workflow that re-use workflows used in the dm metaworkflow. If different metaworkflows re-use the same wfs, those wfs are executed multiple times.
updated layout for the mdstore holding claimed records
derp indentation
renamed wf
updated wf for dataset records
Removed metaworkflow to avoid undesired multiple execution of workflows triggered by the current implementation of msro based on issn notification.
added descriptions
updated wf definition
fixed wf params
dedicated action sets to avoid overlapping
IMEF to ODF and webcrawl store is OAF .
fix invalid profile
merged branch newIndexFeed to trunk
updated provision workflow
added created index node
indexing wf uses the new index service specs
stats update moved to dedicated wf
added flags to enable/disable metadata extraction module
added action sets dedicated to each inference module
aligned parameters with job specs
updated IIS specific property
Fixed bug with datatasource to hbase wf
updated provision wf
reused wf definitions instead of dubling them in a dedicated load wf
added compound index on funder and oaftype for OAI
Fixed json parameters for OAI M/R job node
Added node to set the new current OAI db
Fixed parameters
labels
SetFormatInfoJobNode takes three optional parameters that specify the name of the env properties where format, layout, and intepretation will be put.This is useful to the dm.provision wf, where the OAI nodes need the triple "oaf-index-openaire" instead of "DMF|TMF-index-openaire".
Fixed target node
Updated workflows for OAI feed/post feed.
early definition of overall provision workflow
Context profiles will be fetched by the oozie process, so we pass the isLookupEndpoint as wf param
updated datacite workflows
Updated OAI wf: before the feed we have to drop the store because we can't perform upserts in the M/R job: too expensive.
updated stats workflow
aligned stats wf definition with param specification
updated stats update workflow
factored out job nodes to set the context profile list, and the entity link table. defined stats update workflow
added param names to wf definition
allow to override the xquery to discovery the search service at runtime.
Added post feed workflow to perform operations such as creating indices, counting sets, after the OAI store has been fed.
New blackboard action to create an oai store. This is needed by the wf for OAI feeding.
nicer workflow family and name
idScheme and idNamespace defined as part of the OAI configuration profile
Need to pass idscheme and namespace parameter to the job.
submittable M/R OAI feeding job
simplified index switch wf
manual start
added wf to perform only index switch (BB msg to the search service)
renamed meta wf definition file
user parameter for the hdfs filepath to use as input of the OAI feed
generalized wf node
renamed wf file name
fixed wf definition
refactor
fixed wf syntax