Cleaned family and automatic start of wf and meta-wf
Updated workflow to count sets from the values in the records and the sets configured in the profile.
Ensure indices in background
not using executors when ensuring indexes
cleaning
New blackboard action to create an oai store. This is needed by the wf for OAI feeding.
nicer workflow family and name
Fixed create index problem
now baseUrl is stored also if empty
fixed a return value
fixed sqoop java action
updated sql dao with full url; changed sqoop driver
extended mapping from protocolbuffer objects, implemented ExactMatch and overrideMatch configuration param
#637 introducing methods building organization identifiers based on organization name, organization entities and result to organization relations. Currently those methods are not hooked up with exporter main method.
utilizing renamed invertBidirectionalRelationAndBuild() method
Implemented, create index action
changed api
Pretty printing JSON representation of Avro records in case of error in TestingConsumer
introducing buildOaf(OafEntity oafEntity, DataInfo dataInfo) method for building inferenced entities
extending progress log interval from 10 000 to 100 000
fixed xpath using attributes
added a delete statement
changed a value
vocabulary dnet:datasourceCompatibilityLevel
shortening transformer_export_documentto* action names to be less than 50 characters
replacing "result" string with Type.result.name()
renamed driver compatibility to openaire_basic
adding uoa-espas-dataprovider-service
#354 hooking up primary/main workflow with documenttodataset and documenttoproject transformers skipping export of already existing relations in HBase
updating default job.properties
removing deprecated PersonWithInferencedData avro schema
#354 removing obsolete transformers/export/person transformer along with tests
removing json export test which became obsolete
removing deprecated DocumentWithInferencedData and DataSetReferenceWithInferencedData avro schemas
#354 removing obsolete transformers/export/inferenced_document_without_imported_data transformer along with tests
#354 removing obsolete transformers/export/identifier/referenceddatasets transformer along with tests
remove duplicated puma rule
set of modified transformationrules from skalny for the new beta
#354 removing obsolete transformers/export/identifier/documents transformer along with tests
#354 removing obsolete transformers/export/document transformer along with tests
temporary version
FP7ProjectID parameter also adds 'and (relfundinglevel0_id exact 'FP7')' filter
adding openSearch descriptor
new module
introducing FsShellPermissions utility class, utilizing proprly working chmod in metadataextraction cache
added extended config
branch for updating the mapping to protos, extend the configuration
set datasourceprefix to "webcrawl_wq___"
updated rule for csv import
t
m
added serialization, tests
instantiate one SAXReader for each call
fixed format-layout-interpretation concatenation,doesn't fail when the fieldExtractor returns a null result
added json serialization, builds the matching key one time only
updated profile to new OAI schema
using format-layout-interpretation to define the OAI store collection name
The expected name of collection is format-layout-interpretation
do not upsert sets here in the mapper: we shall delegate to a separate workflow to be run after the OAI feeding is completed.
early implementation of jar upload script
Always new records to test how faster we go
fixed a problem with lower case
renaming root element from list to citations