avoid NPEs
do not fail when the vertex does not contian edges
emit xml files as single lines
map only job that applies an xslt transformation to the input records and emits them as text
it seems we need to manage empty trust values...
implemented use of opt in/out rules for entity fields (#2557)
Keep working with the entity config for customizing XML serialization
#2557: only serialise fields as defined by the Index Config
Enables the fix for #2557: new two fields for the index configuration to include/exclude fields to be serialised in the final XML records.
tests for affiliation relationships
adjusted trust value
added project enrichment events
added journal attributes #2609
test for #2544: Datasets without instance URL in beta
datasources with piwik ids
avoiding to fail with NPE in case of missing relClass(es)
updated config for rel expansion to include result's collectedfrom
enrichment events includes also subjects
added topics for subjects
calculate the trust by comparing duplicates in a group, then scale the similarity value
tests for merged entities
refactor: easier way to build dedup rels
avoid NPE when trying to index missing content from HBase
test record aligned according to the latest transformation for OpenTrials
real opentrials example
added original ids in mock data used by tests
fixed invertion of rel source and target in case of one way relationships (symmetric = false)
align with latest dnet-actionmanager-common
partial refactoring of the job for the generation of broker events
Added test for jsonextrafield
[affiliation] invert rel source and target in case of one way relationships (symmetric = false)
updated tests
expand result url within relationships
test for #2319
funder original name field all in low-case letters
mock trust in enrichment events
fixed mock record used in tests
setting trust in OpenAireEventPayloadFactory
using lit. broker common module, added extra fields in EventMap
commented out code that sets a counter because we reach the max num of counters and the job fails
[broker] created factories for specific event type
#2192: fixed profile and more logs
#2192: entityregistry::* should end up with "other" datas ource type label for the portal. PrepareReduceFeeder now expects a 'ui.other.datasourcetypes' job param with the list of datasource types to be handled this way.
[broker] fixed inverted condition for ENRICH/ABSTRACT
[broker] fixed test
OpenAIRE broker DTOs moved in specific module: dnet-openaire-broker
[broker] avoid leading '/' character in the topic definition. Avoid '.' characters in map keys (elasticsearch doesn't like them)
[broker] implemented job for identification of enrichment events
Testing projects with funders with original name.
#2091: testing dataset claims
updated XML record file used for opentrials testing.
reverted to use json-java-format 1.2
reverted to previous revision
changed WRITE_TO_WAL = true for all jobs writing to HBase tables
added counters to keep track of the relationships provenance
new tst for claim updates
updated opentrial sample record
excluding dateoftransformation from metadata fields, it should be serialised only in the record header
Added dr:dateOfTransformation to some test XML files.For publications dr:dateOfCollection must be set.For datasets dri:dateOfCollections must be set.
Testing OpenTrials dataset record mapping. Depending on snapshot parent.
import cleanup
reverting, we need less getters
tests for dedup experiments
added more getters
dedup experiments
added mapper class for hdfs actions
cleanup
added Mapper class PromoteActionSetFromHDFS
added anchorStats map-only job
added counter for DOIs
removing useless counters
using most recent dnet-pace-core features
fixed DedupDeleteRelMapper
do not export deleted entities
adapted to the removal of contributors as relationships
added utility methods to deal with strings rather than byte[]
sort merged ids
log the documents being compared before failing
test for ARC
introducing support for projects that doesn't provide a link to a specific fundingpath.
implemented job and workflow to export the openaire identifiers
log the number of items clustered on each key
do not consider deleted entities
New test for openaire2.0_data compliance for datasets
updating to dnet-openaire-data-protos:3.5.0
updated to dnet-openaire-data-protos:3.5.0-SNAPSHOT