D-Net

[maven-release-plugin] copy for tag dnet-mapreduce-jobs-0.0.9.6-PIWIK

datasources with piwik ids

avoiding to fail with NPE in case of missing relClass(es)

updated config for rel expansion to include result's collectedfrom

enrichment events includes also subjects

added topics for subjects

calculate the trust by comparing duplicates in a group, then scale the similarity value

tests for merged entities

refactor: easier way to build dedup rels

avoid NPE when trying to index missing content from HBase

test record aligned according to the latest transformation for OpenTrials

real opentrials example

added original ids in mock data used by tests

fixed invertion of rel source and target in case of one way relationships (symmetric = false)

align with latest dnet-actionmanager-common

partial refactoring of the job for the generation of broker events

Added test for jsonextrafield

[affiliation] invert rel source and target in case of one way relationships (symmetric = false)

updated tests

expand result url within relationships

test for #2319

funder original name field all in low-case letters

mock trust in enrichment events

fixed mock record used in tests

setting trust in OpenAireEventPayloadFactory

using lit. broker common module, added extra fields in EventMap

commented out code that sets a counter because we reach the max num of counters and the job fails

[broker] created factories for specific event type

#2192: fixed profile and more logs

#2192: entityregistry::* should end up with "other" datas ource type label for the portal. PrepareReduceFeeder now expects a 'ui.other.datasourcetypes' job param with the list of datasource types to be handled this way.

[broker] fixed inverted condition for ENRICH/ABSTRACT

[broker] fixed test

OpenAIRE broker DTOs moved in specific module: dnet-openaire-broker

[broker] avoid leading '/' character in the topic definition. Avoid '.' characters in map keys (elasticsearch doesn't like them)

[broker] implemented job for identification of enrichment events

[broker] implemented job for identification of enrichment events

Testing projects with funders with original name.

#2091: testing dataset claims

updated XML record file used for opentrials testing.

reverted to use json-java-format 1.2

reverted to previous revision

changed WRITE_TO_WAL = true for all jobs writing to HBase tables

added counters to keep track of the relationships provenance

new tst for claim updates

updated opentrial sample record

excluding dateoftransformation from metadata fields, it should be serialised only in the record header

Added dr:dateOfTransformation to some test XML files.
For publications dr:dateOfCollection must be set.
For datasets dri:dateOfCollections must be set.

Testing OpenTrials dataset record mapping. Depending on snapshot parent.

import cleanup

reverting, we need less getters

tests for dedup experiments

added more getters

dedup experiments

added mapper class for hdfs actions

cleanup

added Mapper class PromoteActionSetFromHDFS

added anchorStats map-only job

added counter for DOIs

removing useless counters

using most recent dnet-pace-core features

fixed DedupDeleteRelMapper

do not export deleted entities

adapted to the removal of contributors as relationships

added utility methods to deal with strings rather than byte[]

sort merged ids

log the documents being compared before failing

test for ARC

introducing support for projects that doesn't provide a link to a specific fundingpath.

implemented job and workflow to export the openaire identifiers

log the number of items clustered on each key

do not consider deleted entities

New test for openaire2.0_data compliance for datasets

updating to dnet-openaire-data-protos:3.5.0

updated to dnet-openaire-data-protos:3.5.0-SNAPSHOT

cleanup, extended tests to include new relationships and mapping profiles

counters

counter test

Tests load gthe XSLT from the TDSRule profiles in dnet-openaireplus-profiles

Back to revision r39888 and updated pom and sh files

added possibility to post-process the result stored in the index documents

ticket #1588 Rename "native" compatibility to "proprietary"

use of external properties

added min distance algorithm, used to identify the connected components (dedup)

limit the job to insttitutional pubsrepository

counter labels

use of Text instead of ImmutableBytesWritable

reimplemented calculatePersonDistribution M/R job to consider only the results from pubsrepositories (not journals)

reuse the same outkey and outvalue objects

added more mapping tests, using xslt picked from services.openaire