D-Net

For incremental harvesting: return the date with date granularity level.

Test the logics of mdbuilder -- just in case

Added snapshot repository from FORTH, just in case

more testing for Ariadne and 3M mappings

removed unused test file

removed mapping from profiles section, mapping is taken only from url; x3m transformation node moved to dnet-ariadneplus module

aligned to aggregator instance

x3ml-engine version reset to 1.9.3

testing usage of 1.9.2-SNAPSHOT for troubleshooting issue with ariadne mapping

align svn folder info

mapping url downloaded from remote url set in workflow parameters

fixed dependencies and deploy for solr772

branch for solr772

x3ml-engine 1.9.1 does not help, so let's go back to 1.9.3

trying with x3ml-engine v1.9.1

trying with x3ml-engine v1.9.1

using x3ml-engine version 1.9.3

[maven-release-plugin] prepare for next development iteration

[maven-release-plugin] prepare release dnet-msro-service-6.0.1-SAXONHE

Updated URL for FORTH repository to avoid the "peer not authenticated" error when compiling

Default source should be the api, otherwise the delete of an api of one ds will trigger the deletion of all records for the data source in the OAI-PMH

Sticking to release version 3.9.0: using synchronised is not helpful, maybe the 1.9.0 release helps

trying to use x3ml-engine inside a synchronised method as suggested in https://support.d4science.org/issues/12919#change-77131

Log input file that cannot be transformed

Parthenos mappings now require new x3ml-engine version and field <thesaurus_info/> is now mandatory.

Updated default mail template with start and end dates in human readable version

Human readable start and end date

BBActorCallback now can pass parameters back to the workflow for logging purposes.

fixed compilation error

merged r52660: email dispatching for wf template based on parent workflow notification configuration

fixed dependency to x3ml-engine

Testing URI shortener

x3ml-engine upgraded to spring4: removed exclusion and enlarged the dependency version range

ensure we do not get 1.8.5-SNAPSHOT

Opened range for x3ml-engine dependency: up to 1.9.0 excluded

Have the x3ml engine generate real UUIDs instead of test values.

cleanup

MERGE: branch updated with trunk changes through r50565

Trying to exclude slf4j deps from x3ml to see if it helps in properly logging.

We can use x3ml version 1.8.2 but not later version (compilation problems due to some spring conflicts, apparently)

Job nodes related to Parthenos now extends AsyncJobNode

[maven-release-plugin] prepare for next development iteration

[maven-release-plugin] prepare release dnet-msro-service-6.0.0-SAXONHE

removed ennoying log

logs

Do not log the content of mapping files

Tests moved from msro to dnet-parthenos, where they should have belonged in the first place

New parameter to decide to pass the full record or only the metadata content to x3m

Using RDF_XML_PLAIN serialization.

Updated mapping for culturaItalia and ignored its test until they fix the mapping.

Add dri:datasourceapi in the header

Updated Clarin 365 mapping for tests

Updated tests and mappings

Fixed logs

More tests fro x3m

Ignore CLARIn test that fails for duplicate namespace prefix

Testing #8664#note-5 for EHRI and namespace

More tests for EHRI and CulturaItalia

Testing Parthenos 3M mappings

Logging

Returning more detailed counters for mdstore feeding.
Fixed bug when trying to store too large XML: now all the page of XMLs is skipped, so we can go on with the other records (MongoBulkWritesManager)

Logging the extracted metadata for debugging

Added tests with real mappings

X3M Transformation node must be able to handle multiple 3M mappings

Handling workflow parameters of type 'list'

New type of workflow parameter that enables multiple selection of values from a list obtained via a function: "list"

Now we can tune the X3ML engine logs: https://support.d4science.org/issues/8117

Moved class in more proper package

Interpretation is not a solr-only property

Handle date range requests

New jobnode to get the last start date of a workflow of the same type of the current. The old job node is still needed to get the date as formatted string

Just one job node for refresh/incremental harvesting

UpdateIndex must consider also the interpretation, not only format and layout

Execution time monitored in debug mode on the jobNode, as the preparation for the mapping application does not impact on the total transformation time (from 0 to 3 ms, while transformation takes 100-200 ms)

logs for monitoring transofrmation execution time

Cleanup

calling the right deliver methods based on the noCursorTimeout param

Using new method to disable cursor timeout when reading from mdstore

Test

checking how long it takes to transform

I do not want to see dri:repositoryIdentifier anymore

removed some DEBUG log

DEBUGGING THE TIMEOUT OF THE MONGO CURSOR: is there a metadata record that it is really slow to transform?

Testing x3ml

fixed xpath for the selection of the metadata body to validate

FORTH said they fixed the mvn repo problem, let's try ona jenkins build!

do not depend on FORTH mvn repo until https://support.d4science.org/issues/8021 is fixed

Need to validate only the content of oai:metadata

deploy.info for SaxonHE branches

changed dependency so that it is clear those are modules using SAXON HE

an instance of default job node for no-op

dateOfTransformation under the dri namespace

fixed call to current-datetime()

LayoutToRecordStylesheet xslt must be passed as property

MDBuilder passes to record template also the datasource name to fill dri:datasourcename

fixed typo on log

Node that validates XML record against their schema or a specific schema given its URL

Added dsName parameter

Created UnaryFunction for X3ML

Building correct D-Net XMLs after transformation with X3M