D6.4 OpenAIRE Maintenance Report (v1, 9th of December 2016)¶

• Table of contents
• D6.4 OpenAIRE Maintenance Report (v1, 9th of December 2016)
  • Overview
  • Deployment
  • Documentation
  • D-NET workflows
  • Services status
    • D-NET backend services
    • Portal
    • Zenodo
    • DM hadoop cluster
    • IIS hadoop cluster
    • Content status
    • Change Log for servers administration operations
  • Major modifications
    • Introducing CDH5 IIS cluster hosted in OCEAN infrastructure
    • Incorporating resources from old CDH4 IIS cluster into existing DM CDH4 cluster
  • System downtimes

Overview¶

This document contains information about the deployment, status of OpenAIRE2020 services and content, and the history of major modifications of the system hosted at ICM, Poland.

The official maintenance of the OpenAIRE2020 services began on January 1st, 2015 when the project started.

Deployment¶

Because OpenAIRE2020 services are a continuation and incremental extension of the services already present that resulted from OpenAIRE+ project, so they are still hosted on that same machines. More details are available at OpenAIRE+ WP5 Maintenance Report.

@Paweł: are there any new services in maintenance since OpenAIRE+ project worth mentioning here?

Documentation¶

The D-Net services are shipped as web applications and deployed on the tomcat application server (v7.0.52) on three distinct systems: dev, beta, production. To facilitate the deployment process all the software artifacts are automatically built on a continuous integration system (Jenkins) and hosted on a dedicated maven repository (nexus), while webapp builds are made available via http server.

• The development infrastructure plays the role of test bench for the integration of the software developed by different institutions. It runs mostly un-released code and contains mock or subsets of the data available on the production system.
• The beta infrastructure runs only released code. Its consists of the final integration stage where all the system workflows are tested on the real data (not necessarily the same data as the production system) before making them available to the production system. Although the software running on the beta system is not yet production ready, its portal is publicly accessible in order to showcase new features and data.
• The production infrastructure runs only code that was tested on the beta system.

DNet deployment is performed following the documentation described at: http://ivy.driver.research-infrastructures.eu/ci/dnet-openaire-doc/

@Claudio: is it up to date?

Zenodo repository is deployed according to the process described at http://invenio-software.org/wiki/Project/OpenAIREplus/ReleaseProcess.

@Lars: link does not work anymore, can you provide the proper one?

Information Inference Service versioning and deployment is described on IIS versioning and deployment wiki page.

D-NET workflows¶

@Claudio: could you fill that in?

Services status¶

D-NET backend services¶

@Claudio: could you fill that in? Probably you can rely on http://wiki.openaire.eu/xwiki/bin/view/OpenAIREplus%20Specific/WP5%20Maintenance%20Report

Portal¶

@Paweł: could you fill that in? You can do it in a similar way it was done in http://wiki.openaire.eu/xwiki/bin/view/Technical%20Specifications/Maintenance%20Report

Zenodo¶

Zenodo repository employs an instance of the Invenio software, developed by CERN.

@Lars: could you fill that in? You can do it in a similar way it was done in http://wiki.openaire.eu/xwiki/bin/view/Technical%20Specifications/Maintenance%20Report

DM hadoop cluster¶

CDH version: cdh4.3.1

IIS hadoop cluster¶

CDH version: cdh4.3.1, replaced with cdh5.5.2 on November 20, 2016, to be replaced with cdh5.9.0

Content status¶

data type	count
publications	17460368
datasets	3226586
projects	653268
organizations	64591
authors	16188328
EuropePMC XML contents	1574358
PDF contents	2227458

Based on IIS report generated on November 20, 2016.

Change Log for servers administration operations¶

@Paweł: could you fill that in? This could be similar to: http://wiki.openaire.eu/xwiki/bin/view/OpenAIREplus%20Specific/Change%20Log%20for%20servers%20administration%20operations

Major modifications¶

Introducing CDH5 IIS cluster hosted in OCEAN infrastructure¶

Slave node specification:

• Huawei RH1288 V3
• CPU: 2x Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz (24 cores, 48 threads)
• RAM: 128GB
• HDD: 4x SATA 6TB 7.2K RPM (HDFS)

Cluster summary (16 slaves):

• CPU: 384 cores, 768 threads
• RAM: 2048GB
• HDD: 384TB (HDFS)

YARN available resources:

• vcores: 640
• memory: 1.44TB
• HDFS: 344TB

Incorporating resources from old CDH4 IIS cluster into existing DM CDH4 cluster¶

@Paweł: are you aware of any other major modifications regarding other services such as new portal version etc?

System downtimes¶

@Paweł: could you fill that in? We could create dedicated with entry similar to:
http://wiki.openaire.eu/xwiki/bin/view/OpenAIREplus%20Specific/History%20of%20system%20outages
and link it here.