D-Net

                     <div class="uk-alert-danger" uk-alert>

                        <h3>Contribute to improve the OpenAIRE Research Graph</h3>

                        <p>You can explore and test the beta release of the OpenAIRE Research Graph via the <a href="https://beta.explore.openaire.eu">OpenAIRE BETA Explore Portal</a> or via data dumps made available in <a href="https://zenodo.org/communities/openaire-research-graph">Zenodo</a>. </p>

                        <p>Help us making the graph ready for its 1st production release by providing your feedback.<br/>

                        Go to the <a href="https://trello.com/b/o1tEJ3rN/openaire-research-graph">OpenAIRE Research Graph Trello Board</a> to report content quality issues, including missing metadata records, wrong values, mistakes in the detection of duplicates and anything else that looks "weird" or wrong.

                        <p>Find the complete information about the OpenAIRE Research Graph, how to test it and contribute to improve it on <a href="https://www.openaire.eu/blogs/the-openaire-research-graph">our blog</a>.</p>

                     </div>

                     <h2>OpenAIRE Research Graph Dumps</h2>

                        For the past ten years OpenAIRE has been working to gather this valuable record. OpenAIRE is pleased to announce the beta release of its Research Graph, a massive collection of metadata and links between 

                     <p>The OpenAIRE Research graph is available via our <a href="https://beta.explore.openaire.eu">BETA Explore Portal</a> and you can download it from <a href="https://zenodo.org/communities/openaire-research-graph">Zenodo</a>.

                     </p>

                       <p>The OpenAIRE Research Graph is exported as several dump files available on Zenodo (go to <a href="https://doi.org/10.5281/zenodo.3516917"><img src="https://zenodo.org/badge/DOI/10.5281/zenodo.3516917.svg" alt="DOI"></a>), so you can download the parts you are interested into. </p>

                      <ul>

                         <li> <strong>publications</strong>: metadata records about research literature (includes types of publications listed <a href="http://api.openaire.eu/vocabularies/dnet:result_typologies/publication">here</a>)</li>

                         <li> <strong>datasets:</strong>: metadata records about research data (includes the subtypes listed <a href="http://api.openaire.eu/vocabularies/dnet:result_typologies/dataset">here</a>)</li>

                         <li> <strong>software:</strong>: metadata records about research software (includes the subtypes listed <a href="http://api.openaire.eu/vocabularies/dnet:result_typologies/software">here</a>)</li>

                         <li> <strong>orps</strong>: metadata records about research products that cannot be classified as research literature, data or software (includes types of products listed <a href="http://api.openaire.eu/vocabularies/dnet:result_typologies/other">here</a>)</li>

                         <li> <strong>organizations</strong>: metadata records about organizations involved in the research life-cycle, such as universities, research organizations, funders.</li>

                         <li> <strong>content_providers</strong>: metadata records about providers whose content is available in the OpenAIRE Research Graph. They includes institutional and thematic repositories, journals, aggregators, funders' databases.</li>

                         <li> <strong>results_by_funder</strong>: metadata records about research results funded by a given funder. Each result includes information about its type (publications, datasets, software or other) and its specific sub-type (check the list of sub-types for <a href="http://api.openaire.eu/vocabularies/dnet:result_typologies/publication">publications</a>, <a href="http://api.openaire.eu/vocabularies/dnet:result_typologies/dataset">datasets</a>, <a href="http://api.openaire.eu/vocabularies/dnet:result_typologies/software">software</a>, and <a href="http://api.openaire.eu/vocabularies/dnet:result_typologies/other">other research products</a>).  </li>

                      </ul>

                       <p>The up-to-date list of funders available on OpenAIRE BETA can be find <a href="https://beta.explore.openaire.eu/search/entity-registries?datasourcetypename=%22Funder%20database%22">here on the BETA Explore portal</a>.</p>

                       <p> In the same <a href="https://zenodo.org/communities/openaire-research-graph">Zenodo community</a> you can also find the dumps of ScholeXplorer and DOIBoost.</p>

                        <p>The dumps contain XML records compliant to the <b>OpenAIRE data model</b> and to the <b>oaf metadata format</b> (the same format of the records exported via <a href="./oai-pmh.html">OAI-PMH</a>):</p>

                       <ul>

                          <li><a href="" target="_blank">See the description of the OpenAIRE data model</a></li>

                          <li><a href="https://www.openaire.eu/schema/latest/oaf.xsd" target="_blank">See the oaf XML schema</a></li>

                          <li><a href="https://www.openaire.eu/schema/latest/doc/oaf.html" target="_blank">See the oaf XML schema documentation (generated via Oxygen XML Editor)</a></li>

                       </ul>

                     <p>Keep reading for instructions on how to consume the dumps.</p>

                     <h3>Consume the dumps</h3>

                     <div>

                        Each dump is a gzipped json file with many lines. Each line is in the form of:

                        <code>{"_id":{"$oid":"59b82504895be144859a9804"},"body":{"$binary":"base64(zip(XML_record))","$type":"00"}}</code><br/>

                        where the <code>body</code> field contains the base64 econding of the compressed XML record. <br/>

                        In order to get the XMLs you have to:

                        <ol>

                           <li>Unzip the file</li>

                           <li>Get only the value of the <code>$binary</code> field</li>

                           <li>Read each line and base64 decode it</li>

                           <li>Unzip the decoded string</li>

                        </ol>

                        For example, to print the XMLs on the standard output you can run this command on MacOS/Unix/Linux based systems:

                        <code>gunzip -c file.json.gz | jq '.body."$binary"' -r | while IFS= read -r line; do echo "$line" | base64 --decode | bsdtar -x -O ; done </code><br/>

                        where 

                        <ul>

                           <li><code>file.json.gz</code> is the name you gave to the downloaded file dump;</li>

                           <li><code>jq</code> is a command to parse json files. It is not installed by default, but you can easy find it on official repositories. <a href="https://stedolan.github.io/jq/download/">Click here for installation instructions</a>.

                           <li><code>base64</code> and <code>bsdtar</code> are two libraries that are typically pre-installed.</li>

                        </ul>

                       Note that you should decide what to do with it (keep parsing XML inline or store them somewhere).

                       We suggest to start with few records to test and decide what to do, by adding a <code>head</code> command after the <code>gunzip</code>, like:

                     <code>gunzip -c file.json.gz | head -n 10 | jq '.body."$binary"' -r | while IFS= read -r line; do echo "$line" | base64 --decode | bsdtar -x -O ; done</code>

                     </div>

                     <h3>Cite us</h3>

                     <p>If you use the OpenAIRE Research Graph for research purposes, please cite it as:<br/>

                        <i>Manghi, Paolo, Atzori, Claudio, Bardi, Alessia, Shirrwagen, Jochen, Dimitropoulos, Harry, La Bruzzo, Sandro, … Summan, Friedrich. (2019). OpenAIRE Research Graph Dump [Data set]. Zenodo. http://doi.org/10.5281/zenodo.3516917</i><br/>

                        If you want to cite a specific version, please follow the suggestion on Zenodo. For the current version (1.0.0-beta), please use: </br>

                        <i>Manghi, Paolo, Atzori, Claudio, Bardi, Alessia, Shirrwagen, Jochen, Dimitropoulos, Harry, La Bruzzo, Sandro, … Summan, Friedrich. (2019). OpenAIRE Research Graph Dump (Version 1.0.0-beta) [Data set]. Zenodo. http://doi.org/10.5281/zenodo.3516918</i><br/>

                        The OpenAIRE Research graph includes data from <a href="https://aka.ms/msracad">Microsoft Academic Graph</a> (MAG): please acknowledge also MAG following <a href="https://docs.microsoft.com/en-us/academic-services/graph/resources-faq#license">this guideline</a>.

                     </p>

                     <h3>License</h3>

                     <p>The OpenAIRE Research Graph is released under CC-BY license.</p>

                     <p>OpenAIRE is working to produce dumps that only contains metadata records that can be re-distributed with the CC0 license: stay tuned!</p>