|
1
|
<section class="section greySection" id="sect-overview">
|
|
2
|
|
|
3
|
<div class="uk-container uk-container-expand uk-container-center uk-scrollspy-init-inview uk-scrollspy-inview uk-animation-scale-up" data-uk-scrollspy="{cls:'uk-animation-scale-up uk-invisible',delay:300,topoffset:-100}">
|
|
4
|
|
|
5
|
<!--Introduction-->
|
|
6
|
<div class="">
|
|
7
|
<h2>Introduction</h2>
|
|
8
|
|
|
9
|
<div class="md-card uk-margin-bottom">
|
|
10
|
<div class="md-card-content large-padding">
|
|
11
|
<p>
|
|
12
|
This document describes the methodology , terms, and definitions of the indicators presented in Open Science Observatory.
|
|
13
|
The Open Science Observatory combines data gathered from OpenAIRE and other data sources to develop and operate an
|
|
14
|
interactive and dynamic portal, which informs users via rich visualizations and reports on different Open Science
|
|
15
|
aspects and facets in Europe. The Observatory is an extendable monitor portal with indicators on open science and
|
|
16
|
the framework uptake by national and sectoral set.
|
|
17
|
</p>
|
|
18
|
<p>
|
|
19
|
The dynamic <strong>Open Science Observatory</strong> leverages existing information collected by OpenAIRE and
|
|
20
|
<strong>at little additional cost or effort, produces and reports value added metrics</strong> regarding Open Science
|
|
21
|
trends and evolution for Horizon 2020 and other funding sources. The goal is to provide an EU Open Science Observatory
|
|
22
|
as key OpenAIRE product, and employing open metrics based on open data gathered by OpenAire to measure the openness
|
|
23
|
of publications and data on various aspects (e.g., gold/green/fair), the metadata completeness, the regional or
|
|
24
|
thematic distributions and will test out preliminary metrics for their FAIRness. This will ensure that research can
|
|
25
|
be evaluated at the right level (including article-level metrics and altmetrics), in an <strong>open and transparent
|
|
26
|
manner</strong>. Such services will deliver accurate metrics for European Research and enable informed recommendations
|
|
27
|
to high-level decision makers.
|
|
28
|
</p>
|
|
29
|
</div>
|
|
30
|
</div>
|
|
31
|
|
|
32
|
</div>
|
|
33
|
|
|
34
|
<!--Methodological Approach and Indicators-->
|
|
35
|
<div class="">
|
|
36
|
<h2>Methodological Approach and Indicators</h2>
|
|
37
|
|
|
38
|
<div class="md-card uk-margin-bottom">
|
|
39
|
<div class="md-card-content large-padding">
|
|
40
|
<p>
|
|
41
|
Monitoring and evaluating the advancements, trends and impact of OS in Europe is recognised as one of the most
|
|
42
|
important steps towards the realisation of the EOSC vision. The implementation of the Open Science Observatory
|
|
43
|
follows the guidelines and <a target="_blank" href="https://eoscpilot.eu/wp3-policy/eosc-open-science-monitor-specifications/methodological-approach-monitoring-open-science">
|
|
44
|
methodological approach</a> which was specified by the EOSCPilot project and more specifically the
|
|
45
|
<a target="_blank" href="https://eoscpilot.eu/content/d32-eosc-open-science-monitor-specifications">EOSCpilot Open Science Monitor Framework (EOSCpilot OSMF)</a>.
|
|
46
|
It aimed to build a model and initial high-level specifications for providing useful analytics to researchers as well
|
|
47
|
as enabling research performing and funding organizations within EOSC to monitor and gain insights about the OS movement,
|
|
48
|
regardless of their service management systems and the technology behind them. The six core steps of the specified methodology are:
|
|
49
|
</p>
|
|
50
|
<ul style="font-style: italic">
|
|
51
|
<li>Step 1. Identification of the Open Science Activities;</li>
|
|
52
|
<li>Step 2. Policy-driven derivation of monitoring targets;</li>
|
|
53
|
<li>Step 3. Identification of the main Open Science Resources and Indicators;</li>
|
|
54
|
<li>Step 4. Design of monitoring processes, tasks and workflows;</li>
|
|
55
|
<li>Step 5. Modelling and implementation of the framework;</li>
|
|
56
|
<li>Step 6. Continuous validation of the monitoring targets;</li>
|
|
57
|
</ul>
|
|
58
|
|
|
59
|
<p>
|
|
60
|
<span style="font-style: italic">Step 1. Identification of the Open Science Activities:</span> The first step addresses the need to identify
|
|
61
|
which parts of the OS lifecycle are of interest in the monitoring process. These can include the conceptualization
|
|
62
|
of a research task, the data and literature collection, the analysis and development of the research output, the
|
|
63
|
publication, the review and evaluation of the research result as well as the reuse and reproducibility of results
|
|
64
|
by the scientific community. These phases entail different open access practices and elements which are being considered
|
|
65
|
by the monitor, with a special focus on the policies that these elements fall under.
|
|
66
|
</p>
|
|
67
|
<p>
|
|
68
|
<span style="font-style: italic">Step 2. Policy-driven derivation of monitoring targets:</span> The proposed monitoring
|
|
69
|
framework adopts a policy-driven approach for deriving high level objectives, i.e., target dimensions to be measured
|
|
70
|
in the monitoring process. Policies on Open Access at different levels, such as the international, national and
|
|
71
|
regional levels, as well as micro policies are considered as primary sources for deriving more concrete measurable
|
|
72
|
targets (e.g., Openness, FAIRness, etc.) that should be monitored in the framework. Furthermore, monitoring targets
|
|
73
|
can be organised into more specialised sub-targets, to measure more specific aspects of OS. For example, a policy
|
|
74
|
recommendation stating that research data repositories should follow a data archiving plan indicates the monitoring
|
|
75
|
target for long-term preservation of OS artefacts monitored by a set of indicators, such as whether an organization
|
|
76
|
applies such a plan, or the period (e.g., months, years) for which preservation is guaranteed.
|
|
77
|
</p>
|
|
78
|
<p>
|
|
79
|
<span style="font-style: italic">Step 3. Identification of the main Open Science Resources and Indicators:</span>
|
|
80
|
In the next step, the monitoring targets are being mapped to OS elements they apply to, as well as to indicators
|
|
81
|
that quantify these targets. OS elements are well-defined artefacts of OS practices, such as publication in open
|
|
82
|
access journals, research data made available in open access repositories, open source software, open educational
|
|
83
|
material, etc. In the context of this framework, these elements are called OS resources to state the importance
|
|
84
|
of their contribution for the development of the “Open Science World”.
|
|
85
|
</p>
|
|
86
|
<p>
|
|
87
|
<span style="font-style: italic">Step 4. Design of monitoring processes, tasks and workflows:</span> Each indicator
|
|
88
|
must be associated with a set of processes, which are employed for the collection of data, the validation and
|
|
89
|
scoring of metrics (e.g., combination and aggregation of metrics for deriving an accumulated score for a target
|
|
90
|
dimension), the visualization of the results, and so on. These processes must be well documented in the form of
|
|
91
|
workflows and tasks, to be performed for the collection and quantification of the indicators.
|
|
92
|
</p>
|
|
93
|
<p>
|
|
94
|
<span style="font-style: italic">Step 5. Modelling and implementation of the framework:</span> The next step
|
|
95
|
involves the detailed design, implementation, and customization of the framework, which includes the design of
|
|
96
|
the functionality at its whole, as well as all added value services offered by the framework.
|
|
97
|
</p>
|
|
98
|
<p>
|
|
99
|
<span style="font-style: italic">Step 6. Continuous validation of the monitoring targets:</span> The last step
|
|
100
|
follows the operation of the OS monitoring framework and refers to the continuous validation and refinement of
|
|
101
|
the monitoring methodology (i.e., targets and indicators) and results in EOSC. Α monitoring process, to be effective,
|
|
102
|
must adapt to new OS practices and new policies, validating and readjusting its target goals as well as the indicators
|
|
103
|
for their evaluation.
|
|
104
|
</p>
|
|
105
|
|
|
106
|
<p>
|
|
107
|
<strong>The Open Science Observatory of the OpenAIRE is the first implementation</strong> of these specifications
|
|
108
|
and it aspires to become a dynamic tool in the future, with benefits both to the organisations using it to measure
|
|
109
|
the OA levels of implementation and impact to their community and for the High Level Stakeholders which are the
|
|
110
|
European Union’s Legal Entities and Bodies, including the Member States and their respective Units. Gaps, implications
|
|
111
|
and new ways of performing OS are among the elements that could be identified through OA Open Science Observatory
|
|
112
|
which could then be easily incorporated within the stakeholders’ scope and strategic planning for OS.
|
|
113
|
</p>
|
|
114
|
</div>
|
|
115
|
</div>
|
|
116
|
|
|
117
|
</div>
|
|
118
|
|
|
119
|
<!--Indicators-->
|
|
120
|
<div class="">
|
|
121
|
<h2>Indicators</h2>
|
|
122
|
|
|
123
|
<div class="md-card uk-margin-bottom">
|
|
124
|
<div class="md-card-content large-padding">
|
|
125
|
|
|
126
|
<div class="">
|
|
127
|
<ul class="uk-tab uk-tab-large uk-flex-center" data-uk-tab="{connect:'#tabs',animation: 'slide-bottom'}">
|
|
128
|
<li class="uk-active" aria-expanded="true"><a href="#">Terms and definitions</a></li>
|
|
129
|
<li aria-expanded="false"><a href="#">Continent overview</a></li>
|
|
130
|
<li aria-expanded="false"><a href="#">COuntry page</a></li>
|
|
131
|
</ul>
|
|
132
|
|
|
133
|
<ul id="tabs" class="uk-switcher">
|
|
134
|
|
|
135
|
<!--OVERVIEW tab-->
|
|
136
|
<li aria-hidden="false" class="uk-active" style="animation-duration: 200ms;">
|
|
137
|
<div class="uk-margin">
|
|
138
|
|
|
139
|
<!--Terms and definitions-->
|
|
140
|
<div>
|
|
141
|
<!--<h3>Terms and definitions</h3>-->
|
|
142
|
<dl>
|
|
143
|
<dt>Publication</dt>
|
|
144
|
<dd>a scientific publication</dd>
|
|
145
|
|
|
146
|
<dt>Repository</dt>
|
|
147
|
<dd>a repository of publications</dd>
|
|
148
|
|
|
149
|
<dt>Dataset</dt>
|
|
150
|
<dd>a dataset, usually associated with a publication</dd>
|
|
151
|
|
|
152
|
<dt>Software</dt>
|
|
153
|
<dd>software (e.g., open source), usually associated with a publication/dataset</dd>
|
|
154
|
|
|
155
|
<dt>Other Research Products</dt>
|
|
156
|
<dd>Other research outputs (e.g., dissemination or educational material) associated with a publication, dataset or software.</dd>
|
|
157
|
|
|
158
|
<dt>Journal</dt>
|
|
159
|
<dd>a scientific journal where authors publish their results in the form of publications</dd>
|
|
160
|
|
|
161
|
<dt>Policy</dt>
|
|
162
|
<dd>an Open Science mandate that organizations follow in order to engage in Open Science best practices</dd>
|
|
163
|
|
|
164
|
<dt>Organization</dt>
|
|
165
|
<dd>an academic institution, company, or any other legal entity with the context of OpenAIRE</dd>
|
|
166
|
|
|
167
|
<dt>Vocabularies</dt>
|
|
168
|
<dd>used in the Observatory are described in <a target="_blank" href="http://api.openaire.eu/vocabularies">http://api.openaire.eu/vocabularies</a></dd>
|
|
169
|
</dl>
|
|
170
|
</div>
|
|
171
|
|
|
172
|
</div>
|
|
173
|
</li>
|
|
174
|
|
|
175
|
<!--OPEN SCIENCE tab-->
|
|
176
|
<li aria-hidden="true" style="animation-duration: 200ms;">
|
|
177
|
<div class="uk-margin">
|
|
178
|
|
|
179
|
<!--Continent overview-->
|
|
180
|
<div>
|
|
181
|
<!--<h3>Continent overview</h3>-->
|
|
182
|
<dl>
|
|
183
|
<dt>Number of OA publications</dt>
|
|
184
|
<dd>The total number of open access publications currently in OpenAIRE, along with the percentage they represent with respect to the total number of publications.</dd>
|
|
185
|
|
|
186
|
<dt>Number of OA datasets</dt>
|
|
187
|
<dd>The total number of open access datasets currently in OpenAIRE, along with the percentage they represent with respect to the total number of datasets.</dd>
|
|
188
|
|
|
189
|
<dt>Number of OA repositories</dt>
|
|
190
|
<dd>The total number of open access repositories currently in OpenAIRE.</dd>
|
|
191
|
|
|
192
|
<dt>Number of OA journals</dt>
|
|
193
|
<dd>The total number of open access journals currently in OpenAIRE, along with the percentage they represent with respect to the total number of journals. Source: DOAJ.</dd>
|
|
194
|
|
|
195
|
<dt>Number of organizations with OA policies</dt>
|
|
196
|
<dd>The total number of organizations in OpenAIRE that have Open Access policies associated with them. Source: RoarMap.</dd>
|
|
197
|
</dl>
|
|
198
|
|
|
199
|
<!--Overview-->
|
|
200
|
<div>
|
|
201
|
<h4>Overview</h4>
|
|
202
|
<p>Each table row refers to a country. For each country, the table shows <span style="font-style: italic">total numbers of OA repositories,
|
|
203
|
journals, policies, publications, datasets, software and other research products.</span><br>
|
|
204
|
The total numbers and percentages are computed as described in the continent overview.
|
|
205
|
</p>
|
|
206
|
</div>
|
|
207
|
|
|
208
|
<!--Open Science-->
|
|
209
|
<div>
|
|
210
|
<h4>Open Science</h4>
|
|
211
|
<p>Open science related resources, specifically OA publications, datasets, software and other research products.</p>
|
|
212
|
</div>
|
|
213
|
|
|
214
|
</div>
|
|
215
|
|
|
216
|
</div>
|
|
217
|
</li>
|
|
218
|
|
|
219
|
<li aria-hidden="true" style="animation-duration: 200ms;">
|
|
220
|
<div class="uk-margin">
|
|
221
|
|
|
222
|
<!--Country page-->
|
|
223
|
<div>
|
|
224
|
<!--<h3>Country page</h3>-->
|
|
225
|
|
|
226
|
<!--Country overview-->
|
|
227
|
<div>
|
|
228
|
<h4>Country overview</h4>
|
|
229
|
<dl>
|
|
230
|
<dt>Number of OA publications</dt>
|
|
231
|
<dd>The total number of open access publications currently in OpenAIRE for this country, along with the
|
|
232
|
percentage they represent with respect to the total number of publications of this country.</dd>
|
|
233
|
|
|
234
|
<dt>Number of OA datasets</dt>
|
|
235
|
<dd>The total number of open access datasets currently in OpenAIRE for this country, along with the percentage
|
|
236
|
they represent with respect to the total number of datasets of this country.
|
|
237
|
</dd>
|
|
238
|
|
|
239
|
<dt>Number of OA repositories</dt>
|
|
240
|
<dd>The total number of open access repositories currently in OpenAIRE for this country.</dd>
|
|
241
|
|
|
242
|
<dt>Number of OA journals</dt>
|
|
243
|
<dd>The total number of open access journals currently in OpenAIRE for this country, along with the percentage
|
|
244
|
they represent with respect to the total number of journals. Source: DOAJ.</dd>
|
|
245
|
|
|
246
|
<dt>Number of organizations with OA policies</dt>
|
|
247
|
<dd>The total number of organizations in OpenAIRE for this country that have Open Access policies associated
|
|
248
|
with them. Source: RoarMap.</dd>
|
|
249
|
</dl>
|
|
250
|
</div>
|
|
251
|
|
|
252
|
<!--Info Box-->
|
|
253
|
<div>
|
|
254
|
<h4>Info Box</h4>
|
|
255
|
<dl>
|
|
256
|
<dt>R&D Expenditure</dt>
|
|
257
|
<dd>The total R&D expenditure for this country since 2008. Source: Eurostat.</dd>
|
|
258
|
|
|
259
|
<dt>Funding sources</dt>
|
|
260
|
<dd></dd>
|
|
261
|
|
|
262
|
<dt>Funding Organizations</dt>
|
|
263
|
<dd></dd>
|
|
264
|
|
|
265
|
<dt>Organizations funded by the European Commission since 2013</dt>
|
|
266
|
<dd>The total number of organizations in this country that have participated/participate in a project that was/is
|
|
267
|
funded by the European Commission.</dd>
|
|
268
|
</dl>
|
|
269
|
</div>
|
|
270
|
|
|
271
|
<!--Green vs. Gold-->
|
|
272
|
<div>
|
|
273
|
<h4>Green vs. Gold</h4>
|
|
274
|
<dl>
|
|
275
|
<dt>Green vs Gold Publications</dt>
|
|
276
|
<dd>The line shows the evolution of the total number of publications that have been published through a green
|
|
277
|
open access route, vs the total number of publications that have been published through a gold open access route.</dd>
|
|
278
|
|
|
279
|
<dt>Gold Open Access</dt>
|
|
280
|
<dd>The chart shows a ranking of the top 15 organizations of this country in descending order of their total
|
|
281
|
count of gold open access publications.</dd>
|
|
282
|
|
|
283
|
<dt>Green Open Access</dt>
|
|
284
|
<dd>The chart shows a ranking of the top 15 organizations of this country in descending order of their total
|
|
285
|
count of green open access publications. </dd>
|
|
286
|
</dl>
|
|
287
|
</div>
|
|
288
|
|
|
289
|
<!--EU Funded Open Science-->
|
|
290
|
<div>
|
|
291
|
<h4>EU Funded Open Science</h4>
|
|
292
|
<dl>
|
|
293
|
<dt>Publications vs Datasets vs Software</dt>
|
|
294
|
<dd>The chart shows a per-year count and comparison of open access publications, datasets and software for this country.</dd>
|
|
295
|
|
|
296
|
<dt>Organizations</dt>
|
|
297
|
<dd>This table shows all organizations from this country, along with the total number of publications, datasets and
|
|
298
|
software that have authors that are affiliated with each organization. The rows can be ordered by any of the three numbers.</dd>
|
|
299
|
|
|
300
|
<dt>Repositories</dt>
|
|
301
|
<dd>This table shows all repositories from this country, along with the total number of publications, datasets
|
|
302
|
and software that reside in each repository. The rows can be ordered by any of the three numbers.</dd>
|
|
303
|
</dl>
|
|
304
|
</div>
|
|
305
|
|
|
306
|
<!--Funding Sources-->
|
|
307
|
<div>
|
|
308
|
<h4>Funding Sources</h4>
|
|
309
|
<dl>
|
|
310
|
<dt>Publications vs Datasets vs Software</dt>
|
|
311
|
<dd>This chart shows colored horizontal bars representing total number of publications, datasets and software
|
|
312
|
for each funder organization associated with this country.</dd>
|
|
313
|
</dl>
|
|
314
|
</div>
|
|
315
|
|
|
316
|
<!--Project Performance-->
|
|
317
|
<div>
|
|
318
|
<h4>Project Performance</h4>
|
|
319
|
<dl>
|
|
320
|
<dt>Top 10 Projects by publications / datasets / software</dt>
|
|
321
|
<dd>A ranking of the top 10 projects in descending order of their associated publication/dataset/software count.
|
|
322
|
Publication/dataset/software count is computed as the total number of publications/datasets/software that have
|
|
323
|
been funded by a project.</dd>
|
|
324
|
</dl>
|
|
325
|
</div>
|
|
326
|
|
|
327
|
|
|
328
|
</div>
|
|
329
|
|
|
330
|
</div>
|
|
331
|
</li>
|
|
332
|
</ul>
|
|
333
|
</div>
|
|
334
|
|
|
335
|
|
|
336
|
</div>
|
|
337
|
</div>
|
|
338
|
|
|
339
|
</div>
|
|
340
|
|
|
341
|
</div>
|
|
342
|
</section>
|