Using Jupyter for reproducible scientific workflows

Marijan Beg; Juliette Belin; Thomas Kluyver; Alexander Konovalov; Min Ragan-Kelley; Nicolas Thiery; Hans Fangohr

doi:10.1109/MCSE.2021.3052101

Using Jupyter for reproducible scientific workflows

Marijan Beg, Juliette Belin, Thomas Kluyver, Alexander Konovalov^*, Min Ragan-Kelley, Nicolas Thiery, Hans Fangohr

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

Abstract

Literate computing has emerged as an important tool for computational studies and open science, with growing folklore of best practices. In this work, we report two case studies - one in computational magnetism and another in computational mathematics - where a dedicated software was exposed into the Jupyter environment. This enabled interactive and batch computational exploration of data, simulations, data analysis, and workflow documentation and outcome in Jupyter notebooks. In the first study, Ubermag drives existing computational micromagnetics software through a domain-specific language embedded in Python. In the second study, a dedicated Jupyter kernel interfaces with the GAP system for computational discrete algebra and its dedicated programming language. In light of these case studies, we discuss the benefits of this approach, including progress towards more reproducible and re-usable research results and outputs, notably through the use of infrastructure such as JupyterHub and Binder.

Original language	English
Article number	9325550
Pages (from-to)	36-46
Number of pages	11
Journal	Computing in Science and Engineering
Volume	23
Issue number	2
Early online date	15 Jan 2021
DOIs	https://doi.org/10.1109/MCSE.2021.3052101
Publication status	Published - Mar 2021

Keywords

Jupyter

Access to Document

10.1109/MCSE.2021.3052101

Beg_2021_CSE_using-Jupyter
Copyright © IEEE 2020. This work has been made available online in accordance with publisher policies or with permission. Permission for further reuse of this content should be sought from the publisher or the rights holder. This is the author created accepted manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1109/MCSE.2021.3052101.
Accepted author manuscript, 3.51 MB

H2020 OPENDREAMKIT: OPENDREAMKIT (partner)
Linton, S. A. (PI) & Konovalov, O. (CoI)
European Commission
1/09/15 → 31/08/19
Project: Standard

Using Jupyter for reproducible scientific workflows
Beg, M. (Creator), Taka, J. (Creator), Kluyver, T. (Creator), Konovalov, A. (Creator), Ragan-Kelley, M. (Creator), Thiéry, N. (Creator) & Fangohr, H. (Creator), Zenodo, 2020
DOI: 10.5281/zenodo.4382225
Dataset: Software

Cite this

@article{68fa4ba8e35c4f62a1fa8db302f3661b,

title = "Using Jupyter for reproducible scientific workflows",

abstract = "Literate computing has emerged as an important tool for computational studies and open science, with growing folklore of best practices. In this work, we report two case studies - one in computational magnetism and another in computational mathematics - where a dedicated software was exposed into the Jupyter environment. This enabled interactive and batch computational exploration of data, simulations, data analysis, and workflow documentation and outcome in Jupyter notebooks. In the first study, Ubermag drives existing computational micromagnetics software through a domain-specific language embedded in Python. In the second study, a dedicated Jupyter kernel interfaces with the GAP system for computational discrete algebra and its dedicated programming language. In light of these case studies, we discuss the benefits of this approach, including progress towards more reproducible and re-usable research results and outputs, notably through the use of infrastructure such as JupyterHub and Binder.",

keywords = "Jupyter",

author = "Marijan Beg and Juliette Belin and Thomas Kluyver and Alexander Konovalov and Min Ragan-Kelley and Nicolas Thiery and Hans Fangohr",

note = "Funding: This work was financially supported by the OpenDreamKit Horizon 2020 European Research Infrastructure project (676541) and the EPSRC Programme grant on Skyrmionics (EP/N032128/1).",

year = "2021",

month = mar,

doi = "10.1109/MCSE.2021.3052101",

language = "English",

volume = "23",

pages = "36--46",

journal = "Computing in Science and Engineering",

issn = "1521-9615",

publisher = "IEEE Computer Society",

number = "2",

}

TY - JOUR

T1 - Using Jupyter for reproducible scientific workflows

AU - Beg, Marijan

AU - Belin, Juliette

AU - Kluyver, Thomas

AU - Konovalov, Alexander

AU - Ragan-Kelley, Min

AU - Thiery, Nicolas

AU - Fangohr, Hans

N1 - Funding: This work was financially supported by the OpenDreamKit Horizon 2020 European Research Infrastructure project (676541) and the EPSRC Programme grant on Skyrmionics (EP/N032128/1).

PY - 2021/3

Y1 - 2021/3

N2 - Literate computing has emerged as an important tool for computational studies and open science, with growing folklore of best practices. In this work, we report two case studies - one in computational magnetism and another in computational mathematics - where a dedicated software was exposed into the Jupyter environment. This enabled interactive and batch computational exploration of data, simulations, data analysis, and workflow documentation and outcome in Jupyter notebooks. In the first study, Ubermag drives existing computational micromagnetics software through a domain-specific language embedded in Python. In the second study, a dedicated Jupyter kernel interfaces with the GAP system for computational discrete algebra and its dedicated programming language. In light of these case studies, we discuss the benefits of this approach, including progress towards more reproducible and re-usable research results and outputs, notably through the use of infrastructure such as JupyterHub and Binder.

AB - Literate computing has emerged as an important tool for computational studies and open science, with growing folklore of best practices. In this work, we report two case studies - one in computational magnetism and another in computational mathematics - where a dedicated software was exposed into the Jupyter environment. This enabled interactive and batch computational exploration of data, simulations, data analysis, and workflow documentation and outcome in Jupyter notebooks. In the first study, Ubermag drives existing computational micromagnetics software through a domain-specific language embedded in Python. In the second study, a dedicated Jupyter kernel interfaces with the GAP system for computational discrete algebra and its dedicated programming language. In light of these case studies, we discuss the benefits of this approach, including progress towards more reproducible and re-usable research results and outputs, notably through the use of infrastructure such as JupyterHub and Binder.

KW - Jupyter

U2 - 10.1109/MCSE.2021.3052101

DO - 10.1109/MCSE.2021.3052101

M3 - Article

AN - SCOPUS:85099730291

SN - 1521-9615

VL - 23

SP - 36

EP - 46

JO - Computing in Science and Engineering

JF - Computing in Science and Engineering

IS - 2

M1 - 9325550

ER -

Using Jupyter for reproducible scientific workflows

Abstract

Keywords

Access to Document

Fingerprint

Projects

H2020 OPENDREAMKIT: OPENDREAMKIT (partner)

Datasets

Using Jupyter for reproducible scientific workflows

Cite this