or2yw: Modeling and Visualizing OpenRefine Histories as YesWorkflow Diagrams

Parulian, Nikolaus N.; Li, Lan; Ludäscher, Bertram

Permalink

https://hdl.handle.net/2142/109699 Copy

Description

Title

or2yw: Modeling and Visualizing OpenRefine Histories as YesWorkflow Diagrams

Author(s)

• Parulian, Nikolaus N.
• Li, Lan
• Ludäscher, Bertram

Issue Date

2021-03-17

Keyword(s)

• Data Cleaning
• OpenRefine
• Provenance
• Workflows

Abstract

OpenRefine is a popular open-source data cleaning tool. It allows users to export a previously executed data cleaning workflow in a JSON format for possible reuse on other datasets. We have developed ortoyw, a novel tool that maps a JSON-formatted OpenRefine operation history to a YesWorkflow (YW) model, which then can be visualized and queried using the YW tool. The latter was originally developed to allow researchers a simple way to annotate their program scripts in order to reveal the workflow steps and dataflow dependencies implicit in those scripts. With ortoyw the user can automatically generate YW models from OpenRefine operation histories, thus providing a ``workflow view'' on a previously executed sequence of data cleaning operations. The ortoyw tool can generate different types of YesWorkflow models, e.g., a linear model which mirrors the sequential execution order of operations in OpenRefine, and a parallel model which reveals independent workflow branches, based on a simple analysis of dependencies between steps: if two operations are independent of each other (e.g., when the columns they read and write do not overlap) then these can be viewed as parallel steps in the data cleaning workflow.The resulting YW models can be understood as a form of prospective provenance, i.e., knowledge artifacts that can be queried and visualized (i) to help authors document their own data cleaning workflows, thereby increasing transparency, and (ii)~to help other users, who might want to reuse such workflows, to understand them better.

Publisher

iSchools

Type of Resource

text

Language

eng

Permalink

http://hdl.handle.net/2142/109699

Copyright and License Information

Copyright 2021 is held by Nikolaus N. Parulian, Lan Li, and Bertram Ludäscher. Copyright permissions, when appropriate, must be obtained directly from the authors.

Owning Collections