University of Illinois Urbana-Champaign

Exploring Evidence Selection with the Inclusion Network

Fu, Yuanxi; Clarke, Caitlin Vitosky; Van Moer, Mark; Schneider, Jodi

Content Files
NetSci2023_S4S_workshop_presentation_YF--2023-07-10.pdf

Permalink

https://hdl.handle.net/2142/120048

Description

Title
Exploring Evidence Selection with the Inclusion Network
Author(s)
  • Fu, Yuanxi
  • Clarke, Caitlin Vitosky
  • Van Moer, Mark
  • Schneider, Jodi
Issue Date
2023-07-10
Keyword(s)
inclusion networks, citation networks, science of science, systematic reviews, evidence synthesis, Jaccard similarity, network visualization
Date of Ingest
2023-07-27T17:26:40-05:00
Abstract
Systematic reviews have been widely adopted in the social sciences, ecological sciences, software engineering, and health sciences, to synthesize scholarly literature and provide guidance for critical decisions. Their authority depends upon several factors, and among them the relatively standardized procedure to find, select, and synthesize evidence. The selection of evidence is thus a crucial step in a systematic review. Evidence selection results in a list of publications, called the included primary study reports, which form a special subset of a systematic review’s citations. Unlike other citations, this set of citations have direct causal effect on the validity and trustworthiness of the conclusions reached by a systematic review. We proposed a new network construct called the inclusion network to study the evidence selection practices in systematic reviews (Fu, Clarke, et al., 2022; Hsiao et al., 2020). The inclusion network is a bipartite network with two types of nodes: one represents systematic review reports (SRRs), and the other represents primary study reports (PSRs). A PSR is “included” in an SRR if it is used in that SRR’s evidence synthesis. In an inclusion network, if an SRR includes a PSR, there is a directed edge from the SRR to the PSR. In our manuscript under peer review (Fu, Clarke, et al., 2022), we use the inclusion network, along with two inclusion network datasets, to address three research questions. RQ1: Do systematic reviews on a given topic consistently include the same evidence or not? RQ2: Does evidence inclusion become more or less similar over time in systematic reviews on the same topic? RQ3: Can we derive insights from the structure of inclusion networks regarding evidence selection?
Has Part
  • https://openreview.net/pdf?id=KYHoRtut9fj
  • http://doi.org/10.31222/osf.io/zh9vp
  • https://doi.org/10.13012/B2IDB-4614455_V2
  • https://doi.org/10.13012/B2IDB-6128763_V2
  • https://github.com/infoqualitylab/code_for_the_inclusion_net_manuscript
Type of Resource
  • Presentation
  • text
  • still image
Genre of Resource
conference paper
Language
en
Sponsor(s)/Grant Number(s)
NSF 2046454

Owning Collections

Faculty and Staff Research and Scholarship - Information Sciences PRIMARY
Articles, papers, and other research and scholarship from iSchool faculty and staff