Exploring Evidence Selection with the Inclusion Network

Fu, Yuanxi; Clarke, Caitlin Vitosky; Van Moer, Mark; Schneider, Jodi

Permalink

https://hdl.handle.net/2142/120048 Copy

Description

Title

Exploring Evidence Selection with the Inclusion Network

Author(s)

• Fu, Yuanxi
• Clarke, Caitlin Vitosky
• Van Moer, Mark
• Schneider, Jodi

Issue Date

2023-07-10

Keyword(s)

inclusion networks, citation networks, science of science, systematic reviews, evidence synthesis, Jaccard similarity, network visualization

Abstract

Systematic reviews have been widely adopted in the social sciences, ecological sciences, software engineering, and health sciences, to synthesize scholarly literature and provide guidance for critical decisions. Their authority depends upon several factors, and among them the relatively standardized procedure to find, select, and synthesize evidence. The selection of evidence is thus a crucial step in a systematic review. Evidence selection results in a list of publications, called the included primary study reports, which form a special subset of a systematic review’s citations. Unlike other citations, this set of citations have direct causal effect on the validity and trustworthiness of the conclusions reached by a systematic review. We proposed a new network construct called the inclusion network to study the evidence selection practices in systematic reviews (Fu, Clarke, et al., 2022; Hsiao et al., 2020). The inclusion network is a bipartite network with two types of nodes: one represents systematic review reports (SRRs), and the other represents primary study reports (PSRs). A PSR is “included” in an SRR if it is used in that SRR’s evidence synthesis. In an inclusion network, if an SRR includes a PSR, there is a directed edge from the SRR to the PSR. In our manuscript under peer review (Fu, Clarke, et al., 2022), we use the inclusion network, along with two inclusion network datasets, to address three research questions. RQ1: Do systematic reviews on a given topic consistently include the same evidence or not? RQ2: Does evidence inclusion become more or less similar over time in systematic reviews on the same topic? RQ3: Can we derive insights from the structure of inclusion networks regarding evidence selection?

Has Part

• https://openreview.net/pdf?id=KYHoRtut9fj
• http://doi.org/10.31222/osf.io/zh9vp
• https://doi.org/10.13012/B2IDB-4614455_V2
• https://doi.org/10.13012/B2IDB-6128763_V2
• https://github.com/infoqualitylab/code_for_the_inclusion_net_manuscript

Type of Resource

• Presentation
• text
• still image

Language

en

Sponsor(s)/Grant Number(s)

NSF 2046454

Owning Collections