CITREC: An Evaluation Framework for Citation-Based Similarity Measures based on TREC Genomics and PubMed Central

Meuschke, Norman; Gipp, Bela; Lipinsk, Mario

Permalink

https://hdl.handle.net/2142/73680 Copy

Description

Title

CITREC: An Evaluation Framework for Citation-Based Similarity Measures based on TREC Genomics and PubMed Central

Author(s)

• Meuschke, Norman
• Gipp, Bela
• Lipinsk, Mario

Issue Date

2015-03-15

Keyword(s)

• data analytics and evaluation
• recommender systems
• information seeking/retrieval

Abstract

Citation-based similarity measures such as Bibliographic Coupling and Co-Citation are an integral component of many information retrieval systems. However, comparisons of the strengths and weaknesses of measures are challenging due to the lack of suitable test collections. This paper presents CITREC, an open evaluation framework for citation-based and text-based similarity measures. CITREC prepares the data from the PubMed Central Open Access Subset and the TREC Genomics collection for a citation-based analysis and provides tools necessary for performing evaluations of similarity measures. To account for different evaluation purposes, CITREC implements 35 citation-based and text-based similarity measures, and features two gold standards. The first gold standard uses the Medical Subject Headings (MeSH) thesaurus and the second uses the expert relevance feedback that is part of the TREC Genomics collection to gauge similarity. CITREC additionally offers a system that allows creating user defined gold standards to adapt the evaluation framework to individual information needs and evaluation purposes.

Publisher

iSchools

Series/Report Name or Number

iConference 2015 Proceedings

Type of Resource

text

Language

English

Permalink

http://hdl.handle.net/2142/73680

Copyright and License Information

Copyright 2015 is held by the authors. Copyright permissions, when appropriate, must be obtained directly from the authors.

Owning Collections