University of Illinois Urbana-Champaign

Evaluating the CC-IDF citation-weighting scheme: How effectively can ‘Inverse Document Frequency’ (IDF) be applied to references?

Beel, Joeran; Breitinger, Corinna; Langer, Stefan

Permalink

https://hdl.handle.net/2142/96778

Description

Title
Evaluating the CC-IDF citation-weighting scheme: How effectively can ‘Inverse Document Frequency’ (IDF) be applied to references?
Author(s)
  • Beel, Joeran
  • Breitinger, Corinna
  • Langer, Stefan
Issue Date
2017
Keyword(s)
  • Recommender systems
  • Common Citation Inverse Document Frequency (CC-IDF)
  • Digital libraries
  • Weighting schemes
  • Inverse Document Frequency (IDF)
  • Related document search
Abstract
In the domain of academic search engines and research-paper recommender systems, CC-IDF is a common citation-weighting scheme that is used to calculate semantic relatedness between documents. CC-IDF adopts the principles of the popular term-weighting scheme TF-IDF and assumes that if a rare academic citation is shared by two documents then this occurrence should receive a higher weight than if the citation is shared among a large number of documents. Although CC-IDF is in common use, we found no empirical evaluation and comparison of CC-IDF with plain citation weight (CC-Only). Therefore, we conducted such an evaluation and present the results in this paper. The evaluation was conducted with real users of the recommender system Docear. The effectiveness of CC-IDF and CC-Only was measured using click-through rate (CTR). For 238,681 delivered recommendations, CC-IDF had about the same effectiveness as CC-Only (CTR of 6.15% vs. 6.23%). In other words, CC-IDF was not more effective than CC-Only, which is a surprising result. We provide a number of potential reasons and suggest to conduct further research to understand the principles of CC-IDF in more detail.
Publisher
iSchools
Series/Report Name or Number
iConference 2017 Proceedings
Type of Resource
text
Language
en
Permalink
http://hdl.handle.net/2142/96778
Copyright and License Information
Copyright 2017 Joeran Beel, Corinna Breitinger, and Stefan Langer

Owning Collections

iConference 2017 Preliminary Results Papers PRIMARY