The case for citation count ranges

The case for citation count ranges

Citation counts have a prominent role in bibliometrics, but do they really represent all citations of a document? Paul Donner shows the different ways in which citation counts can vary and proposes a method for presenting citation counts with more nuance: citation count ranges.

Citation counts have a central role in scientometrics, but unlike references, they are not directly created by researchers writing papers. Citations are produced, or mediated, by citation indexes through the large-scale automated processing of cited reference data. This makes the citation the conceptual inversion of the cited reference. Citations and their counts primarily exist in citation databases, and citation counts for the same publication commonly differ across databases. This can be for multiple reasons: databases index different bodies of literature; they use different indexing and matching methods; or they differ in how they define what counts and what does not count as a citation to a publication.

Citation counts are uncertain and can be defined in different ways

In a present-day citation database, a publication record will have a single citation count associated with it. This value can increase over time as new documents citing the publication are indexed. But citation counts may also change for other reasons. A publication record’s citation count may decrease, for example, if systematic errors in the source data are discovered and corrected, or if the citation matching algorithm is improved and incorrect matches are removed. Considerable levels of inaccuracies in citation counts, often arising from citation matching with high precision at the cost of high recall, are well documented.

Another factor that partially determines citation counts is the conceptual specification of which cited references should be included when calculating the total number of citations of a document. Many works exist in more than one version. Research articles can exist as proceedings papers, preprints, or journal-published articles, and they may also be reprinted and included as chapters in books. Books are published in translations and are issued in successive editions. Throughout these stages, the content may remain identical or change substantially.

Citation databases should have unambiguous policies for how such versions are represented as records and for how citations to versions of an identifiable work with more than one version are tallied or possibly unified across records. Depending on the use case, it could be helpful to have separate citation counts for all these versions or to have a single integrated citation count for the complete family of versions.

For example, both Web of Science and Scopus include preprint papers from various repositories as source documents and display citation counts from cited references, indicating these versions in their preprint index sections. If these papers are also published in journals, WoS and Scopus will create additional records. Those will have their own distinct citation counts, and there is no straightforward way to see the combined citation counts. Some preprint versions have accumulated hundreds of such shadow citations, which remain invisible if one only uses the citations received by the journal version. This biases conventional citation analysis against green open access publications as it underestimates their full citation impact. Consequently, researchers who make the effort to publish green open access may find themselves at a disadvantage when it comes to citations.

Citation counts are not only time-dependent, but they also depend on indexing policies, data quality, the performance of reference matching algorithms, and conceptual definitions. It is a sound convention that citation databases generally display a single numerical value, but I argue that users could often benefit from more and contextualized citation data: a publication can have more than one citation count at a given time.

Citation count ranges

For these reasons, I propose the introduction of citation count ranges for records in citation databases for more advanced use cases. In the discussion above, two such use cases can be distinguished.

The first case concerns uncertainty in the matching of citing references (data/algorithm uncertainty). Here, the minimum value is based on the strictest matching of cited references, while the maximum value results from more error-tolerant processing. Equal minimum and maximum values would indicate that there is no uncertainty arising from data quality or reference matching. Large differences would make such uncertainty explicit—uncertainty that already exists but is currently obscured.

The second use case is to distinguish between the citation count of a specific version of a publication and the combined citation count of all related versions (definitional pluralism). Such specification-derived ranges are defined according to the most specific and the most inclusive definition of a work in the database. 

An example can help illustrate these arguments. I will use the book Vom Kriege (On War) by Carl von Clausewitz, first published in 1832. This book is a relevant example because it became widely influential over time and was reissued numerous times and translated into several languages. As a result, many different versions have been cited in the literature. I calculated four alternative citation counts by changing the matching criteria and searching for cited references in the Web of Science.

To calculate the citation count range for the first use case (data/algorithm uncertainty), two searches for the work are conducted: one using stricter criteria and the other using more error-tolerant criteria. Using the cited author string ‘von clausewitz, carl’, cited work ‘vom kriege’ and cited year 1832, I find 43 citing articles in the WoS Core Collection in June 2026. Next, I allow more inaccurate reference strings (cited author: ‘clausewitz*’, work: ‘vom krieg*’, year: 1832). This search returns 66 citing works. Note that the version of the work is kept fixed to the original first German-language edition.

To demonstrate the second use case, let us examine how references to different versions of the book can lead to a wide range of citation counts. I have made a cited reference search for cited author ‘von clausewitz, c*’ and decided that, of the returned matches, I count those with no obvious errors in the name and those with cited work titles (besides the German original) that are accurate translations. This amounts to 689 citing works. Finally, I drop the accuracy restrictions on the name and title and count all cited reference strings that recognizably refer to Clausewitz’ name and the book. This gives a citation count of 1,487.

The figure below illustrates the citation count ranges calculated for Vom Kriege, varying along the two use cases of data/algorithm uncertainty and definitional pluralism.

Figure 1 Figure 1
Figure 1. A mock-up of what citation count ranges might look like in a citation database.

A user-friendly feature of citation index databases is that citation counts are clickable hyperlinks that lead to lists of citing papers. Moving towards citation count ranges, such lists could be extended by showing the actual cited reference string that was identified and matched, and showing whether that cited reference contributed to the minimum and maximum values of the citation range. This would establish a means of cross-checking and error-correction and thereby enhance transparency.

Citation count ranges would allow us to go much beyond the basic conception of citation counts. As I have shown in this blog post, displaying them would make explicit what is currently inaccessible: what constitutes citations and the uncertainty that comes from aggregating them. Integrating citation count ranges in citation indexes could help prevent the inadvertent undervaluation of green open access publications and allow for robustness checks of bibliometric studies with alternative citation counts. Broad adoption of citation count ranges might even spell the end of the notion that citation counts are singular values without any uncertainty.

Header picture by Dong Xie on Unsplash
DOI: 10.59350/6m54z-g1655 (export/download/cite this blog post)

1 Comment

Serhii Nazarovets

I really like this idea. One thought that came to mind is that part of the problem might be addressed by using the relation metadata that already exist in Crossref and DataCite. Many records are explicitly linked as preprint-journal article, accepted manuscript-published version, translations, corrections, and so on. Instead of reporting only the citation count of a single DOI, citation indexes could also calculate the citation count of the entire family of related records. This would help solve the "shadow citations" problem, where citations to a preprint are separated from citations to the published article. Users could then see both values: citations to this specific record and citations to the complete work. It would also make the aggregation policy much more transparent, since different use cases may legitimately require different definitions of what should be counted as the same work.

Add a comment