Skip to main content

Review of "Indicating interdisciplinarity: A multidimensional framework to characterize Interdisciplinary Knowledge Flow (IKF)"

Published onAug 12, 2022
Review of "Indicating interdisciplinarity: A multidimensional framework to characterize Interdisciplinary Knowledge Flow (IKF)"
key-enterThis Pub is a Review of
Indicating interdisciplinarity: A multidimensional framework to characterize Interdisciplinary Knowledge Flow (IKF)
Indicating interdisciplinarity: A multidimensional framework to characterize Interdisciplinary Knowledge Flow (IKF)

This study contributes to the recent discussions on indicating interdisciplinarity, i.e., going beyond mere metrics of interdisciplinarity. We propose a multi-dimensional and contextual framework to improve the granularity and usability of the existing methodology for quantifying the interdisciplinary knowledge flow (IKF) in which scientific disciplines import and export knowledge from/to other disciplines. To characterize the knowledge exchange between disciplines, we recognize three dimensions under this framework, namely, broadness, intensity, and heterogeneity. We show that each dimension covers a different aspect of IKF, especially between disciplines with the largest volume of IKF, and can assist in uncovering different types of interdisciplinarity. We apply this framework in two use cases, one at the level of disciplines and one at the level of journals, to show how it can offer a more holistic and detailed viewpoint on the interdisciplinarity of scientific entities than plain citation counts. We further compare our proposed framework, an indicating process, with established indicators and discuss how such information tools on interdisciplinarity can assist science policy practices such as performance-based research funding systems and panel-based peer review processes.

As a signatory of Publish Your Reviews, I have committed to publish my peer reviews alongside the preprint version of an article. For more information, see

This paper proposes a bibliometric methodology for characterizing interdisciplinary knowledge flows. Instead of trying to quantify interdisciplinarity, the authors propose to characterize interdisciplinary knowledge flows. They refer to this as ‘indicating interdisciplinarity’, in line with recent suggestions in the literature.

I very much support the general idea of the paper, but I also believe that the framework proposed by the authors for characterizing interdisciplinary knowledge flows needs further development. The framework focuses on three dimensions, referred to as broadness, intensity, and heterogeneity. For each dimension, the authors provide a bibliometric indicator to quantify the dimension. From my point of view, there are two crucial aspects of the proposed framework that need further development.

My first point is about conceptualization: The framework proposed by the authors needs a proper theoretical or conceptual foundation. At the moment, the authors offer little conceptual reflection. The three dimensions are introduced by providing mathematical formulas instead of clear conceptual arguments. As reader of the paper, it is not clear to me why the authors have chosen to focus on these three dimensions rather than others. I would expect the authors to introduce their framework by first making a conceptual argument that clarifies why broadness, intensity, and heterogeneity are considered to be the relevant dimensions to characterize interdisciplinary knowledge flows: What does each of these three dimensions represent? What is the reason for focusing on these three dimensions rather than others? Why do we need all three dimensions instead of just one or two of them? And why don’t we need additional dimensions beyond these three? Clear answers to these questions are missing, and therefore I am not convinced that the authors have indeed identified the relevant dimensions for characterizing interdisciplinary knowledge flows.

My second point is about operationalization: After providing a conceptual foundation for their framework, the authors need to operationalize the different dimensions of their framework, which can be done by introducing a bibliometric indicator for each dimension. The authors indeed provide bibliometric indicators in their paper. However, what is missing is an explanation of the way in which each indicator offers an operationalization of one of the three conceptual dimensions. This also requires an analysis of the properties of the indicators: Does an indicator indeed have the right properties to operationalize a particular conceptual dimension? For instance, is an indicator size-dependent or size-independent (i.e., does the indicator tend to increase or decrease when the number of documents in X or Y increases, or not) and is this indeed desirable from a conceptual point of view?

In more concrete terms, I think it is desirable for the indicators to be size-independent, but I am not sure whether they are indeed size-independent. For instance, consider two sets of documents, X and Y. Suppose we obtain specific values for the broadness, intensity, and heterogeneity indicators for these sets X and Y. Suppose X includes 1000 documents and suppose we randomly select 500 of these documents. This yields the subset X’. I think the values of the broadness, intensity, and heterogeneity indicators obtained for the sets X’ and Y should be approximately the same as the values that were obtained for the sets X and Y. For the broadness and intensity indicators, this indeed seems to be the case, which means they are size-independent for X, but for the heterogeneity indicators I think this is not the case, so this indicator seems to be size-dependent for X. This would mean that the heterogeneity of, for instance, a field or a journal is partly a reflection of the size of the field or the journal, which may not be desirable (e.g., two very similar journals may have different heterogeneity values simply because one journal is five times larger than the other). From the perspective of Y instead of X, it seems that all three indicators are size-dependent, and again this raises the question whether this is desirable or not.

I hope the authors will be able to further develop their framework for characterizing interdisciplinary knowledge flows. I refrain from commenting on the empirical part of the paper. In general the empirical analysis looks good to me, but I find it difficult to interpret the empirical results without having a more fully developed conceptual framework.

I look forward to reading a revised version of this paper!

No comments here
Why not start the discussion?