Critically evaluating collaborative research: why is it difficult to extend truth tests to reality tests?

We argue that critical evaluation achieves the reflexivity needed to facilitate collaboration by proposing boundary-negotiating artefacts to configure a joint action domain. Those objects become mediators for innovation by triggering controversies, conceived preventatively via an organized extension of what Boltanski calls ‘truth tests’ to ‘reality tests’ so that they dynamize ongoing affairs. However, critical evaluation must also anticipate actors’ reappropriation of boundary-negotiating artefacts in the effort to protect their rights, stakes or room for manoeuvre. Three scenarios commonly arise: avoidance or utopian projecting, enactment of inverted reality tests, and disavowal through role exchange. The article develops these propositions through the reconstruction of a modified theory-based evaluation of a collaborative research programme. The programme set out to explore how evidence from health research could be used rapidly and effectively in the context of practical problems and organizational challenges, so an internal evaluation was set up to facilitate learning during the process. What ensued, however, was a loss of trust between partners, resolved only by repositioning the evaluation as a reflective academic study, reducing its reflexive capacity to intervene on the level of activity and organizational integration. We conclude that doing successful critical evaluation and, more generally, achieving political pertinence for social scientific discourses depends on creating the conditions in which actors are able to take the risks and share the costs associated with the enhanced level of reflexivity necessary to engage in collective action as well as knowledge production.