This paper presents an in-depth case study about the Dialogue between Scientific Councils, also referred to as the Beirätedialog, which is a format for cross-sectoral science policy consulting on sustainable development in Germany. Set up to address current trends, it is designed to facilitate deliberation and collective knowledge creation between scientists and policymakers. Based on 4 years of participatory observation, we analyze to what extent this goal can be achieved and present some empirical insights about the main difficulties that occurred. We argue that creating a space for interaction does not guarantee collective knowledge production and identify key learnings that can help design such a process. In support of the growing interest in communication at the intersection of science and policymaking, our research seeks to deepen the understanding of the dynamics of co-creative processes and offer some insights on how to overcome the main challenges.