The promise and pitfalls of co-creation

It is a huge pleasure to have the opportunity to speak with this audience today — an audience of people who are deeply committed and working hard, to bring the best of the “evidence agenda” to bear on important issues that affect millions and millions of people around the world.

As I sat down to write this talk a few weeks ago, here’s what first came to mind:

We are warriors in a battle.

When I say that we are in a battle, you might think of the daily struggles. Some days it might feel like a fight against funders with inflexible budgets, clients with unrealistic timelines. Other days, it might feel like a fight against the statistical bias that creeps in when survey respondents refuse to answer the phone. But the bigger fight we are in is to elevate reason and facts over reflex and disinformation.

This is the central battle of this age.

It is the central battle of this age because facts and the ethical application of scientific reasoning are essential for rescuing us from existential threats, whether those are devastating pandemics or the destruction of the planet. Only through collective investment in evolving science, paired with a collective confidence in scientific findings, will we meet the challenge of creating a habitable and joyful world for our children and theirs. And that includes not just physical and biomedical sciences, but crucially the study of human societies.

It is also the central battle of this age because democratic institutions cannot thrive without the capacity to respond to the needs of citizens, of voters. And responding to those needs requires more than rhetoric about values. It takes generating and using evidence about the effectiveness of social and economic policies and programs, and smart, data-informed day-to-day decision making. We see time and again that lack of confidence in the ability of democratically elected governments to deliver opens the space for dictatorship and fascism.

But, you may be wondering, what does this have to do with the title of the talk, “The Promise and Pitfalls of Co-creation”? Stick with me, and I’ll try to convince you that winning the war I’ve just referenced requires evidence not only to be generated but to be used. Use requires trust. Trust requires benevolence ¹. And benevolence requires co-creation. To win the war, we have to co-create evidence with those who must trust evidence to use it.

Let’s start with who we are. Many of us are motivated by an unfailing belief that the work of both governments will best serve people who live in conditions of vulnerability if the right kind of data and evidence are used for decision making. What I mean by the right kind of data and evidence is:

• information that fully represents the realities of life in disadvantaged and marginalized communities;
• evidence that accurately captures the net impact of programs that are intended to preserve life and enhance livelihoods;
• aggregated evidence about “what works” in distinct circumstances to improve the efficiency with which precious communal resources are used; and
• on-going, representative feedback from people who are involved in offering or receiving services, to inform ongoing management and adaptations.

This type of data and evidence, used well, is the lever that has the potential to lift investments that offer small or zero social returns to ones that propel social mobility, the closing of social gaps, and much more. In short, it is a fundamental instrument that turns political rhetoric that wins votes into the allocation of resources and the design of public policies and programs that changes lives for the better.

If democratically elected governments are going to maintain legitimacy in the eyes of their citizens, they have to deliver. And the use of data and evidence is essential to doing this. In theory, that’s where we come in. Our work should be seen as the most precious input to decision making — and yet we often find ourselves wringing our hands about how little our work is used, huddling together and affirming our mutual importance, vainly wishing that the people holding power as well as the powerless would see the wisdom of evidence-based actions.

So we have to apply the same analytical mindset and data-informed decision making introspectively, to our own work.

Why is this battle so hard to win?

To fulfill the potential of the use of data and evidence, we have to recognize that people use it when it is relevant to their lives; when it is in a form that can be understood; when using it brings rewards (or at least not punishment); and when it is from a source they trust.

I believe it is this last aspect — whether data and evidence come from a trusted source — that merits far more effort than we have put into it so far. And I think this is at least in part because we ourselves have been acculturated through professional training to assume trust rather than to earn it.

A long line of academics has boiled trustworthiness down into three characteristics: ability; integrity; and benevolence. To trust us, people have to believe that we know what we’re doing, that we’re being honest with them, and that we have their best interests at heart.

Now, you may be thinking: we’ve got this covered. As a community, we are all about ability. We have advanced training and we train others — for heaven’s sake, we are the ones who build others’ capacity. We develop and document sophisticated methods, and codify our standards in hierarchies of evidence, we peer review. In fact, we spend the vast majority of our time as a community of practice on establishing our ability.

We also have put a lot of energy into integrity. Think of the work on research transparency, to squeeze out the space for conflicts of interest to bias our findings. We have codes of conduct. We censure the miscreants, shun the fraudsters.

What we haven’t spent nearly enough time on, in my view, is establishing that we have at heart the best interests of the people whose trust we are seeking. And the best way to do that is to involve them in a true partnership.

Whose trust are we seeking?

We are seeking the trust of two types of people who are notoriously hard to persuade away from pre-established beliefs: public officials, and those people who are affected by public policies.

We want to gain the trust of public officials, including politicians because it is often their decisions we wish to inform with our data and evidence. A central way to demonstrate our trustworthiness to the public officials making the decisions is to understand and learn from their perspective. At IDinsight, for instance, we start all our work by trying to understand our clients’ operating environment and the type of decisions they face. We are sitting side-by-side in ministries and departments, trying to squeeze meaning out of administrative data, and designing evaluations that fit tight budgets and timetables. It is from our clients that we learn about formal and informal power centers, internal organizational incentives, and the politically acceptable solution space. It is in service of their needs that we orient all our work.

We have explicit agreements with our clients about what information we will share publicly and what we will not share. We work intentionally behind the scenes, not claiming public credit unless that is acceptable to our clients. These approaches are a few ways we reinforce relationships of trust with the governments with which we work — and what we see is that often (although definitely not always) they help to create a virtuous cycle of more and more use of data and evidence.

Then let’s turn to the trickier question of how to establish trustworthiness with the people who are affected by public policies, and why that’s important. Here, we at IDinsight have a long way to go, and I suspect that is true for many members of the evidence community.

There are a couple of reasons to focus on trust-building with the public.

First, because these are the people who should be represented — both democratically and statistically. We are working for them, just as politicians and public officials are working for them. It is their interests that matter most of all.

Second, if our work is not trusted by those who are holding politicians to account — if it is seen as a means of serving the interests of those in power — it will ultimately undermine rather than reinforce democratic institutions. To put it in a more positive light, if the work we do on data and evidence is seen as part of the way in which people have their voices heard, their experiences represented in an authentic way, this will contribute to pressure on politicians to use that input in their decisions.

Okay, so now let’s drill down on how we might increase the trust that people have in us. Again, I’m talking about everyday people, and particularly people who may have a history of experience with systems that do not have their best interest at heart.

Let me suggest that we need to relax our iron grip on “expertise,” and lean into forms of “co-creation.” While it flies in the face of professional traditions many of us have internalized, we won’t be well served by conceptualizing expertise in exclusionary terms, as something held by people only with certain degrees or pedigree. Instead, we need to think about expertise as multi-faceted and broadly owned. As evidence professionals, we can do this through deep engagement and appreciation of knowledge brought by others, leaving behind our understanding of the world as being made up of those with expertise and those without. And we can do this by creating space — moving aside — for more equitable representation within the evidence community.

A frontier is in the domain of “research equity,” which places communities that are being studied at the center. Bias and expert-focused exclusivity often is manifested throughout the research lifecycle — from determining the priority questions and the need for data, to collecting it, reporting, use in algorithmic routines, and interpretation. At each stage, then, there is an opportunity to open things up by involving — as full partners –the people whose data is being captured and making sure their needs and interests are driving the work. In concept, by reorienting data to the needs of the community, it becomes more genuinely representative and empowering.

I’ve been inspired by the work of Chicago Beyond, an organization founded by a former school principal, Liz Dozier, that is focused on increasing opportunities for Black youth in Chicago through programming and advocacy. Chicago Beyond is deeply grounded in justice and respect for the community.

Here’s one passage that motivates their work toward more community-engaged and -owned research. I think it will resonate with much of our collective experience:

“What is the history of research, in Chicago, nationally, and beyond? Rather than a partnership of equals, there is a legacy of researcher “brains” and community “brawn.” In many communities, the remembered history is that when the community and research institution interact, the institution benefits. Countless research surveys mine communities for the raw material of lived experiences, without yielding much for the community — or worse. Yet, there remains a lack of evidence about the value of interventions for those from whom the most has been taken. That “lack of evidence” justifies investing less still. Community organizations often cannot afford access either to large datasets or to the kinds of researchers that institutions attract, and their own data and stories have limited influence on decision-making until those institutions authenticate them. Like most people outside of universities, community organizations may not be aware of the options in research, or what they may risk for scientific rigor. Some may not dare to assert themselves, internalizing biases about who the experts are.”

The guidebook then offers specific recommendations about how affected communities can be engaged in authentic ways, from identifying the questions, through to learning and acting on the answers.

One of the clear messages is that a partnership is required between civil society organizations that themselves operate in relationships of trust because they have a track record of representing and serving the community and research teams and institutions. It is a challenging “co-creation” agenda, but one that is essential to work toward if we are to fulfill the potential that data and evidence have to improve lives.

I’ve talked about a lot of things in a short time, so let me just recap in simple — perhaps simple-minded — terms:

If we believe that data and evidence are integral to saving lives, expanding opportunities, protecting our children’s futures, we need to step up our game. We are not winning this war, even if we take pride in the occasional successful battle.

Stepping up our game is not about elevating ourselves. It is about elevating others to join us in formulating questions, collecting information, interpreting observations through multiple perspectives.

When we invite others into our work, and when we behave respectfully and wholeheartedly as others invite us into their work and lives, we have opportunities to build trust. And it is trust-building that creates the conditions for data and evidence to be used to their fullest.

In closing, I want to reiterate that this is very much an aspirational agenda for me and not one in which I claim proficiency. I do encourage us all, though, to think carefully not only about how to gain the trust of those who share our professional bubbles but about how to gain the trust of those we serve.