Leigh Parise: Policymakers talk about solutions, but which ones really work? Welcome to Evidence First, a podcast from MDRC that explores the best evidence available on what works to improve the lives of people with low incomes. I’m your host, Leigh Parise.

While MDRC was honing its ability—over the last 50 years—to build knowledge in the public policy field, it was also developing a robust set of strategies and tools for helping partner organizations and agencies implement that knowledge. As program staff [members], practitioners, and researchers know, just knowing what works isn’t all it takes to reach the goals you’ve set for your program. MDRC helps organizations build new evidence, use existing evidence, and harness data to advance their goals, utilizing deep programmatic expertise and creative collaboration. This is known as technical assistance (TA).

In its 50th anniversary year, MDRC announced the launch of Evidence to Practice, which draws on those years of on-the-ground experience to establish its approach to technical assistance work. The initiative outlines MDRC’s approach to technical assistance and the four types of technical assistance MDRC offers: using data, assessing impact, diagnosing and addressing challenges, and innovating and expanding services.

Today we are joined by John Martinez, MDRC’s first vice president for Evidence to Practice, [who will] tell us more about this exciting initiative—from the familiar to the fresh. John, welcome to Evidence First. I’m so excited for our conversation. You and I have been in lots of different venues together over the years, but this is the first time we get to do Evidence First together and talk specifically about Evidence to Practice. I’m really excited.

John Martinez: Thanks, Leigh. I’m really excited to be here as well, not just to spend time with you—which I always enjoy—but because I get to talk about this really exciting work.

Leigh Parise: Great; let’s jump right in. This might seem obvious to some, but let’s start at the top: Can you tell us what this phrase “evidence to practice” actually means to you—and then more broadly, for MDRC’s work?

John Martinez: It’s really interesting. As I reflected on taking on this role, I thought a lot about the tagline that we often use at MDRC to describe the work, which is “building knowledge to improve and inform policy and practice.” I think the “building knowledge” is obvious: That’s the research that we do and it’s what we’re well-known for. But that piece about informing policy and practice—that’s where I see the Evidence to Practice work playing out in an important way. It’s easy to think about large-scale and small-scale evaluations where we’re trying to figure out what works and who it works for. But then what happens? Often, we think of these long research reports that document all the good work that was done and all the important findings that emerged from that work, and then they sit on a shelf.

That’s not what we want. That’s not what MDRC is about. We believe it’s really important for people to actually use that evidence to make a difference for the participants that they’re serving. And I think at its heart, that’s what Evidence to Practice is. It’s taking the knowledge that we and others have built around what works and then applying it to the real-world challenges that program operators, administrators, and frontline service staff are actually facing. To me, that’s important—not only for MDRC, in terms of our value to the field, but for the participants, at the end of the day, because we want to make sure that they’re benefiting from services that have been shown to work.

Leigh Parise: I appreciate that that’s where you landed, just now. That ultimately is why we do this work.

I know that some people might be familiar with MDRC as a place that conducts randomized controlled trials (RCTs), but tell us more about the technical assistance we offer and how you think that builds and complements some of what MDRC has learned over the years (and may also be working on now).

John Martinez: I think the answer to this question—I want to go back 50 years, the time machine going backward and thinking a little bit about MDRC’s history.

Leigh Parise: Got to love a time machine. All right.

John Martinez: When we were created—50-plus years ago, now—we were actually created as an intermediary that operates programs and delivers technical assistance and makes sure that the programs that were on the ground were working. We outsourced all of our research to subcontractors and managed the research. So it’s really interesting to think [that] now, 50 years later, most people associate us with these large-scale random assignment studies. And we have done many of them; I feel like we’re pioneers in that field. But the reality is, when we started, we were essentially delivering technical assistance. I’m proud to say that we never lost that. That’s part of our DNA; it’s part of who we are. I think that this work that we’re doing now—it isn’t new, it’s just a new way to think about how to talk about and how to frame the work that we’re doing and building on that 50 years of experience.

Leigh Parise: I think that’s really important. I appreciate the “time machine,” partly because I think people are sometimes surprised to hear that that’s how MDRC started. When we talk about our Evidence to Practice work, my worry sometimes is that they think, like, Oh, now MDRC is deciding, “This is a space that other people are in so we’re going to jump in too.” But that’s not the reality: It’s always been part of what we do. So thanks for taking us back there.

John Martinez: As we think about this work and think about what it is we’re doing on the ground, I want to spend a couple of minutes helping people understand: What does this actually look like? What does it mean when we’re working with organizations in an Evidence to Practice engagement? There are a few things that I think are important, and I think you teed them up nicely in your introduction.

One is [that], across our 50 years and all the technical assistance that we’ve done, you might look at that work and think, Ah, they’ve done all these projects. There are all these frameworks, there are all these approaches, there are all these ways to engage with organizations. But we spent the last two years diving deeply into all of our technical assistance work and realized it fits into four buckets of work, all of them centered around helping organizations do what they do—using evidence to inform that work—to ultimately benefit the participants that they’re working with. It’s really about driving outcomes for the participants. You’re going to hear that over and over again because that is central to our approach. Through all of this work, we were able to essentially characterize all of our technical assistance into what we’re calling four service lines or four buckets of work.

One bucket is around helping organizations use data. A lot of organizations collect a ton of data and some organizations are better adept at using that data to inform how they approach their work—where other organizations might use it more for compliance reporting to funders and things like that. We can partner with organizations and help them think, How do I maximize the data I’m already collecting to ensure that we’re helping participants achieve the goals that they want to achieve? One of the things that I think we bring to the table, in this area of work, is [that] we have staff [members] from our Center for Data Insights that are experts on how to think about analyzing and using data.

A second area of work, which I think centers the work that MDRC does, is assessing impact. What does that mean? Well, organizations want to know that they’re actually making a difference, over and above what participants would’ve been able to do on their own. The assumption is Great, let’s launch right into a rigorous study to be able to answer that question. The reality is that not every intervention—not every organization—is ready for rigorous evaluation and some may never be. And that’s okay.

We work with organizations to figure out what’s the most rigorous thing we can do to help them tell their story about the impact they’re making for the participants that they’re serving, over and above what would’ve happened if they hadn’t been around. This can look a lot of different ways. It could be helping them think about the data that they’re collecting, think about their logic model, their theory of change—all the way to setting them up to being able to do more rigorous studies. I think of that as a continuum of work, and we might work with an organization for years to get them from where they are to being able to demonstrate impact through a rigorous study. All of that takes collaboration and a lot of thinking to be able to do that.

A third area that we try to tackle: working with organizations to diagnose and address challenges that they might be facing, whether it’s in their service delivery [or] it’s in how their participants are navigating the program. It could be about recruitment of participants; it could be about engagement of participants. But all of this is about Let’s figure out what your key questions are. Let’s diagnose why that challenge exists, and then let’s use evidence to come up with solutions that can help solve that particular challenge.

And because we’re MDRC and we think this is really important, we think it matters to build in learning cycles—as we’re doing this—to make sure that the technical assistance that we’re delivering and that they’re implementing is actually making a difference for the participants. This also allows us to continue to tweak what they’re doing on the ground to be able to make a difference for their participants.

The last bucket is about innovating and expanding services. It’s really about scaling: whether it’s helping organizations serve more participants or helping organizations scale their interventions to other places. In either case, you can just go full steam ahead and do it, or you can think about how evidence can inform that scaling effort to make sure that what you’re doing is going to lead to impacts for the participants that you’re serving.

That can look different in different scenarios. But I think the bottom line for why this is important is—  Over the years, there’ve been lots of evaluations of what’s called replication. So we take an intervention, we’ve replicated it somewhere else, an evaluation’s done on that replication, and we don’t see the results that we saw in the original iteration of that particular intervention. That’s a real challenge. You want to make sure that as you replicate, as you scale, you’re making that same difference.

And we can come in and help organizations identify why there are particular challenges or barriers that may lead to there not being that same impact, wherever it is you’re replicating. So what does that mean? It means understanding the evidence and then being able to contextualize that evidence—have it make sense in the new place that the organization is expanding to.

Those, in a nutshell, are the four areas that we think of as we work with organizations. We can work with organizations in one of those areas, two of those areas, three of those areas, or all four. Often, over the years, we have worked with organizations across all four, meeting them where they are and then figuring out where it makes sense to pick the work up.

Leigh Parise: I love the detail that you provided for each of those areas because sometimes people ask me, “What does it mean that MDRC does technical assistance?” Now I get to just send them this podcast.

For anybody listening who might be thinking, Hmm. All right, but is some of this aspirational? Do you actually have examples of where you’ve done all of this in the past? How do you think about that?

John Martinez: Yes, we have done all of this in the past. Across our 50 years, we have many examples of engagements that fit into one of these four service lines. Just as a little bit of a teaser, we are going to start releasing case studies—examples—of each of these. Because we can sit here and talk about a lot of this work at an aspirational level or at a high level. But I think it’s much better to engage and understand this work if you actually read real world examples that talk not only about the reasons why the organization we partnered with wanted to do this work, but what the result of that engagement was and how it helped the organization and their participants.

Leigh Parise: I think that’ll help any organizations that we might work with also see themselves potentially mirrored in some of those case studies. I also really like having inventoried everything that we’ve done. I think it both helps to clarify [what] the buckets are where we actually seem to be providing the best and the most support to people and allow us to make sure that we’re drawing on the past experiences that we’ve got. Thank you.

A lot of organizations offer technical assistance. Tons. In all kinds of topics, in all kinds of spaces. Can you say a little bit about what you think sets MDRC apart?

John Martinez: Sure. I’m going to talk about this in a couple of different ways. I’m going to start from the perspective of the organizations that we work with. I think one of the key values that we bring to all of our engagements is coming in with a sense of humility, empathy, curiosity. I know those sound kind of soft and fluffy, potentially, especially coming from a research organization. But that is so critical because the organizations we’re collaborating with, they’re the experts on the ground. We are bringing in expertise in terms of evidence: what the evidence suggests, and 50 years of accumulated organizational experience. But the bottom line is they’re the experts, and so it’s really important for us, when we engage with them, to come in with that sense of humility and curiosity and really listen to them so that we can meet them where they are.

I would say that’s the second thing: We don’t think of our technical assistance as just being off-the-shelf TA. We think it’s really important to meet organizations where they are. And frankly, that takes a little bit of time up front to get to know the organization, to get to know how they operate and how we can best add value to what it is they’re doing. Because that's our bottom line: If we’re not adding value to what they’re already doing, then why do this? It doesn’t really make sense.

Two other things that I think are really important in this space, then I’ll switch gears a little bit. One is making sure we engage the frontline staff—to understand their perspectives and how they approach serving their participants. Because at the end of the day, they’re the ones who have to implement this technical assistance. If it doesn’t make sense for them, it’s not going to really help the participants achieve the outcomes that they want to achieve. So really trying to center the knowledge and the experiences of the staff: [that is] critical to the work that we do.

The last thing I will say—certainly not the least important, I would actually say it’s the most important—is centering the needs of the participants. You can’t design recommendations about service delivery if you don’t take into account the perspectives of the participants that are actually navigating those services. We use a lot of different tools—like journey mapping and focus groups and participant engagement—to make sure that we’re centering those experiences so that the recommendations that we make make sense to the participants that are actually receiving those services. That is absolutely critical. I think those are some of the values that we bring to the work.

Now the other piece that is absolutely central to what makes our technical assistance, in my opinion, different from other types of technical assistance is that centering of evidence. It’s using the evidence to inform our recommendations. In everything that we do, we try and understand, Well, what does the evidence say about who this works for and what context it works for? As I noted in the beginning, one of my strengths—and one of our strengths—is then translating that so that it makes sense for the organization we’re working with and the context that they’re in.

Leigh Parise: It’s good for people to hear that, partly because I think [one] of the reasons that we do this is that many of us have experiences ourselves as practitioners working in nonprofits, working as teachers, working in colleges, working in agencies. Bringing the humility, and that we always lead with that. I think that’s been part of our DNA at MDRC for so long, and so I love that you’re at the helm here.

John Martinez: I think that’s so important, Leigh. One of the things that we always joke about at MDRC (but it’s based in fact): We don’t parachute in, do our work, and disappear.

Leigh Parise: John, I know we’ve worked with a lot of nonprofits over the years, and I’m curious how you think about finding partners and potentially working with partners over long periods of time?

John Martinez: That’s a great question, Leigh. It’s one of the things that I’m really proud of, here at MDRC. We often joke about the organizations that keep coming back to us when we approach them about another opportunity, another engagement. I think that says a lot about the relationships that we’ve been able to build with these organizations, and speaks to the values that I mentioned earlier (in terms of how we approach these engagements). People vote with their feet and I think that if an organization did not have a good experience with us, they likely wouldn’t want to work with us again. I can think [of]—in the course of my career—organizations that I’ve worked with three, four, or five times, or I have been chatting with the staff there for the last 15, 20 years of my career. That just really speaks to, not only the relationships we’ve built, but the value that these organizations think that we can bring to the work that they’re doing.

Leigh Parise: I hope, and imagine, that that will continue to be true. It’s really cool to get to know a program so well over such a long period of time. I think that makes understanding the context and thinking about the opportunities in some ways much smoother and much easier for an organization and for MDRC to be able to partner with them.

John, this has been a really great conversation. Thanks so much for joining me. It’s been fun. Is there anything else that you want to say before we sign off?

John Martinez: First, thank you for having me. I’ve really enjoyed spending time with you and chatting about this work. I’m so excited about where we’re headed with this work. The second thing that I’ll say is if anyone listening to this is interested in learning more, they should reach out to us. We have an email address. It’s [email protected]. Someone will get back to you.

Leigh Parise: That’s great. Thank you. John, this is really exciting work. I’m pumped that you’re leading it and I know that a lot of people at MDRC are excited to continue.

To learn more about Evidence to Practice and MDRC’s technical assistance work, visit MDRC.org. Did you enjoy this episode? Subscribe to the Evidence First podcast for more.