#teacher | Straight Up Conversation: Can Technology Help Districts Improve Teacher Hiring? – Rick Hess Straight Up

Lauren Dachille is the founder and CEO of Nimble, which has built data-driven applicant-tracking software to help school districts identify and hire the best-fit teachers for their classrooms. Nimble serves close to 50 school districts across a dozen states, affecting outcomes for nearly half a million students. Prior to founding Nimble in 2016, Lauren worked on human-capital teams at D.C. Public Schools and the nonprofit StudentsFirst. I recently talked with her about Nimble and how to identify the most promising teacher applicants, and here’s what she said. 

Rick Hess: So, just what is Nimble?

Lauren Dachille: Well, Nimble’s overarching goal is to help schools predict which teacher applicants will be most effective in their classrooms—and the way we do this is with our applicant-tracking software. Put simply, school districts post their job openings via Nimble, then when candidates apply, we use a predictive model to give HR and hiring managers, usually principals, insights about which applicants are most likely to be a strong fit. Over the last few years, there’s been a growing body of research pointing to several pre-hire predictors of classroom success. Nimble is building on that research and putting the findings at districts’ fingertips during the hiring process. And we’ve found that districts are pretty excited about this work. We’re already working with nearly 50 districts across 12 states, including several big-city districts, which tend to be the ones with the biggest pain points around hiring.

Rick: What prompted you to take this on?

Lauren: We all know how impactful great teachers are. And although there are certainly shortage areas, there are also plenty of subject areas with an excess of applicants and no way for districts and principals to sort through them. What ends up happening is that principals go around the district-hiring process, through their personal networks, to find talent. Often this means they’re missing out on high-potential hires. If there’s actually data out there that can help us predict which teacher applicants will most move the needle on student learning, we should be using it! With all the controversy over the last several years about evaluating and firing teachers, I think smarter hiring is a simple and obvious solution to ensure students have the right people leading their classrooms from day one.

Rick: You’ve noted that when you were on the human-capital team in Washington, D.C., you found antiquated HR software could get in the way of school improvement. Can you say a bit more about what you mean by that?

Lauren: Yes, even the strongest and most thoughtful human-capital and HR teams are often hampered by outdated tools and processes. The landmark study on this is TNTP’s Missed Opportunities. They looked at hiring data from four “hard to staff” districts and found that with aggressive recruitment, they were able to recruit more than enough applicants in shortage areas. However, applicants often withdrew after months in limbo—and those who withdrew were not only the most qualified, but they were serious applicants with a preference to work in these districts! Some people ask me—what does software have to do with this problem? And the answer is a lot! There are often huge bottlenecks in application processing. In my first year at DCPS [D.C. Public Schools], it wasn’t atypical for someone to apply and then wait several weeks to hear anything back from us. Those folks would often end up in surrounding districts, where they got a higher touch and a faster response. Since then, DCPS has invested quite a bit in the applicant experience, and it’s really paid off in their ability to recruit high performers. Districts need hiring tools that promote efficiency and prioritization.

Rick: And what kind of tools can help?

Lauren: This could mean something small, like an automated bottleneck notification that prompts the HR team to follow up with candidates who have been invited to interview but haven’t signed up yet. Or it could mean something big, like a predictive model that leverages push notifications to get the top-tier candidates onto principals’ radar quickly. Nimble does both of these and more. The model plays a big role, but it’s only one piece of the puzzle!

Rick: Can you tell me a bit about what factors you look at to predict whether a candidate will be a good hire?

Lauren: At a high level, we’ve explored any and all factors that have been shown in historical research to have predictive power as it relates to teacher effectiveness and retention. We test these predictors in real-time in partnership with the districts using Nimble, many of whom provide us with anonymized effectiveness and retention data for their new hires at the end of each academic year. What we’ve found so far has been quite interesting. First, metadata, meaning information about how a candidate interacts with the application process itself, can play a really important role in identifying high performers. As the applicant-tracking system, we have access to many unique data points that haven’t been looked at previously. For example, we know from prior research that candidates who apply earlier in the hiring season are more likely to be effective. With Nimble, we can take it one step further—how many jobs has this candidate applied to, across which job categories, and how many days passed between the date the job was posted and when they applied?

Rick: And what do you look at when it comes to the candidates themselves?

Lauren: Then, of course, there’s a treasure trove of application content that we can analyze. We’re going beyond just the basics like GPA and undergraduate institution and moving toward analyzing unstructured application data like content assessment essays, pedagogical approach, and more. Someday soon, we hope to introduce some interesting machine-learning techniques to look at things like teaching demo videos. Essentially, with any given applicant, there are hundreds of data points we can look at and match with student growth and retention outcomes. Over time, we’re building up a uniform applicant data set that can be leveraged across districts to get really granular conclusions about which teachers are effective in which contexts.

Rick: As you know, better than I do, predicting which teachers will be effective is one of the most difficult challenges in education. Lots of researchers have struggled to find meaningful predictors of teacher effectiveness. So how do you approach that challenge?

Lauren: Yes, you’re right that this is a tough problem, but it’s also one of the most worthwhile and exciting problems I can think of. Living in Silicon Valley, I see hundreds of companies focused on using data to predict the most mundane things—how we can shave three minutes off the morning commute or which Tinder photo will make someone more likely to swipe right. These are often very technically challenging problems to solve, but with the right resources behind them, data scientists and engineers have been able to gain insights that we wouldn’t have thought possible a few years ago. What we’re trying to do at Nimble is simply apply some of that same thinking to a problem that can impact the life trajectory of millions of students. Now, one of the challenges of historical research in this area has been the approach. Most studies on this topic have been between a single district and a researcher. The data set is limited to that district or, in the best case, a small group of districts with inconsistent application data, only some of which can be accurately exported from the applicant-tracking system. Often there is no applicant behavioral data to speak of and minimal standardized application content. If you try to build a predictive model on a data set like that—especially in the case of such a nuanced problem—of course you’re not going to have much luck. What you’ll see in some of the newer research on this topic is that where researchers have gotten creative and districts have worked hard to collect rich, standardized data over time, the results have been much more interesting.

Rick: What goes into evaluating if a candidate is a good fit for a particular district?

Lauren: To us, “fit” means two things: How successful will they be in moving student learning and how likely are they to be retained over time? So, our model takes both of those factors into account when looking for predictors. Right now, we have a model that’s relatively simple, but over time, we aim to make our conclusions more and more specific. We envision a future where we can identify which candidates are best-suited to a specific school culture, principal leadership style, student population, and so on. In order to get to that level of granularity, we’ll need a fairly large data set. So actually, this is a great referral incentive for our current partners, because the more districts that come on board the more powerful our predictive model becomes!

Rick: So do you have any data or anecdotes that speak to how effective your efforts have been thus far?

Lauren: We don’t yet have classroom-effectiveness outcomes from our most recent cohort of district hires, but I can say without a doubt that our districts are feeling the effect. One of my favorite examples is one of our larger districts out in California. Before implementing Nimble, they had this massive Google tracker with all of these tabs. One tab was their “high potential” tab, which was used to identify the prospects they wanted to pursue quickly. Another was their “closing file,” which was for those who had offers pending and still needed to be convinced to come to the district. And there were at least 10 other tabs, each with their own purpose. They hired a temporary employee during peak season to keep track of their web of spreadsheets and ensure they didn’t lose out on any strong candidates—which of course they still did. All of these workarounds simply because their system wasn’t working for them. They implemented Nimble last year, and I’m happy to say that tracker is now a thing of the past, and their team is focused on proactive recruitment and high-touch, personal interactions with candidates instead of navigating a sea of Google docs. Most importantly, they started school this year with almost 100 percent fill rates, which they definitely could not have said last year. We’re also undergoing a two-year efficacy study beginning this hiring season with several districts and charters. It will look at the efficacy both of the predictive model and of the workflow-management tools in their ability to attract and hire high-performing teachers. So stay tuned! 

Rick: What are a couple of risks or downsides associated with the work of Nimble? For instance, are there valid concerns that relying upon a “fit” measure might unintentionally lead districts to narrow the diversity of their teacher workforce in problematic ways, whether in terms of experience, temperament, race, or anything else?

Lauren: Absolutely. This is a critical point. There’s plenty of research to support the assertion that diversity in the teacher workforce is important to student outcomes. Obviously, racial and gender diversity are two dimensions we struggle with as a field, and it goes beyond that, as you note in your question. We’re still in the early stages here, but we do look closely at each factor in our model to tease out interactions with race and gender, and we collect voluntary candidate-reported data that allows us to check the implications of our model in real-time to avoid pitfalls. And the outcome variable matters a lot, too. In fields where success is measured by something highly subjective like supervisor evaluations, the danger of creating a predictive model that simply amplifies human bias is huge. We use a value-added model of student growth to evaluate the effectiveness outcome, so I think that helps us to some degree. Ultimately, this is a question that we will never consider fully resolved—it’s something that we’ll continue to revisit over time.

Rick: Another of your goals in streamlining the hiring process is to save time for HR teams. Can you offer some illustrations or data that provide a sense of how successful you’ve been?

Lauren: Yes, we do look at efficiency as one lever for impact, with the thinking being that if we can make our districts more efficient in the hiring process, they’re more likely to attract the high achievers once we’ve helped identify them. The Missed Opportunities study gives us some reason to believe this is true. Our preliminary results have shown that our clients have decreased processing time by 20 percent and increased yield on offers by 15 percent. We’ve also seen jumps in principal engagement with the centralized hiring process, which shows that they’re getting buy-in from school leaders.

Rick: For districts, what are the costs of using the platform?

Lauren: Actually, for districts, there’s little or no incremental cost to implementing Nimble. We’re quite price-competitive with the applicant-tracking systems they’re already using—which tend to be pretty antiquated and lack the predictive capabilities we bring to the table. Using Nimble actually saves districts quite a bit of time on some of the processes that are manual workarounds for them today. One quick example—lots of districts use a whiteboard or spreadsheet to track their fill rates, progress to filling vacancies, over time. Nimble actually has a built-in vacancy dashboard that does this for them. For me at DCPS, that process used to take up five to 10 hours per week, easily. There are certainly a few weeks of change management and training on the front end, but I typically find that districts far overestimate the burden of switching to a new applicant-tracking system. I’d say Nimble pays for itself in time savings over the first two months of the very first hiring season.

Rick: Since you’ve launched Nimble, what’s been the biggest challenge you’ve encountered?

Lauren: I honestly think selling Nimble to districts is probably a lot like what selling an iPhone was like back when half of us were using Blackberries. We thought, “Well I can already send emails and make phone calls—what’s really the difference?” My point is that you can’t really fully understand the benefits of a clean user interface and more efficient workflows until you’ve experienced them firsthand. And, when it comes to predictive insights, districts have been promised this before with little success, so I think they’re a bit jaded. Convincing them that Nimble is taking a novel approach to solving the full suite of challenges around hiring is difficult. The good thing is that word of mouth catches on quickly in education, and we have some customers who really love us, so it’s just a matter of time!

Rick: What’s the best advice you’d offer to someone who’s thinking about launching an education startup?

Lauren: Do it! One of the reasons districts aren’t sold yet on the benefits of novel approaches and user-friendly tools is because there are too few companies and founders willing to inject a bit of healthy competition into this space. Products should have to be outstanding to get districts to use them, but unfortunately there are too many complacent incumbent companies and products out there today. That will only change if more founders and companies take the leap into this important work!

This interview has been condensed and edited for clarity.


Source link