Jordan Lawson brings the statistical and the personal to City Connects

Jordan Lawson thought he was going to be a clinical psychologist.

But after learning more about the field, he discovered psychometrics – and the idea of using statistics for social good.

“Statistics is interesting in and of itself, but sometimes, people, myself included, can lose sight of the fact that it’s just a tool that could be used for good,” Lawson says.

Now he’s at Boston College’s Lynch School of Education and Human Development where he is a doctoral student in the Measurement, Evaluation, Statistics, and Assessment Department and a research associate who is helping to address one of City Connects’ data analysis challenges.

As we’ve blogged before, there is no way to do a classic randomized study of City Connects. Because the model is a whole-school intervention, there is no control group of students in the same setting who do not receive City Connects services. And schools are not randomly assigned to receive City Connects services. So there is no random selection of who gets City Connects and who doesn’t.

That’s why researchers use other approaches. For Lawson and the research team he’s a part of, this means using data from the system that Boston uses to assign children to the city’s schools.

Boston’s enrollment lottery uses a matching algorithm that considers a number of factors, including family choice, how close students live to schools, and whether a sibling attends a school.

The lottery also has a random component: it randomly assigns numbers to each student in the lottery. Whenever the other algorithm factors result in school assignment ties, the lottery gives preference to the student with the best lottery number.

“We capitalize on that random assignment to create a treatment and control group for City Connects and non-City Connects,” Lawson says, “and then we estimate treatment effects for students facing the same risk of assignment to see how effective City Connects is in improving academic and non-cognitive outcomes.” This approach uses lottery data to very closely approximate a randomized control trial.

The result is a randomized group of 3,300 students who are the core of a four-year research study funded by the U.S. Department of Education’s Institute of Education Sciences. A third of these students were randomly placed in a City Connects school and the other two-thirds were assigned to a non-City Connects school.

“These students should be equal on all observable and unobservable factors. So the only thing attributable to any difference in outcomes is the treatment, City Connects.”

Boston shares its data, providing anonymous student information on grades and standardized test scores, as well as work habits, effort, behavior, graduation status, and college enrollment.

Lawson and his team are also looking at the impact of City Connects on English language learners, students of color, and students from socio-economically disadvantaged backgrounds.

The research is underway, but there’s one thing Lawson has already learned:

“Research is a highly collaborative effort. It’s simply not as easy as sitting behind a computer and crunching numbers all day.”

Lawson works with an interdisciplinary team that includes research associates and graduate students. The team meets with an advisory board of faculty members from Boston College and other institutions. One advisory board member is Parag Pathak, a Massachusetts Institute of Technology economics professor, who developed the approach of using randomized school lottery information to evaluate educational outcomes. Last year, Pathak won a John Bates Clark medal. An annual award given to economists who are younger than 40, the medal is considered to be only slightly less prestigious than the Nobel prize.

Getting the quantitative and qualitative approach right is crucial, Lawson says, because, “Numbers are nice, but they don’t get at the whole story.”

For Lawson, who is the single father of a 12-year-old son, there’s also a personal element.

“The reason I resonate with the work we do at City Connects is because I also consider my son to be a City Connects kid.”

Lawson’s son did not attend a City Connects school, but he is a “real world example” of how services can make a difference for a student who has lived through family challenges.

“My interest in applying statistical methodology to examine things of substantive interest that could impact lives aligns really well with what we’re doing at City Connects. We use the statistical results to make informed decisions about things that have real life impact on people, especially people from more disadvantaged backgrounds.”