I became really attached to an implementation. Not just the idea, but the fully formed study.
I spent months refining it. I worked through logistics, software, measures, analyses, edge cases, participant experience. I engineered an entire tool with embedded AI that recorded transcripts. As a social scientist.
Eventually, a study stops feeling like a study and starts feeling like the study. That's where I found myself after my proposal defense.
About a week before, though, I woke up in a panic because "OMG, I'm not actually measuring what I say I'm measuring!" That is the best realization I could have possibly gone into a room of lifelong methodologists across 5 different fields whose entire job is to judge my work with, right?
Essentially I was told that the theoretical lens was still interesting, but if I want to complete my dissertation, I should strongly consider measuring what I was saying I was measuring. Which, to be honest, seems like a reasonable criticism.
It sounds kind of technical. But it wasn't. It sounds like it would be devastating. But it wasn't. I mean, it definitely caused some anxiety, but luckily I have a supportive committee who focused on the "good theory" part more than the "figure out your methods" part. My committee almost immediately said "We're not saying you have to completely start over, we're saying use the data you have as another pilot." Helped keep me from getting defensive as I was told that I will get to plan, create, and implement a new study. It worked, because I immediately looked at Dr. Reiter-Palmon and sarcastically said "What do you know about measuring problem construction?" Which could have gone in a much worse direction if it wasn't clear that I was on board with the critique.
It wasn't that the study was poorly designed. It's just that I'd become pretty good at answering a different question than my own.
I appreciate that this is a very particular kind of feedback. In a way, it's also a surprisingly optimistic kind of criticism. Nobody questioned the theory or whether the question was worth asking. If anything, the criticism was almost the opposite:
This is an interesting way of thinking about the problem, but the measures definitely miss the phenomenon you're describing. Definitely more encouraging. Also the start of a whole lot of rework.
I wanted to defend the study because I'd invested so much into building it. It was surprisingly hard to separate my attachment to the design from my commitment to the question. But I started to realize that I wasn't being asked to abandon the work; they were asking me to actually be more aligned to the work I wanted to be doing.
It's kind of a hidden part of research I'm learning about. You see a published paper, you read about a study that naturally emerges directly from good ideas. In reality, though, there's often a whole bunch of starts and stops, even some after you're already running the study. Maybe your implementation has quietly drifted away from the thing you actually want to understand. It feels like that isn't talked about quite as much outside of the school setting. Probably because by the time it reaches publication, all the moments like this had been carefully edited out.
So, I worked really hard to change how I think about the dissertation itself. Every revision initially felt a little existential. That's an unsustainable way to work. Instead, I framed my work in terms of its part in my larger research program. Now I'm not trying to execute a single perfect study. I'm trying to answer the same question, and the theory still supports it. I've just changed the implementation, which, given my panic a week earlier, I had to acknowledge was exactly what needed to happen.
When you're mid-dissertation, redesigning a study can feel like the end of the world. Or at least the end of any remaining energy I had. But if I'm serious about building a research program instead of just completing a single dissertation, this probably wasn't quite the defining moment if felt like. It's probably a relatively minor hiccup in a much longer trajectory. ... Which I will definitely believe a few years from now when I'm not right in the middle of it, I'm sure.
Sarah, remember: this is allowed to be unfinished. If it were finished, it wouldn’t be a Research Note.
As always, these notes reflect my own intellectual work. AI was used for organization and conceptual scaffolding, both as support of and as an object of iterative inquiry into the creative process and cognition.