*I put a disclosure at the bottom of all of these notes, but in this case, I would like to move the acknowledgement of AI involvement in things like the organization and conceptual scaffolding of this note while also validating that this is my own intellectual work.
Given that my chosen field of study includes the letters AI, I tend to be a bit more conscientious about documenting the evolution of my ideas. (I mean, that's also what the research is about, but this time I'm talking about from a "these thoughts are mine" standpoint.
Over the past few years, I've tried to become more careful about keeping evidence that my ideas are mine fairly retrievable. For example, for my dissertation I save old documents that show I was thinking about certain questions before generative AI was even publicly available. This includes my grad school application, early copies of my proposal, papers and notes from early in my cohort, and even my external funding request sent to my employer in 2019. When I'm starting to articulate something that is feeling new, I try to explicitly mark it so I can reference that moment later. I retain conversations that show how ideas developed, and I now compulsively export my various LLM transcripts like I eventually learned to do with saving in Word. Some of this is totally reasonable, but some of it is also starting to feel like I need to be perpetually ready to prove that my thinking happens in isolation. Which is especially difficult because it doesn't. No one's does. Even without various forms of AI.
I'm finding that the concerns I see about using genAI in research and writing will often look at the final artifact, questioning if AI literally produced any of the words, if so, how many, and could the researcher have still created this paper without it? Important questions, but only one, perhaps more easily measured, part of what happens when humans use AI as part of their intellectual work.
When done well, or even sort of well, I guess, research and writing will generally change the person doing them. Often, this implies that they will struggle through a problem (thank you for the great framing Rohit, Susannah, Kimberly, Sara), read a bunch of new sources, notice contradictions, organize evidence, revise explanations, and respond to criticism. I'm told it's through all that super appreciated feedback that we develop our research and writing skills and learn to form, much less articulate, our own arguments. Even when two people produce fairly equivalent papers in terms of output, they still might emerge from it with varied ability to: explain the findings, recognize their limitations, answer relevant questions, or extend the work.
And I absolutely agree that there are concerns around AI replacing intellectual effort and that we should be taking them seriously. I've seen the competent output, and understand that can happen even without the work that would usually generate that output. (Here I'm talking about things like critical thinking, reading comprehension, pattern recognition, synthesis, and judgment. This just once again happens to align perfectly with my often-stated stance that we shouldn't only evaluate an interaction by the quality of the output. In this case, it's because it can entirely possibly conceal meaningful differences between the human's pre- and post-AI interaction states. I say all this to say, the question of authorship seems to have become even more complicated.
As far as I know, academic authorship hasn't required the absence of outside influence unless specifically indicated as part of some assignment. The researchers I've seen work develop ideas through collaboration, editors, articles, books, peer reviews, advisers, conferences, conversations, participating in their field, and lots of use of the suggesting mode in Google Docs. For example, one researcher suggests an analysis, another offers a different option. (This is especially easy to see in interdisciplinary work.) Depending on what field they're in, what collaboration expectations were established, and the actual contribution, someone may show up in an acknowledgment or in the citation, or maybe even not at all. This was already so straightforward that we developed literal taxonomies to help researchers determine who gets credit and what kind.
Here's where genAI complicates the already existing confusion. It can do several kinds of work at once, including: generate language, summarize, propose structure, question assumptions, suggest interpretations, and even help the researcher develop an incomplete initial thought. Several of those sound like what we associate with using editors or research support and others sound like intellectual contributions of colleagues and collaborators. And some are more related to the type of sustained interaction you have with a system that can respond immediately and doesn't mind answering the question 15,000 different ways as you develop your thought. Did AI add any sentences isn't really accounting for all that.
Obviously, this is pretty important to me. I do the academic writing that I say is mine, but I also use AI extensively to "talk" through ideas, challenge my reasoning, organize my thoughts and material, and even to help me determine what I actually think about something. (That is not to say by it declaring, but rather that I develop my thinking through sustained interaction, questions, evaluation, and revision.) What I'm writing now is mine, but yes, the trajectory it took to get here has been influenced by interaction with AI (oh, and also discussion with Rohit, Susannah, Kimberly, and Sara around this article: "The Only Reason You'll Ever Need Not to Write with AI".)
My thinking has always been influenced by other people, articles, technology, institutions, and experience. To me, that is an indication that the presence of influence shouldn't be the only measure to use to determine authorship. It might be more useful to question what kind of influence occurred, how much decision-making the human retained over what influences they engaged with, and what evaluations the human did to discern the involvement of the influence in the argument. Here's where I try to be particularly detailed in showing that I evaluate the system's suggestions repeatedly, evaluate and correct errors, revise the framing as I introduce new ideas and understanding, and can explain and defend the ideas within my work. Still, a lot of that interaction could possibly be closer to, for lack of a better term, collaborative generation.
We've all seen tools intended to identify AI-generated content, and if they become sufficiently valid, they will probably also become valuable for understanding how the deliverable was made.
But even identifying that some of the text was likely generated by AI is not the same as understanding where the idea originated. It also doesn't capture what the human learned, or how their subsequent thinking was impacted. A reframing that emerges from a long exchange, conceptual changes, and questions that made me rethink my research questions are influences that are harder to identify. Given enough interaction, the influence almost certainly will persist in some form even after the generated contribution has disappeared from the language.
So now we have two problems. 1. The narrow focus on literal text output might be an understatement of the role of the AI. 2. If generated language shows up, but the human has substantially evaluated, revised, added context, and determined what is included and how, there might be an overstatement of AI's authorship. The final product does reflect the process, but the process certainly isn't fully represented in the output.
So, some important questions I'm seeing in this area move beyond what credit goes where. I'm super interested in understanding what kind of thinking and thinker the interaction is helping to create. For example, the use of AI might have reframed the problem, inspired a stronger explanation of their own reasoning, expanded their ability to identify patterns, or helped them better articulate ideas. I'm not using these examples in the sense of substitution of real skills the researcher needs to employ, but as possible contributions that might have arisen through the interaction.
I won't even get into the idea that it could be both increasing some and weakening different skills within the same interaction.
All I'm saying here is that I'm not really in favor of an overarching claim that AI replaces thought, or that it augments thought. Did it influence the researcher? Almost certainly. But authorship isn't just the absence of influence. I'd argue that a more complete understanding considers the human's evaluation and judgment in helping determine the level and form of that influence across the interaction.
BTW, obviously I'm fully in support of disclosing AI use. I also understand that it is both important and exceedingly difficult to understand how meaningfully the human's evaluation, judgment, and synthesis impacted the final outcome. I'm just saying that if all we look at is the output, it's possible we're missing the development and understanding that evolve through the process. We might also miss when that development and understanding didn't actually occur.
Sarah, remember: this is allowed to be unfinished. If it were finished, it wouldn’t be a Research Note.
As always, these notes reflect my own intellectual work. AI was used for organization and conceptual scaffolding, both as support of and as an object of iterative inquiry into the creative process and cognition.
(Maybe a little convenient this time, given the subject.)