Data Culture Group @ Northeastern University

Learn CARefully (keynote on teaching AI for data journalism at SciCAR)

Thu, 17 Sep 2026 00:00:00 +0000

The following is a write up of a keynote I was invited to give at SciCAR, the annual German gathering of scientists, journalists, and researchers. Consider it under-construction thoughts, aligned with the tradition of blogging to think out loud.

Everyday we see a new gaffe, book, lawsuit, or report about AI and journalism. In the practice of data journalism, like many others, this new technology is already influencing how we approach learning in newsrooms and classroom. I want to pause to reflect my own approaches and catalog some inspirational examples here to tease out how we can ensure any engagement with AI in data journalism work is in line with the fundamental principles that underlie the practice of being a journalist.

SciCAR, like NICAR and the Computation +Journalism Symposium, is one of the few places where data journalists, academics, scientists and others come together to discuss and develop our collaborative work. You’ll note that NICAR and SciCAR share a now-antiquated acronym: “Computer Assisted Reporting” (CAR). The term sounds a bit dated to the ear because there few areas of reporting that aren’t assisted by computers, but “CAR” reminds us that computing was once considered a radical and unfamiliar force in newsrooms. CAR came to popularity as a label for exploring the possibilities of how computers could help in reporting, and the ways it interfered with the regular practice of journalism.

I was honored to be invited to deliver this keynote talk at SciCAR in Dortumnd.

I think we need to return to that way of thinking in an era of chat-based large language models (LLMs), considering how and what we want computers to assist with. So CAR is the basis of my somewhat tongue-in-cheek reminder: we must teach and learn extra CAR-efully right now, committed to thoughtful consideration of how and where we use AI to assist our data reporting. See what I did there?

Some Context

Like so many of you I’ve been tracking developments in AI and journalism, helping launch a series of related news round-ups on Storybench that our Northeastern J-School grad student Vivica D’Souza continues to release every month or two. A core tension emerged for me from watching developments over the last two years: practicing journalists are both adapting and resisting. In parallel, universities that train new journalists are widely adopting AI, causing many student groups to organize and push back (arguing that “AI is adverse to the idea of learning”).

The flood of news updates about AI and journalism is almost impossible to keep up with. We’ve been writing round-ups on Storybench to highlight developments every month or two.

I interpret this resistance as centrally about power: who gets to decide how and for what these new tools are used in newsrooms and J-School classrooms? With this lens I’m immediately reminded of the Luddites of the early 1800s. Luddites weren’t opposed to the idea of industrialization, they were skilled workers fighting for the application of those new machines in ways that continued to produce high quality goods, were under the control of workers, and didn’t violate existing labor practices.

Does that list sound familiar? It sure does to me. Luddites come up often in media discussions about resistance to AI. In the field of data journalism, AI-driven work risks polluting product quality, is being forced into many learning settings, and is putting careers at risk. Yet practicing data journalists are rapidly starting to use AI to help develop stories, create interactive visuals, support investigative analysis, and more. There’s that central tension again.

So I choose to engage with AI technologies in my teaching and collaborations with learners of all types. I focus on thoughtful use of AI, critiquing misuse of AI, and principled refuse of AI. Data journalists have agency in how AI is engaged, or not engaged with, in their work. Let me run through a few central principles and help me reflect on how to do this well.

Be Specific In Your Learning

Once of my guiding principles here is to always be specific when I introduce AI for data journalism. AI companies market that their models can and will help with everything, but that’s mostly because they are unproven and untested; these are new tools that are looking for market validation and dominance. In that way they remind me of the introduction of the Apple Watch, which was pitched as a communication device (remember the idea of drawing hearts to send to your loved one?), a fashion accessory (remember the 24 karat gold watch?), a high-precision tool to tell time and a health and fitness tracker. A decade later it is clear that the last on those list is the market they found; it is primarily a health and fitness wearable.

AI hasn’t gotten there yet. We don’t really know yet which applications it will really help us with. This makes it hard to introduce in productive ways. Can AI help us brainstorm, automate workflows, edit our work, code for us, scrape datasets, make charts, add annotations, surface insights in CSVs? We don’t know yet. I find that being specific about those as separate potential capabilities helps significantly.

We’re beginning to see examples of recurring concrete use-cases for AI in data journalism, which I classify into buckets of data production, data shaping, data analysis, and data representation. One example on the data shaping side is CNN’s recent work identifying a pattern of Trump promoting companies on his Truth Social blog days after buying their stocks. Reporter Isabelle Chapman shared how they “used artificial intelligence to compare a database of Trump’s Truth Social posts with a full list of stock trades released in his annual financial disclosure”. I imagine using AI to link between the two datasets helped them with scale and fuzzy matching logic. These two tasks are well-known challenges in investigative data work, so they offer very specific (and verifiable) opportunities to leverage AI.

Another example, from the realm of data representation, is using AI to write the code that renders complex and novel interactive graphics. This past spring graphics developer (and Northeastern J-School grad) Yuriko Schumacher shared with my Data Storytelling class how she’s using AI coding in her work at the Athletic. Code generation has helped their teams iterate faster, robustly support desktop and mobile, and focus on interactive design so their readers are informed and engaged with complex visualizations that represent large sports-related datasets.

I point to these kinds of examples in my teaching to support discussion about how AI might help or hurt our ability to perform specific parts of our data storytelling. We don’t discuss “AI” as an abstract; we instead focus on concrete applications of chat-based LLMs within a list of concrete data journalism tasks.

Learn by Trying Things Out

Specific examples help push against the pervasive industry hype about AI “changing everything,” offering a path to concrete things learners can try out. Scholars from various domains have led research into how organizations working for the public good often repurpose technologies, putting them into service for purposes other than those they were designed for (including in journalism). Over and over this research shows that the process of adapting technological tools begins by simply trying something out to see if it helps.

One example from my own teaching is related to infographics. Early in my Data Storytelling course I ask students to write a short story about a small dataset with two standard charts. After this I have them put a one paragraph narrative summary, alongside the dataset, into a nano-banana model (via Gemini), asking it to turn their story into an infographic. The results are mixed and often hilarious, prompting opportunities for visual critique and sometimes sparks of symbols that could be repurposed. Most of the graphics produced last year were terrible but the point is that we practiced assessing some hype: nano-banana is supposed to be great at infographics, but the results we got by trying it out quickly were unimpressive. That’s important, and builds confidence in students to try things out and make their own judgments rather than trusting the marketing hype.

Examples of infographics created by my students using Gemini in spring of 2026.

Trying things out like this sounds great, but we have to consider the costs incurred in the playgrounds we create for ourselves and others. Legitimate and real concerns about water use, energy supply costs, training data copyright theft, big tech motivations, increasing costs, the spread of “slop” AI generated content, out-of-date models, and impact on cognition are all raised regularly and written about in depth. These can’t be ignored and have to continually inform how much we’re willing to try out AI tools that have so many negative impacts. I don’t have an easy answer here, so instead I engage with my peers and students to see how they feel vis-a-vis these concerns after trying out an AI tool.

Shaw & Nave in their oft-linked “cognitive surrender” paper found that people using AI to assist a task just kind of did what it told them.

However, I do think trying to generate code is a distinctly specific task within that space. Yes, I know and have seen the mixed results from research on learning to code with AI, but put that aside for a minute. My concerns about those costs I listed above are reduced in the case of AI for coding for a few reasons. First, much of the training material was obtained legally, scraped from online open source repositories with licenses. Second, data representation or analysis in journalism contexts is short-lived, with a ship-date after which code seldom needs to be revisited so technical debt accrued doesn’t matter that much. Third, local models are getting better quickly, meaning that cloud costs and corporate control aren’t as relevant. Finally, code is verifiable. You can see what the AI model produced, and the code you run is deterministic, unlike the usual probabilistic outputs of an LLM. Generating code to help in your data journalism feels like a specific use of AI that stands distinct from many others and I have far fewer concerns about it. To be clear, that’s only when you consider learning data journalism as the goal, not if your goal is learning to code; that’s an entirely separate discussion.

The Generative AI in the Newsroom blog from Diakopolous and Gilert at Northwestern University is a great live reflection of what the industry is trying out Regular posts from global newsrooms catalog the results of experimentation with AI; I read every single one. Their recent Agentic AI Investigative Challenge dove even deeper; the winners all showcased prototype investigative AI tools that centered provenance and reliability. These journalists are all trying things out and reflecting out loud about how it went.

Getting your hands dirty with tools, like authors for that blog do, is central to understanding their capabilities. I ask learners I work with to try things out because that is the best way to understand the opportunities and limitations of AI in data journalism.

Center Journalistic Principles

ProPublica’s AI principles page offer that they “treat AI-generated material as unverified source material.” Accountability , verification, transparency, integrity, privacy & security–their list is a pretty good set of core principles to guide your journalism. Sadly we have a growing number of AI gaffes in the daily operation of normal journalism practice that we can point to as the opposite, from the US state dept generating an incorrect map of Africa, to the New York Times correcting that a statement was an AI summary and not a quote of the Canadian opposition leader.

I take inspiration from well-written AI policy transparency like ProPublica’s, especially when it the ideas are so well-aligned with journalistic principles.

Readers see this. A study from Trusting News found that readers across 10 global news organizations overwhelmingly think AI disclosures are “very important” or “extremely important.” However, other research found that while readers value these disclosures, they still left the impression that AI connected journalism was less trustworthy. I have my students all turn in statements based on guidance from Trusting News’ “AI Trust Kit”. Writing these considered AI disclosures builds a strong reflective practice and reminds them that these core principles must guide all our work, no matter if it uses AI or not.

But how we develop these AI guidelines is also important. Most organizations start by looking at what AI is being used for and then consider what rules they need for those situations. This kind of approach, argues researcher Daniella DiPaola, is reactive and offers poor long term impact. Instead, she posits that we should start with what we care about most, and then develop AI policies that reinforce those core values. I find her values based approach to developing AI policies extremely helpful, and though it was developed for school systems I think it could be immensely valuable to newsrooms (and underlies how ProPublica presents their’s).

This kind of consideration of what principles guide engaging with AI in newsrooms came up as well in a recent Dutch thesis by van Bree. They describe how journalists engage AI tools in three ways: a colleague, an assistant, or an outsider. The colleague role cedes some authority and agency to AI while looking over it’s shoulder or helping it work. The assistant puts the human in charge of a collaboration with clearly defined tasks the AI would undertake. The outsider AI was seen primarily as a threat, devoid of verifiability and mostly needing attention as a subject of reporting. These three roles echoed my own experiences, where I move between considering what role the AI tools are playing. As noted, my CAReful approach means that I most often take the position that AI can be my assistant, following my directions and leaving the primary journalistic or design work of data journalism to me.

I appreciate van Bree’s naming of roles, and choose to highlight using AI as an assistant in my own learning and teaching. This is my own simple illustration.

Of course, this talk for SciCAR is in Germany, so I had to again laud the amazing Nazi party membership database project from Die Zeit as a well-considered inspiration in the investigative domain (which I admittedly know less well). They used AI up and down the chain of data processing for this deep investigative project. Throughout they offer strong examples of centering transparency, such as how they include an option for readers to offer a correction if some aspect of the processing of the hand-written cards led to an error or omission. Staff have shared with me that the story was a huge hit with readers, drove a major bump in subscribers and in turn created deeper investments within their newsrooms for AI investigative capabilities.

No matter what you learn or try with AI, centering journalistic principles is key. In the hype cycle and push to AI-ify everything we can’t lose sight of the core values that make data journalism what it is.

Stay Critical

My last guiding principle responds directly to the marketing and hype cycle of AI tools: we have to keep asking critical questions. In keeping with the rest of this piece, here I have more questions to offer as insights rather than answers. Sorry!

Is Big Tech a reliable partner? Remember the failed promises of a “pivot to video”? Offers of money, attention, and more from Big Tech are best viewed with skepticism. AI tools don’t cite news sources that they were trained on. AI company leaders don’t seem to view the misinformation and news-slop problem as critical. Our business models are not aligned. The recent history of journalism and Big Tech should make you constantly question if they are a reliable partner or provider of services.

Can we create alternatives? The AI moat might seem to large to cross, but if we can’t trust the California-centric monopoly perhaps we need to try. Model development is becoming more well understood, leading to public LLMs from countries from Switzerland, India, Poland, Spain and beyond. Even Reuters is getting in on the LLM-model building game, releasing their own “Thomson” model earlier this year. There are opportunities for custom AI work on specific tasks related to data journalism as well, as evidenced by the work Zetland did to build Good Tape, a revenue-generating transcription service now used by millions. Newsrooms and data journalism teams have long been an innovator in technologies (d3.js, Django, Svelte, etc.), A recent workshop I joined, led by my friend Harini Suresh at Brown, on “Collective Governance of AI in Newsrooms” left me hopeful that this energy is being directed at creating industry-led alternatives.

What of value is lost? Even when automation helps with tasks such as transcription, we have to consider the cost. Older journalists around me agree that the nuance of a speaker’s voice and repeated listening to a quote over and over build a subtle and deep relationship with a source when transcribing, one that is often critical to telling the right story. There are clear advantages to automated transcription, yet we also must introduce the loss of that relationship as a tradeoff to learners. In class I simply ask students to read a sentence, and then listen to the same thing read aloud from the source; the difference is obvious. This question is important to keep asking ourselves. For instance, what of value is lost when a robot is writing the d3.js code for your next interactive data visual?

Is AI good for learning? There is still much to see, but early evidence suggests that using AI can help in general assignments like homework assignments, but hurts in longer term learning like on exams. As I’ve already noted that I see a real gap in learning to code with AI; students who have a certain level of knowledge already can use and learn with AI coding tools quite quickly, but those who haven’t met that threshold learn significantly slower. Frank Elavsky writes about the misperception of how AI might help us wonderfully, nothing while we want AI to fill in skills needed to implement our wonderful ideas, in fact it is often used to produce high quality technical projects for poorly refined ideas. Conceptualizing and building are different and intertwined processes, requiring reflective use of tools to avoid the “slop-zone” he points to.

These four questions are the type of things I keep asking myself, and others I learn with, in regards to how AI might be used in data journalism. Asking ourselves hard questions is how we stay critical in this time.

Reflecting on Knowing

If you were expecting me to tell you to make charts with AI, or to not make chart with AI… sorry! These reflections are about the learning of data journalism in an AI era. If you know my background this won’t be surprising; I studied learning and computation for my M.S. at the Lifelong Kindergarten group at the MIT Media Lab. Yes, I went to kindergarten twice! The approach to designing creative learning experiences I learned there continues to inform all my teaching and research. More deeply, the idea of thinking about thinking continues to lead me to these kinds of moments of reflection.

It turns out that ability I was introduced to years ago is critical right now, because AI tools are changing how we think, learn and know things. The relevant academic term for this is “epistemology”: theories of how we know things. It’s a little meta but important to consider. Consider the simplistic example of ways you could know a soup is too salty: measuring the amount from bowl to compare to the recipe, tasting it yourself, hearing a trusted friend tell you, etc. I’m convinced that using AI tools to learn something changes how to you know it, though I think we don’t quite understand precisely how yet.

In older work I cited Heron’s concept of “extended epistemology” to highlight and explain that we have different ways of knowing things. I argue data journalist has a lot to learn from this.

This idea of how we know things matters in data journalism. In a talk for AoIR I argued that the off-screen arts-based approaches I work on in community data projects are a form of “epistemological resistance”, fighting the dominant formalized ways of knowing. This borrows from Turkle and Papert’s “epistemological pluralism” in software learning and theories of fighting colonialism from sociologist de Sousa Santos. AI changes how you think and learn in important ways, and you should be aware of that so you can resist it as needed. Reflection is an important part of learning, and one I’m trying ot practice right now with this piece.

Learn CAR-efully

So what does this mean for your learning and teaching? How do we apply my themes of being specific, trying things out, centering journalistic principles, and staying critical while AI technologies continue to be adapted into data journalism settings? Reflecting on their work in the Google / LSE JournalismAI initiative over the last few years, Schjøtt and Schaetz label what they saw as “domestication” of AI in newsrooms. They lay out four parallel paths the technology followed in newsrooms: demystifying AI to understand what it is, cultivating skills to use it, managing the limits, and appropriating it into practice. van Bree used the same term: “domestication.” If AI is a wild animal to tame in our data journalism, what methods are we to use?

I find those four paths of domestication useful to pause and consider for myself as a reflective learner. Sometimes you need to step away from the hustle and bustle of a tool and project to reconsider it from the ten-thousand foot level. In the end their report saw the processes led to both (1) increased confidence to try and engage with AI and (2) a growing sense of the inevitability of AI in the newsroom. There’s that tension again. That mixed outcome mirrors my own experiences on this space, and I’m sure will continue to complicate my under-construction approaches to teaching and learning AI in data journalism. Across those principles my key message at SciCAR is to learn CAR-efully. Hopefully the approaches and the examples I’ve shared inform and connect to your own experiences.

My closing side: learn CARefully.

Whose Quotes Get Cleaned Up?

Sat, 16 May 2026 15:00:00 +0000

In mid-February, The New York Times ignited a storm of criticism with a piece about Representative Alexandria Ocasio-Cortez at a security conference in Munich. The article was critical (the headline included the phrase “with some stumbles”) but the anger it provoked was caused by a verbatim quote that included repeated “ah”s, “um”s, and “uh”s. We all speak like that while thinking out loud, but even the Times’ own Guidelines on Integrity indicate that “The writer should, of course, omit extraneous syllables like ‘um’ and may judiciously delete false starts.” Was this sloppiness? A hit piece on AOC? Or was this a signal of something deeper about how mainstream news edits quotes? I dove in with some code, genAI, and theories to try and find out more about this kind of “quote cleaning.” What I found suggests that including “um” is incredibly rare and opens up deeper questions about how quotes operate in news to normalize some speakers and scrutinize others.

The quote as published by the New York Times

The Politics of Quotes in News

It’s typical, and even helpful, for journalists to tidy up a direct quote for better readability. We teach this in journalism schools and many media outlets codify this in editing policies (as those Times guidelines demonstrate). That said, the practice is largely invisible: I doubt most readers think about whether the words in quotes exactly match every sound the speaker uttered. This kind of quote cleaning is an editorial choice, and like all others, it embeds values. In this case those values are likely to reflect underlying ideas about what counts as articulate, authoritative speech. Who sounds smart? Who sounds right?

This isn’t a surprising comment in the context of our racialized politics. Black Speech and Spanglish are often used by political leaders in those communities, yet seem to show up less often in news quotes. Any multi-cultural person knows the norms of code switching in language to match the dominant audience expectations. A high-profile recent example is Kamala Harris, who was criticized as being inauthentic because she spoke using different vernaculars to differing audiences.

Consider a more contemporary counter-example: President Trump. His often-incoherent rambling statements have challenged editorial quoting norms (among so many others norms he’s challenged). A whole new term — “sane-washing” — has come into vogue as part of related media critiques. Responses have ranged from the NPR Public Editor’s explanatory engagement, to op-eds decrying the coverage, to criticism of how little what Trump actually says is quoted.

These anecdotes suggest a pattern where high-status white politicians often get their quotes cleaned up as a matter of course, but others who don’t belong to that dominant group might not receive the same treatment. AOC sits at the latter group, as a Latina progressive woman. This editing from the Times lands differently with that in mind.

Can Data Help Us Understand This?

To investigate this specific example, I decided to see how often “um” and “ah” were included in quotes in the most visited US online news sites on the same day that article was published (Feb 13, 2026). Conveniently, when we want to answer questions like these we have a wealth of technologies at hand to help. Specifically, with my colleagues Meg Heckman and Elisabeth Hadjis, I just published a paper in News Research Journal about using genAI prompts to extract quotes from news stories (something traditional machine learning doesn’t do well). I remixed methods from that work to answer this question about just how rare including those utterances was that day.

My analysis included 36,045 quotes across 8,066 articles published by 45 news sources on Feb 13th. From those quotes, just eight quotes included “um” — one was from AOC while the rest were non-politicians. That’s just 0.02%! Even fewer, just three stories included “ah”; none from politicians. I can confidently conclude that including “um” or “ah” in a quote from a high-profile politician on Feb 13th was a very statistically rare choice.

The eight quotes identified with “um” or “ah”

Of course, since I don’t have raw transcripts for all those stories I can’t tell you how many dialectical markers, code-switches, and instances of non-standard grammar were removed. I’d be curious to investigate this more for politicians who carry multiple identities, to see if there are keywords from a dialect indicative of code switching that I could track across how they’re quoted in news stories. Consider this an invitation to explore more: my little experiment suggests that we have the tools to answer these kinds of questions.

How’d I Do This?

Curious to try this kind of thing yourself? Here’s a bit more about how I found these results. I remixed our prior approach from that paper with a set of the most-visited online US news sources, the Media Cloud online news archive, a revised prompt for genAI, evaluation with the DSPy library, and a small set of duct-tape scripts to see what the data said about how normal — or not — that quote was.

I’m a hesitant generative AI user, so at each step where I used it I tried to be thoughtful about how to check the validity of the results it gave me. I call this my “genAI-last” approach: I try to use any other reasonable computational methods before falling back to genAI to help me solve some research analysis problem.

A diagram of the data pipeline used for this investigation.

I began by defining what media sources I cared about. The seed here was the Times, so I thought it made sense to focus on some reasonable set of large mainstream online news outlets. Media Cloud has a collection of “Most Visited” news websites from January 2026, based on SimilarWeb data as published monthly by the PressGazette. I can’t really speak to the full methods they use to collect their numbers, but SimilarWeb data is widely used and the list looked reasonable to me. This particular list from Media Cloud was already filtered for US-based publications, which worked well because I wanted to look at American editorial norms.

With that list of publications in hand, I used the Media Cloud web-based search tools and API to pull all stories from those websites on Feb 13, because I just wanted a one-day snapshot for this short blog post. Then I removed duplicate stories based on whether there was a story with the same title from the same outlet on that day, because often the same story is published at multiple URLs. That left me with 8,066 unique stories.

In parallel, I iterated on creating a genAI prompt and evaluation set to help me extract quotes from stories and attribute them to speakers. An earlier version of that News Research Journal work we presented at the 2024 Computation + Journalism Symposium gets more into details about why, but there isn’t really an off-the-shelf open solution for extracting and attributing quotes. In general for this kind of work with AI models it is important to (1) define your task, (2) build a ground-truth evaluation dataset, and (3) create an evaluation and scoring rubric. To start the genAI eval I pulled a handful of stories and identified quotes and speakers by hand; this would be my evaluation dataset to see how well different prompts and models worked. To test things I used DSPy, a great Python library for iterating on and evaluating genAI prompts. An existing account on the cloud-based Groq model provider let me run test prompts against this evaluation set with various genAI models. I iterated with Claude, ChatGPT, and Gemini on refining the prompt, and ended up seeing best performance on a prompt that Claude expanded for the llama-3.3-70b-versatile open-source model (recall and precision both over 0.95). If this was a more developed project, I would have evaluated against more hand-labelled ground truth data and evaluated multiple runs of the same data (because genAI models aren’t consistent, by design).

With a prompt and dataset in hand, I then just needed to run the prompt against each story. Here at Media Cloud we have an internal-for-now data pipeline tool called “Sous Chef” for this, so without too much effort I was able to get the results back as a CSV of quotes with related story metadata. I used some simple code in a Python notebook to extract and identify quotes with “um” or “ah” in them, using Spacy to tokenize the terms (see the code and data). For each of these stories I went back to read it to make sure the quote and speaker identified by the AI model were correct.

This process took me about a day of work, split across a week of divided attention. This array of tools really has put this kind of quick media investigation within reach and presents very exciting opportunities to learn about norms in the news.

So What?

Why does it matter if editors and journalists “clean up” speech like this? I spoke with long-time Boston Globe editor Mary Creane to get her take. Creane started by noting that in news it’s “not our job to make people look stupid; on the other hand our job is to present people as they are.” As a rule, she said she would “never change their quote.” Instead, if the choice was made to remove an “um” or “ah”, she would use ellipses to make clear that some words in the quote had been taken out.

Creane emphasized the difference between speaking well and making a good point. Former vice-presidential candidate and Alaska governor Sarah Palin, for instance, claimed to be a foreign policy expert because you can see Russia from Alaska, which Creane noted was “not ungrammatical, but points out a weakness.” In situations where the speaker you want to quote is speaking poorly but making a relevant point, an alternative is to paraphrase. In a very classic sense, Creane believes that as journalists “our job is to say ‘here’s all the information, make of it what you will.’” When it comes to editing quotes appropriately, the core question for her is “if we’re not fair, if we don’t have integrity, then who are we?”

For me, this integrity was lacking in how the Times quoted Ocasio-Cortez. The quote used was aligned with the critical tone of the article, but intentionally broke policy and norms to portray her as ill-equipped due to the form of her comment, not the substance.

Despite all the change driven by social media, online news is still a strong driver of public understanding and shapes perceptions of those in power. This example says something about what types of voices we amplify and showcase in society, and where and when everyone is allowed to speak as their true self. That’s important beyond just research, because it is a statement about inclusivity, which we should all be aware of. I think it’s good news that the tools we can use to study the influence of news at scale are finally within reach.

Originally posted to Storybench

Data Drums wins a Data Sonification Award

Wed, 29 Apr 2026 15:00:00 +0000

We’re thrilled to share that Data Drums has won a 2026 Data Sonification Award in the Arts category!

“Data Drums: A Rhythmic Call to Climate Action” is a live performance that brings global CO₂ emissions to life through the dynamic beats of a Brazilian-style drum ensemble. Drawing on carbon emissions data from Brazil, India, and the United States, the four-movement piece translates numbers into rhythm. At various points in the performance each drum and beat represent a different slice of the data; Volume, tempo, and pattern shift to reflect emissions levels and growth rates. Through drumming the contrasts and connections between countries becomes visceral and immediate. You feel the emissions. The piece culminates in an interactive finale where the audience joins in, embodying the collective responsibility and possibility for climate action.

The project was co-created with Marcus Santos and Lily Gabaree, and supported by the Northeastern CAMD Climate Change and Public Communication Impact Group and the Northeastern Center for Transformative Media.

We’re proud of how this project pushes what data storytelling can be — not a chart or a dashboard, but a room full of people making noise together about climate change. Get in touch if you’re interested in bringing Data Drums or other participatory data experiences to your community. You can read more the ideas behind about Data Drums in my Community Data book.

Beyond Accuracy: A Case Study of Sociotechnical ML Evaluation

Sun, 22 Feb 2026 15:00:00 +0000

The use of large language models and generative AI has significantly impacted computational social science, creating new capabilities and raising new questions for those using software to support social analysis. Key among those is how to evaluate if large models or prompt-based workflows are actually performing better than conventional machine learning. Against this backdrop, we want to propose a more sociotechnical approach to evaluation, one that includes contextual metrics and discussion beyond the standard recall and precision scores. This brief case study shares what happened when we tried this approach on a classifier we’ve built as part of the Counterdata Network’s collaboration with the nonprofit, US-based organization Pregnancy Justice.

Some Background

Building on software systems built as part of the Data Against Feminicide project, the Counterdata Network partners with groups tracking human rights violations via online news to partially automate some of the content discovery and evaluation. In practice, this looks like working together with organizations to train custom ML classifiers that score candidate news articles from online news archives (based on geographic filtering and on keyword matching) for relevance to the violations they want to track.

One of our more recent partnerships is Pregnancy Justice, a national legal advocacy organization that advances and defends the rights of pregnant people, no matter if they give birth, experience a pregnancy loss or have an abortion. Pregnancy Justice’s research team tracks pregnancy criminalization cases across the U.S. in collaboration with academic partners. These are cases charging pregnant people with crimes related to pregnancy, pregnancy loss, or birth. Their extensive and rigorous work is documenting the impact of the Dobbs supreme court ruling, and supporting people brought into the criminal system because of their status as pregnant. Our software system surfaces relevant news articles about cases that fit their focus each week.

The platform where the Pregnancy Justice team reviews retrieved articles and labels them as relevant or not relevant.

Improving the Model

After collecting and annotating a sample training set and experimenting with ML models, the first version of the model was deployed into the email alerts system to retrieve articles in November 2024. The Pregnancy Justice team reviewed retrieved articles each week and labeled each as Relevant / Not Relevant. When our team analyzed the results, we found the false positive rates were near 40%, meaning that every 2 out of 5 articles returned to the team was not relevant to their search.

From a purely technical standpoint, this was alarming. The model seemed barely better than random chance. As ML researchers, our instinct was to fix the model. We designed three parallel improvement experiments to reduce false positives and boost conventional ML performance metrics. What we learned through the process challenged our concepts about what better even means.

Experiment 1: Should we incorporate better data?

First, we updated our TF-IDF model using a subset of newly annotated data from PJ’s weekly reviews. After incorporating the new data, the model’s false positive rate decreased from 55% to 10%. Our analysis showed that much of the original misclassification was a result of domain-specific terms, such as “midwives,” which appeared frequently in real-world news cases but were absent from our initial manually sourced training set. By incorporating the team’s feedback, the dataset better matched the real-world linguistic context, allowing the simple model to more accurately generalize.

Experiment 2: How about more complicated models?

Our first model used TF-IDF, a method that identifies important words that are common in documents we care about but relatively rare in the entire collection. It’s a relatively simple approach by modern NLP standards, as it counts words rather than attempting to understand semantic meaning.

We hypothesized that more sophisticated embedding models might perform better. Unlike TF-IDF, large language models (LLMs) like BERT, Universal Sentence Encoder, and MPNet are designed to understand the semantic context—the meaning behind sentences. Theoretically, they should capture the nuanced relationships inherent in complex social issues like pregnancy criminalization.

We tested several state-of-the-art embedding models against our current TF-IDF baseline. However, given the ongoing debates about the ecological impacts of LLMs, we decided to evaluate them on more than just accuracy. We also evaluated their environmental and computational costs.

The Trade-off: Accuracy vs. Environmental Impact

The AI community is increasingly recognizing that model performance cannot be divorced from its environmental cost. Extensive research has emphasized the nontrivial energy consumption and associated CO2 emissions from training and deploying LLMs. This environmental footprint —through pollution, resource extraction, and climate change— disproportionately affects marginalized communities, many of the very communities Pregnancy Justice serves. As a result, our team was committed to prioritize environmental sustainability. With this in mind, we evaluated not just accuracy metrics but also running time and CO2 emissions (estimated using the codecarbon package).

Embedding Model	FPR	Time	CO2 Emissions
TF-IDF (w/ new data)	0.12	1x	<1 mg
MPNet	0.07	105x	>10 mg
USE	0.07	31x	>5 mg
Sentence-BERT	0.09	18x	>10 mg
mxbai-embed-large	0.23	353x	>100 mg

Model performance (FPR) and computational cost (time, CO₂ emissions) across embedding approaches. FPR = false positive rate (lower is better);

Our results show that while some embedding models achieved low false positive rates, the computational costs were staggering. The mxbai-embed-large model, for example, took 353 times longer to run and emitted significantly more carbon than our original model. The accuracy gains were small, especially compared to the retrained TF-IDF model, with substantial environmental costs. This matches findings from other research, where simpler models such as static embeddings have been shown to perform comparably on digital humanities tasks while requiring significantly less compute (Ehrmanntraut et al., 2021).

After discussing these results with the Pregnancy Justice team, we reached a critical conclusion that the marginal gain in accuracy does not justify the increased environmental and computational harm.

Experiment 3: How confident should the models be?

Our final experiment involved adjusting the model’s decision threshold. Our model outputs a probability score between 0 and 1 for each article, indicating the model’s confidence that the article is relevant. For the first model we used a threshold of 0.5: articles scoring above 0.5 were classified as relevant.

Histogram showing the distribution of scores on stories from the first model. Note the majority after under 0.5, our threshold for including in results sent to the team.

We hypothesized that If the threshold were raised, e.g. to 0.7 or 0.8, the model would reduce false positives by only returning articles it was very confident about. Our experiments confirmed this, but also showed a tradeoff: a stricter threshold means missing some truly relevant articles.

When we presented this tradeoff to the Pregnancy Justice team, they offered a surprising insight: they wanted the false positives.They explained that reviewing even the false positive articles, those not directly about pregnancy criminalization, was actually valuable. These articles helped them understand the broader landscape: adjacent policy discussions, related criminal justice issues, and the wider context in which pregnancy criminalization occurs. What we labeled as irrelevant from a technical standpoint was often relevant from a strategic advocacy standpoint.

This insight fundamentally reframed our evaluation. The model wasn’t just a filter to remove noise—it was a discovery tool that exposed the team to a broader information ecosystem. The “noise” contained signals we hadn’t anticipated.

Towards Sociotechnical and Community-Centered ML Evaluation

This case study highlights three critical lessons for evaluating ML systems with stakeholders and communities:

Bigger isn’t always better: Improving the training data might yield far significant improvements than increasing model complexity.
Let communities define success: Technical metrics like accuracy and false positive Rates are proxies for utility. What counts as correct output depends on contexts, workflows, and values that these metrics cannot capture. We recommend co-designing metrics and success criteria with the people who will use the system.
Evaluate the System, Not Just the Algorithm: Efficiency is an ethical metric. By quantifying the time and CO2 costs, we made a value judgment that aligned our technical infrastructure with our project’s justice-oriented values.

Data Journalism in Dark Times: notes from the C+J keynotes

Sun, 14 Dec 2025 15:00:00 +0000

I just returned from the Computation + Journalism Symposium in Miami, and I’m still processing the weight of what I heard. The four keynotes this year painted a sobering picture of data journalism operating under pressure—from authoritarian governments, algorithmic complexity, and information warfare. But they also offered glimpses of resilience and innovation that feel essential right now. Here are short summaries of each keynote, drafted with some AI assistance based on my live notes.

Me ready for Miami in December ☀️

When Data Journalism Isn’t Enough

Attila Bátorfy opened with 15 years of experience doing data journalism in Hungary under Viktor Orbán’s increasingly authoritarian government. His message was blunt: sophisticated data journalism hasn’t stopped democratic erosion. Despite producing groundbreaking investigations–including tracking private jets to predict and photograph secret meetings of oligarchs–the work had limited impact on political reality.

What struck me most was his catalog of how governments actively undermine data journalism: releasing low-resolution data, commodifying public information, creating datasets with bad methodology, and even sponsoring “independent” counter-data. His COVID dashboard became a trusted source (5 million uniques in 2 years, cited in dozens of scientific journals) precisely because government data was so unreliable. But he’s now moving away from data journalism, noting that audiences “don’t want facts” and that Hungarian newsrooms see it as expensive with bad ROI. The one exception? Sports and health data still cut through the noise. Attila’s upcoming work focuses on historical representatos of data, and I’m excited to learn from what he shares.

Data as Weapon and Shield

Kae Petrin’s talk on anti-LGBTQ+ data policies brought this theme of governmental manipulation home to the US context. The Trump administration’s systematic removal and alteration of CDC data—changing “gender” to “sex,” taking down youth risk behavior data, adding warnings that datasets are “threats to society”—isn’t just censorship. It’s erasure through datafication.

Petrin highlighted a crucial methodological problem that journalists fumbled: when the CDC improved its estimation methods for trans youth populations, the numbers roughly doubled. News coverage reported a “sharp rise” in trans youth, playing into “social contagion” narratives, when the real story was that better measurement revealing what was always there. Meanwhile, the demographic data showing far more trans young people than older adults raises haunting questions about where trans adults over 24 are. These are questions we can’t answer if the data disappears.

But Petrin also showed data’s dark side: data can make people legible and vulnerable.

User Agency vs. Algorithmic Power

Homa Hosseinmardi’s work offered a different kind of complexity, and reveals that the villain might not be who we think. Her research on YouTube radicalization used an innovative “counterfactual bots” method to separate user intention from algorithmic influence. The finding? Users who relied exclusively on YouTube’s recommender actually consumed less partisan content than those who actively searched and navigated themselves.

This doesn’t absolve platforms of responsibility, but it challenges the narrative of algorithms as “great radicalizers.” The real picture is messier: a small group of users consuming vast amounts of far-right content, driven largely by external links and their own preferences. When her team simulated users without agency (random video selection, first recommendation only, etc.), those bots were showng content that drifted towards the politcal center over time. The algorithm appears to reflect and amplify existing preferences rather than creating them from scratch.

The implications for journalism are significant: we may be overexamining algorithms while overlooking user agency and the broader information ecosystem that drives people to platforms in the first place.

Navigating Narrative Warfare

Yevheniia Drozdova closed with work from texty.org.ua, showing what data journalism looks like when your country is literally at war. Working from Kyiv, her team has moved beyond fact-checking individual claims to mapping entire narrative ecosystems. They train models to identify manipulation techniques, track coordinated bot networks across Telegram (where 50%+ of Ukrainians get news), and detect patterns in TikTok campaigns using shared filters and timing signatures.

Her team found 2,000 TikTok bot accounts and documented coordinated Telegram networks. They trained models to detect emotional manipulation techniques, finding that 90% of messages from Russian accounts use fear and doubt. But their work isn’t just technical detection, it’s about providing context before narratives go viral. As she put it: “Fast news only satisfies the hunger for information, while slow journalism addresses a different deficit—deficit of understanding.”

What Stays With Me

These four talks trace a dark arc: data journalism struggling against authoritarian data practices, vulnerable populations made more visible and endangered through datafication, algorithmic explanations that may miss human complexity, and full-scale information warfare requiring new methodological approaches.

But I’m also struck by the practitioners’ persistence. Bátorfy’s COVID dashboard becoming a trusted institution. Petrin tracking articles through the Trans News Initiative to understand coverage patterns. Hosseinmardi’s methodological innovation in creating counterfactual bots. Drozdova’s team prebunking narratives in real-time during a war.

These keynotes show various ways data journalism remains essential for documentation, for providing sources of truth, for understanding complex systems, and for serving communities when governments fail them. C+J’25 offered a critical moment of trans-national solidarity that I won’t soon forget.

Notes from MISI Summit on Building Trust in Climate and Science Journalism

Mon, 10 Nov 2025 15:00:00 +0000

I attended the inaugural Boston University Center for Media Innovation & Social Impact (MISI) Summit recently, where a diverse group of leaders came together to discuss the theme of “Communicating Climate.” Hosted by the Center’s Directory Eric Gordon, the event included sessions on related topics. Here’s a quick blog post about key points from one session that focused on how journalism, academia, and public media can restore trust in an era of economic instability and eroding credibility.

The start of the “Business of News” Panel

The Business of News: Audience Trust and the Future of Science and Climate Policy Journalism

This panel included Brian McGrory (BU Journalism Dept Chair, formed Boston Globe editor), Sue Robinson (Helen Firstbrook Franklin Professor of Journalism at the University of Wisconsin–Madison), and Dan Mauzy (WBUR’s Executive Editor for News).

It was moderated by Meghan Irons (investigative journalist and Professor of the Practice at Boston University), who opened with the fundamental question: how do we restore public trust when newsrooms face economic instability, eroding credibility, and an increasingly complex information landscape?

Local News, Attention, and Transparency

Dan Mauzy opened with an optimistic take: local news has a leg up compared to national outlets, and public media has an advantage among those familiar with it. He focused don transparency as a tool for building trust, particularly related to showing how journalism happens and showing your work. Dan noted a sobering statistic: “8 or 10 people have never spoken to a local reporter”—a problem that needs solving. In parallel, to ensure they are representative WBUR conducts audits on their source demographics, comparing them to local census data to ensure representative coverage.

For climate coverage specifically, the answer is making it local: to “literally bring it home to people.” Instead of abstract global warming statistics, WBUR focuses on local watering holes, jobs, heat pumps, energy bills, and solutions. Dan pointed to their Cape Cod plumbing contamination story about nitrates as an example of making climate impacts tangible. WBUR now has 3 reporters plus an editor dedicated to their climate team.

Brian McGrory acknowledged the widespread distrust across institutions, but noted an interesting pattern: crises drive attention. The Boston bombing, Trump’s first presidency, and COVID all led to subscription bumps for the Globe. Digital subscriptions actually doubled during the pandemic. “These are votes of confidence on things we do and how we do it,” Brian said. But he expressed uncertainty about how to reach people who aren’t actively seeking out journalism without alienating current loyal readers. His philosophy: “when readers see things they like, admire, trust, they will stay, otherwise they will leave.” The answer for him? “Focus on interesting, important, consequential stories and the business model will follow.”

Sue Robinson opened arguing that journalism education and newsrooms aren’t teaching key skillsets: radical transparency vis-à-vis audience engagement and restoring trust. She pointed to organizations modeling this approach: ProPublica, Seattle Times, WITF, Baltimore Banner. WITF’s 2024 election coverage particularly stood out to Sue as a model, with Q&As, pledges about misinformation and accuracy, and messaging about what’s happening behind the story.

She pushed back on Brian’s faith in the story, pointing out that “The audience doesn’t come to you—they’re being told to hate us.” Instead of waiting for audiences to find them, news organizations need to actively work on finding common values with communities.

What’s Actually Working

Sue shared how listening sessions can change hearts and minds of those reaaders who are being told to disengage from local news. Through the Trusting News project, she works with journalists who talked to people disengaged from news brands. The sessions were challening for journalists who had to listen with “devastating results that traumatized [them].” But here’s the remarkable part: after those conversations, over half of participants said it built confidence and one-third said they’d probably subscribe. The takeaway? “Bring people together around a shared topic of concern—listening and convening can change hearts and minds.”

Dan echoed this from WBUR’s experience hosting listening sessions on various topics, particularly in response to parachute journalism in the Merrimack Valley. It’s “enriching for reporters to hear what people think is overlooked, what makes them proud.” The challenge? How to scale these personal interactions.

Brian agreed that focus groups are great, creating opportunties where meeting the person behind the reporting can “change the tune of the haters”. He pushes to have journalists that get out into community regularly: “in local news we can do that.” Brian also emphasized inviting readers into the process itself—not just the meta aspects like ethics policies, but the exposing the actual work of reporting. When people understand the process, trust follows.

Sue highlighted that public media is out front on innovative approaches. She pointed to Lookout Santa Cruz as a leading example. They partner with influencers in the community itself to scale their reach, working with people who have huge groups of followers not already connected to their local media organizations. Lookout is even fundraising with non-profits together. We “need more creativity like this,” Sue urged.

Dan shared that WBUR is doing more vertical video now and reaching new audiences. They’ve done partnerships with trusted community members on niche topics, which has expanded reach and built trust. The catch? “Platforms are not our friends, so how to build loyal communities developed within those platforms is challenging.”

The Hard Questions

Are Some People Unreachable? Brian posed a difficult question about major outlets like the NYT, WSJ, and Globe: while subscribers are growing, they’re “not reaching a critical part of the country.” His question: “BUT are those people reachable, and what might be the sacrifice to reach them?”

The Talent Pipeline is Broken Sue ended on a sobering economic note: pay rates aren’t good enough, so students coming out can’t afford to go into traditional newsrooms. Instead they’re going to NGOs and corporate communications to exercise similar skills. This threatens the entire pipeline of journalism talent.

The Continually Shifting Media Ecosystem Dan raised another challenge for the future: “how to create loyalty in a fractured media space? Discovery is harder in an AI-mediated internet search world.” As platforms and AI change how people find information, news organizations need new strategies for building direct relationships with audiences.

I left with a sense of both sobering challenges and promising experiments. While there’s no silver bullet for restoring trust, the path forward Sue, Brian, and Dan pointed at focused on doubling down on transparency, community engagement, and creativity. I can get behind that.

The views out the new-ish BU Computer and Data Science building are alwasy amazing 🍁☀️💦

Notes from csv,conf,v9: Community-Centered Data Stories

Mon, 29 Sep 2025 15:00:00 +0000

The University of Bologna graciously hosted csv,conf,v9 in Palazzo Malvezzi. Accordion music drifted in through the windows while we gathered, echoing the mix of technical innovation and creative application that folks shared. Speakers from around the world shared how they’re using data not just to inform, but to transform communities. Here are gently editing notes from the sessions I attended (edited with some help from AI); any errors are probably my fault.

Photo of all the attendees at csv,conf,v9 (source: John Chodacki)

Filling Information Voids with Chatbots: Using LLMs on WhatsApp for News Access

Mathias Felipe (he/him), InfoAmazonia

In the Amazon forest, communities often face critical information voids around environmental issues, while relying heavily on WhatsApp as their main communication platform. This talk presents BOTO, an open-source chatbot developed by InfoAmazonia that delivers localized, thematic environmental information—such as deforestation alerts and wildfires—through WhatsApp. Powered by Large Language Models (LLMs) and designed with a low-bandwidth architecture, BOTO aims to make local news information more accessible in under-served areas. In this talk, we’ll discuss how BOTO was created with participatory design to empower communities, bridge information gaps, and put news directly in the hands of those who need it most.

Felipe showing what Boto looks like in action

The most striking thing about BOTO isn’t its technical architecture, others are also exploring WhatsApp Business API integration with RAG-based article search; it’s how InfoAmazonia approached the design process. Focus groups with different indigenous communities shaped every decision, from interface choices to content summaries. The result? A system where people can select their region and themes, then receive data summaries and links directly to their phones.

Felipe emphasized something crucial: their audience doesn’t have a problem with chatbots. The hesitation comes from outside assumptions about what “non-technical users” want. When you actually ask communities what they need, the solutions become clearer. BOTO soft-launched recently with official rollout planned for next month, sponsored by ICFJ. The real test will be sustained engagement, but the co-design foundation suggests they’re building something people actually want to use.

Blazing Fast Automagical Metadata

Joel Natividad

With the pervasiveness of low quality, low resolution metadata in existing Data Catalogs, and new metadata standards like DCAT3 and Croissant demanding even more FAIR metadata, how do you make it easier for Data Stewards to maintain High Quality Data Catalogs?

Natividad giving his talk in a somewhat distractingly ornate room at the palace

Natividad’s (from datHere) talk hit on every data steward’s nightmare: the metadata maintenance burden. New standards like DCAT3 and Croissant (for machine learning) demand even richer metadata: properties, data dictionaries, summary stats, frequency tables. The solution? Flip the workflow. Instead of forcing metadata entry before data upload, let people upload first and generate suggested metadata automatically. Their qsv and describegpt tools infer data dictionaries, descriptions, and 40+ summary stats. It creates a “data steward in-the-loop” approach—human oversight with automated assistance.

Most intriguingly, they’re moving toward “chat with your catalog”—custom chatbots that can surface relevant CKAN datasets when answering questions. It’s RAG applied to data discovery, potentially transforming how people find and use datasets that are published and publicly available.

Guillaume Plique (he/him), médialabSciencesPo, Paris

CSV is ubiquitous in social sciences and in the humanities. CSV data is indeed the perfect bridge between social scientists, accustomed to dealing with tabular data, and research engineers needing to process the same data. That is why SciencesPo’s médialab has been building many of its Open-Source tools around CSV files, from well-designed web apps such as Table2Net to convert tabular data into graph data, down to powerful CLI tools such as minet to collect data from the web or xan to process tabular data using constrained resources.

Plique, and the lab where he works, are _all_ about CSV

Plique delivered a love letter to CSV. His argument is compelling: CSV is affordable, understandable, and free. It functions as a bridge between researchers, students, and engineers. It doesn’t require complex compute resources and naturally encourages data sobriety. The médialab’s tool ecosystem is impressive—table2net for web-based CSV-to-graph conversion, takoyaki for clustering CSV data to find clerical errors, minet for CLI-based CSV processing, and xan for visualization. All built on the principle that tabular data is inherently more comprehensible than JSON hierarchies. His closing advice: “If you need random access, use SQLite. Otherwise, CSV.” It’s a reminder that sometimes the simplest tools are the most durable—you’ll still be able to open a CSV file in 50 years.

Keynote: Developing tech with community: the example of Open Data Editor

Sara Petti, Open Knowledge Foundation

Open Data Editor is a desktop application specifically designed to help people detect errors in tables. It has been developed in constant interaction with the community from a very early stage. These interactions helped us understand what was really helping the community and what not, and especially made us aware of how much the use of such a tool could actually be helpful in increasing data literacy.

Petti dove into how they built a data cleanup and analysis tool with partners

Petti’s keynote was a masterclass in community-centered design. OKFN initially tried to build “an app to do it all”—review data, prepare it, map it, write articles. The scope was overwhelming and misaligned with user needs. The breakthrough came through extended pilot testing with organizations like ACIJ in Argentina and StoryData in Spain. This led to dramatic simplification and focus. The second round of pilots with five different organizational types revealed something crucial: researchers worried about changing metadata, navigation and design were hugely impactful, and people don’t read documentation.

Key lessons that resonated: language matters enormously (“validation,” “execute” carry technical baggage), there’s only so much you can do to make highly technical tasks accessible, and many challenges are fundamentally non-technical. Most importantly—iterate based on feedback, keep things simple, and don’t be confused by the latest tech. The project pivoted from React to a more sustainable framework, emphasizing the importance of maintainable code over popular technologies.

Democratizing data: data literacy for community action

Emily Zoe Mann (she/her/hers), University of South Florida Libraries

This presentation will share the results of Data Literacy for Community Action, a year-long pilot in the community of St. Petersburg, Florida, USA that provided data literacy to non-profit groups and interested members of the community, with a focus on using local data sets and data related to social determinants of health.

Mann shared lessons from hands-on data workshops with library patrons

Mann’s project exemplifies academic-community collaboration done right. The partnership between University of South Florida, the public library system, and a philanthropic health equity organization created something none could achieve alone. The workshop structure was brilliantly simple: introduce a concept, share a local dataset demonstrating it, bring in a local person to discuss real-world application, then run group activities. Her red tide sonification example from South Florida shows how hyperlocal data becomes immediately relevant when paired with familiar environmental experiences.

One insight that stood out: attendees came with vastly different question types, from basic to nuanced. Mann’s library background prepared her for meeting people at all skill levels, but it reinforced how “data literacy” isn’t a single skill—it’s a spectrum of capabilities that communities develop over time. The flexibility lesson is crucial for similar pilots: stay responsive to what people actually need rather than predetermined curricula.

Keynote: Clara Jiménez & David Fernández Sancho

Maldita.es Foundation

Clara Jiménez Cruz is the co-founder and CEO of the Maldita.es Foundation, a leading organization in the fight against misinformation… David Fernández is the current CTO of the Maldita.es Foundation, where he leads the development of innovative digital tools to combat disinformation.

The team shared a timeline of work they’ve done

This keynote hit hardest, perhaps because of recent events. Jiménez and Fernández positioned disinformation not as isolated false information, but as structured narratives—emotionally engaging, rationally grounded, amplified by trusted figures, global in scope. Their Valencia floods case study was chilling: the same narrative emerging simultaneously from Russian sources, far-right groups, and anti-vaxxers, all attacking institutional trust. The message was consistent: “You can’t trust government, trust me.”

Their AI-assisted approach uses Wikidata entity extraction to cluster claims, with human-in-the-loop narrative association. They’re tracking 600+ claims monthly—a scale that demands automation while recognizing AI’s limitations in understanding social context.

The most powerful moment was Jiménez’s declaration: “data as a means of resistance.” In an environment where “the internet is becoming more dangerous on purpose,” systematic documentation and analysis of disinformation patterns becomes a form of civic defense.

Data without borders: Straight out of spreadsheets and into the streets

Michael Brenner (He/Him/His), Data4Change

At Data4Change, we work with communities around the world to explore how data can be collected, understood and shared in radically different ways. This talk takes you through a series of surprising, community-led projects that didn’t begin with data, but with burning questions our community partners wanted to find answers to.

Brenner’s examples were extraordinary—from addressing intimate partner violence in Uganda’s LGBTQ community, to flute music rendering Ethiopian census data audible to the blind. Each project started not with data, but with burning community questions. I mention many of them in my Community Data book.

The data murals project with Haki Data Lab in Mathare particularly resonated. Rather than imposing external visualization methods, they recognized murals as a standard local communication mode, then created input visualization murals with string and stencils. The “power portraits” methodology for the Equal Victims project was brilliant—co-created visual representations that function simultaneously as art and data points about lived experience, avoiding retraumatization while enabling systematic documentation.

Brenner’s language shift is important: moving from “data literate” toward “data curious, data critical, and data confident.” It reframes the relationship from deficit-based (lacking literacy) to asset-based (developing capabilities).

Keynote: Fifteen Years into the Open Data Movement

Giorgia Lodi & Andrea Borruso

Fifteen years into the open data movement, we take the stage as an unusual duo to reflect, provoke, and laugh about where we’ve landed. A dialogue, shaped by friendship, a few successes, many failures, and a bit of fun—think less keynote, more late-night talk show.

This closing keynote was part celebration, part wake-up call. Lodi and Borruso’s Italian open data examples were both hilarious and heartbreaking—ministry electric charging station data published with beautiful graphs but completely locked down, forcing community scraping to make it actually useful. Their analysis cuts deep: following the letter of the law isn’t enough when datasets lack description, remain readable only by specialists, or simply don’t exist (particularly around gender-based violence). Communities get pushed out by technicalities, formalism, and missing data.

A case study about electric car data exemplifies the problem—officially “open” data published as unusable pivot tables instead of raw, machine-readable records. When the ministry finally released proper data, it came with semicolon separators and complete messiness. But their vector geography data example shows what’s possible: despite delivery delays and nested zip file complexity, raw land ownership data triggered new software, plugins, and services precisely because it was genuinely machine-readable.

Their conclusion resonated throughout the conference: exploit technology to support data management, think about machine-to-machine AND machine-to-human AND machine-to-community interfaces, and recognize that technical solutions alone can’t address fundamentally social challenges.

Reflections: Data as Community Practice

Walking through Bologna after two days of talks, I kept thinking about the accordion player who inadvertantly soundtracked the opening day. There’s something fitting about that mix of structure and improvisation, tradition and innovation. These presentations collectively argue for a fundamental shift in how we approach data work in and with community. We must shift from extraction to engagement, from literacy deficits to community assets, from technical solutions to socio-technical systems. Whether it’s BOTO’s co-designed news delivery, Data4Change’s participatory data projects, or Open Data Editor’s community-driven development, the most compelling projects start with community questions rather than data capabilities.

I gave a keynote related to my _Community Data_ book (source: Marco Cortella)

The CSV theme threading through many talks isn’t just about file formats—it’s about accessibility, sustainability, and democratic participation in data practices. In a world increasingly dominated by complex AI systems and proprietary platforms, there’s something radical about insisting that the most important data work can happen with simple, open, maintainable tools. As disinformation accelerates and institutional trust erodes, these community-centered data practices feel more urgent than ever. The question isn’t just how we make data more accessible, but how we make data work serve community needs rather than institutional convenience.

ACH Keynote on "Civic Jugaad - Reclaiming Digital Innovation for Social Good"

Fri, 01 Aug 2025 15:00:00 +0000

The following is a condensed write-up of a keynote talk I gave at the ACH 2025 conference (Association for Computers and the Humanities)

I grew up as an 80s kid watching MacGyver use some duct-tape to save a city from destruction. Frequent visits to India shows me real-world street equivalents, where this kind of hacked-together solution has a name: jugaad (pronounced “jugaar”). Jugaad is the Indian culture of improvised problem solving. This can range from repurposing a motorcycle to run an irrigation pump to mashing an air-conditioning unit into a wall it was never designed for. Looking back, it is obvious that these experiences played not small part in my decision to study engineering in college.

Jugaad in action in India.

The term was briefly taken seized on by business innovators in the late 2010s, confusingly tied to both orientalism and politics of self-emancipation, and then faded again in the 2020s under criticism as “shortcuts”. However, I believe this jugaad mindset has something urgent for those of working in digital humanities, public interest tech, and journalism right now. Big Tech is consolidating and concentrating power and resources in the hands of the few, using “innovation” to describe tired attention-based monetization strategies. In parallel, the current U.S. regime is attacking universities and their funding streams. We live in disorienting times for those like, who grew up with relatively stable patterns of public investment in knowledge for the common good.

Local needs, contexts, and constraints drive jugaad-style innovation; those operating under harsh situations figure out a way to make things work. They connect people, amplify resistance, build with not for, and study systems of power. I want to tease those four points to structure my argument about the “civic jugaad” we need right now, bringing the improvisational, community-rooted ethos of jugaad into digital humanities and public interest technology practice.

1. Connect people

Facism and authoritarianism thrives on isolation and “othering.” One small counter-move is building tools that cross boundaries instead of reinforcing them.

Journalists are at the forefront here, constantly connecting their readers to each other via the stories they choose to highlight. A lightweight example from my own work on interactive journalism is Our Cup, a static website I rebuild for every World Cup that recommends matches to watch based on the largest local immigrant populations in your area, then connects you to food, music, and news from those communities. It’s genuinely a “quick and dirty” project, deployed as a static site on GitHub Pages. There’s little infrastructure and no grant funds, just an attempt to use a shared cultural moment to reduce distance between neighbors.

El Surtidor built a chatbot letting readers interact wth a women in prinson.

A more inspiring and ambitious example comes from El Surtidor, a Paraguayan news outlet that built a chatbot named Eva out of hours of interviews with a woman awaiting sentencing for international drug trafficking. With Eva they are trying to open up public conversation on a stigmatized issue: women incarcerated for smuggling. It has logged more than 15,000 interactions with the public, all built on a modest $5,000 grant. The team even considered a WhatsApp bot before landing on something cheaper to build and maintain. That’s the jugaad approach in action: focus on the local context, and just make it work with what you have.

The key lessons for practicing civic jugaad to connect people?

Design for local contexts, where the problems at hand and resources are well understood.
Make it work – if you have multiple approaches, just use the least complex options.

2. Amplify resistance

There are groups around you already fighting against marginalization they experience. You can innovate with groups in neglected areas to create higher potential for social impact.

Some of the most important public interest technology happens in areas mainstream institutions have simply neglected. My good friend and colleague Catherine D’Ignazio’s research with anti-feminicide activist groups is a good example. Official records systematically undercount gender-related killings of women and girls, so activist groups around the world build their own “counterdata” to fill the gap. Our Counterdata Network grew directly out of watching that struggle: we build bespoke, lightweight machine-learning classifiers, in consultation and collaboration with each group, that score and cluster news stories and deliver alerts by email or web dashboard. It’s essentially a custom-built replacement for Google News Alerts, tuned to each community’s specific needs. Over the last five years almost 50 groups have used it regularly to monitor human rights and civil rights violations.

System diagram and screenshot of Data Against Feminicides tool created by the Counterdata Network.

A smaller-scale version of the same instinct is a local protest mapping tool I built so newsrooms can see and embed maps of nearby demonstrations in their coverage. Many are criticizing the coverage of mass protests in response to Trump’s destruction of long-standing government programs as underwhelming. This website helps local journalists create compelling visuals to discuss incidents and patterns of resistance, perhaps pushing back on episodic vs. systemic framing issues so often encountered in this type of reporting.

The key lessons for practicing civic jugaad to amplify resistance?

Rely on and respond to problem statements the local community has developed.
Build by putting together existing pieces.

3. Build with, not for

Digital tools for the public good can’t be built with standard deploy-oriented mindsets. Co-creation gives us the best chance at relevance, impact, and empowerment

I first heard the phrase “build with, not for” from Catherine Bracy during her time at Code for America. It has stuck with me ever since, in part because it focuses on the language technologists use. We talk about “deploying” solutions, as if communities were terrain to be occupied rather than partners in the work. Recent research on “AI for good” partnerships bears this out: funders’ agendas routinely override community needs, technical metrics don’t align with social goals, project don’t meet expectations of the target audience, and technology gets centered ahead of the actual social.

One counterexample in the same space is my Northeastern colleagues’ AI-EP project. They brought together school systems, students, researchers, and parents to identify points of confusion around their children’s Individualized Education Programs (IEPs). These are legally mandated documents detailing education supports their children can receive, but are often dense and hard to understand. Only after hearing those concerns did the team build a translation engine and chatbot to answer parents’ specific questions. It’s a small pilot, still iterating, but it’s a model for sequencing: listen first, build second.

The AI-IEP team and some of the artifacts they created.

An off-screen example comes from my own work on creating participatory data murals I’ve helped communities paint in public spaces with my wife, Emily Bhargava. Data is increasingly front and center in civic participation, but spreadsheets and charts aren’t universally accessible forms; the materials and forms of data change who is able to work with it. Borrowing from rhetorical power of community art, and related participatory processes, we facilitate communities finding a story in the data about them, designing a visual to tell that data story, and painting it together in a local setting as a public mural. This builds on the longstanding power of murals to impact public conversation and policy. Similar ideas are in my recent Community Data book.

The key lessons for practicing civic jugaad to build with, not for?

Engage people with participatory design methods and distributed ownership of tools and processes.
Iteratively learn through real work experimentation, while acknowledging the potential for harms.

4. Study systems of power

Communities trying to use technology for civic good are often subject to the power structures those contain. Peeling back to the layers can present opportunities to reclaim power.

The last principle borrows a phrase from anthropology: “studying up.” Anthropologist Laura Nader argued back in 1972 for turning ethnographic attention toward the culture of power and affluence, rather than only studying the powerless. In technology research, that means training our tools on owners, corporations, and institutions instead of only on the communities affected by their technologies.

A wonderful example from my colleague Wonyoung So is Landlord Tech Watch: the team builds simulated tenant screening reports to reverse-engineer how algorithmic scoring systems embed racial bias into rental decisions. These systems, relying on criminal records, eviction histories, and credit scores, disproportionately exclude people of color as tenants. Their findings have directly supported tenants’ rights advocacy and a 2024 legal settlement with a major screening vendor.

The Landlord Tech Watch team created explanatory websites to show their findings.

With another research hat one I’m co-PI of the Media Cloud project, which offers a directory of global online news sources and a searchable index of almost 2 billion stories. Our tooling operates as public data infrastructure, supporting study of media narratives pushed out by various parties. It allows anyone to dig into media framing and shifts over time.

The key lessons for practicing civic jugaad to study systems of power?

Identify any existing oppressive power structures around your innovation and work to dismantle them.
Think about how your methods can help other “study up.”

Civic jugaad as a response

I offer this concept of “civic jugaad” as a hopefully helpful way to think about aligning your/our work to respond to the challenge of the moment (summer 2025). This is a model that western research settings can learn from. We’re not going to out-resource the platforms consolidating power around us, and we can’t wait for research funding to return to its old levels. But we can borrow from a tradition built for exactly this kind of scarcity: pay close attention to local needs and constraints, connect rather than isolate, amplify what’s been neglected, build alongside the people affected, and turn our tools toward studying power rather than only the powerless.

Civic jugaad is still very much a work in progress; more provocation than framework at this point. But if any of these examples resonate with your own work, please reach out and let me know.

Context, needs, and constraints drive jugaad innovation.

Protest Mapper: a new tool for journalists covering protests

Thu, 08 May 2025 15:00:00 +0000

As protests against the Trump administration spread, journalists are covering local rallies — but putting them in context of larger movements can be a challenge. To help, I built a simple tool that lets you search for recent protests near you and create an embeddable map you can use in your reporting. Try out Protest Mapper.

A map of the more than 400 protests in eastern Massachusetts since the start of the year.

Media Attention to Protests

One of the responses to the nationwide April 5 “Hands Off” and May 1 Mayday rallies was shock at the perceived lack of coverage from mass media outlets. Critics pointed to a lack of both print and digital coverage, arguing that it didn’t reflect the sheer volume of people that came out on the streets.

A sample of headlines critiquing media coverage of the April 5 “Hands Off” protests.

At the same time, local and regional media did show a noticeable bump in coverage. A query for articles mentioning both “hands off” and “protest” in the Media Cloud online news archive shows almost 400 matching stories on the 5th.

A Media Cloud query for “‘hands off’ AND ‘protest’” shows a peak of coverage in state and local papers on April 5.

Yet it appears that local reporters are covering protests in their area, but not often connecting them to larger movements. That might be because coalitions like #50501 aren’t as well known as unions and long-standing activist groups; they don’t have communications people with long-standing relationships to journalists.

Finding & Evaluating a Public Protest Dataset

One approach to help reporters make those links for readers, and put individual events in a broader context, is to use data about local protests. Connecting this weekend’s rally to events over the last few weeks might connect dots for audiences that are seeing public displays of resistance. I wondered if I could quickly map protests in my area based on existing data sources.

I first reviewed existing data sources to see what was available. A quick summary of what I found:

The GDELT project appears to have massive scope and has a “Protests” event category, but I found it too hard to access and use, with many broken links and inconsistent information.
The Carnegie Protest Tracker seems to focus on broad movements, not individual events.
LiveUaMap seems well established, with lots of social media results, but I found the interface challenging to use and they charge for access.
Amnesty International’s Protest Map focuses on police violence in response.
ACLED (Armed Conflict Location & Event Data) appears to be a well staffed non-profit that makes data easily available, updated once a week.
The Harvard Ash Center Crowd Counting Consortium (CCC) appears to be an academic project that is updated once a month.

This led me to focus on ACLED and CCC, since they are both regularly updated and easily accessible/usable. Each includes a list of protests and related manually curated metadata. They include protests about any topic.

My next goal was to evaluate these datasets. One metric that came to mind was to search for media coverage about protests in a fixed place and time, and then to compare that to the datasets at hand. I decided on Massachusetts in February, a context I know well, and pulled datasets from both Media Cloud and ACLED. To find media stories I queried Media Cloud for relevant anti-Trump/Musk protests with this query:

article_title:(
(protest* OR rally OR strike OR boycott OR walkout OR demonstrat* OR vigil)  
AND 
(Musk OR Trump OR Tesla OR closure OR federal OR university OR Israel OR Palestine OR cuts OR speech OR student OR research OR congress OR ICE OR agency OR USAID OR NOAA OR NSF OR NIH)
) and language:en

This discovered 76 related stories. I then coded them by hand to indicate if they were about protests in Massachusetts, identifying 41 stories about anti-Trump protests in the state from Media Cloud. In parallel, I pulled February data for the US from ACLED and filtered by the admin1 column to just Massachusetts events, identifying 51 entries in their database. To evaluate the two, we then reviewed every story in the Media Cloud corpus to identify which ACLED event it was about (linking it by the unique event_id_cnty provided from ACLED). Just two protests were in the Media Cloud dataset that did not have a related ACLED entry, both small small events reported by university student-run news sites. 95% of the sample I pulled from the basic Media Cloud query were already in ACLED, leading me to believe it was a trustworthy source for a list of protests. You can review the coded evaluation data.

A Mapping Solution

Dataset in hand, I decided to create a simple tool that would allow journalists to embed maps of recent local protests in their stories. The map at the top of this article is one example. Visit https://dataculture.northeastern.edu/local-protest-map/ to try it out yourself. You can filter by location, timeframe, and customize the map a little.

Tools like these require ongoing support, so I’ve made some decisions to try and minimize maintenance while maximizing utility. Here is a list of design decisions, concerns, and commitments:

I’ve mimicked the well known DataWrapper workflow to try and make it familiar and usable for data journalists. At the end you get an embed code (HTML iFrame) you can copy and paste into your webpage, just like I did above
This is deployed as a static HTML/CSS/JS site with no database or server compute power required, written with the open-source Svelte UI framework.
Some protests are marked in the exact same spot, for instance in the center of a city. This makes them very hard to see on the map. For these events I’ve a small random bump to the location in order to make them more visible. That is noted in the popup for impacted markers.
The code is open source on GitHub for full transparency.
I’m hosting the tool on GitHub Pages so I don’t have any ongoing expenses.
IFrame embeds can be tricky and limited, so the tool also offers the option to download a static image of the map in .png format.
You can pick between the ACLED (updated weekly) and CCC (updated monthly) data. For now I plan to manually pull the updated data each week, but hope to automate that later.
I’ve used a light background map base layer from Stadia Maps. This means it can’t support more than 200,000 hits across all uses. If this somehow magically becomes very popular I can easily change that via the Leaflet mapping library.

Give it a try, share it, and let us know what you find in the comments. Hopefully this tool and explanation will help you and others include data about the wave of protests to your reporting and add context to individual events you’re covering.

Originally published on Storybench

AI for Good: Five Key Principles for the Pro-Social Sector

Wed, 26 Feb 2025 02:00:00 +0000

Artificial Intelligence (AI) is reshaping industries across the board, but its impact in the pro-social sector is worth digging into more to understand emerging norms and challenges. Our typical examples stem from years of work on “civic tech”, “public interest tech”, “data science for good”, and related efforts. Most of these focus on challenges like species conservation, disease outbreak surveillance, deforestation tracking. Who doesn’t want to save threatened zebras? I describe these as accepted good: problems that most people agree need to be solved.

But there are problems less people agree are for the common good. What about AI projects that support for gender equity? Or projects working against surveillance capitalism? Not everyone agrees that these are projects that support the common good (though I do). These could be collectively called AI for contested good, and that’s the concept I want to dig into. Specifically, what are some key principles and methods to use when designing AI projects that support work on contested good in the pro-social sector.

This post introduces that thinking and some related examples. This line of thinking stems from an invited talk I gave recently at the UTSA MATRIX consortium’s AI Spring School, which offered an opportunity for me to spend a little more time conceptualizing and formalizing thoughts about these issues for the highly technical audience.

I took a quick selfie from the stage while the audience was doing a quick chat about projects they would describe as “AI for good”

1. Center Underserved Issues

Instead of focusing on well-studied and understood problems, turn your attention to innovate in neglected areas. This presents an opportunity for significant social impact.

This is particularly relevant in the context of feminicide, where mainstream institutions fail to collect and analyze data effectively. As Catherine D’Ignazio reminds us in “Counting Feminicide”, “What isn’t counted doesn’t count.” Over the last few years she’s been working with groups that create “counterdata” on feminicide to fill gaps in official records on gender-related killings of women, girls (cisgender and transgender). The work of these civil society groups to produce datasets about this violence supports advocating for policy, enforcement, and garnering public attention.

Producing this data is hard, both logistically and in regard to the human toll. Global groups struggle to create datasets of gender-related killings with ill-fitting tech. This creates significant data gaps and emotional trauma. As one partner noted,

It is not easy to read ten cases of feminicide and put them on a table, disaggregate them, have to put a name, age, circumstances and all that detail, without it affecting you emotionally.

In response, I’m part of the the Counterdata Network that works with groups to create bespoke machine learning classifiers that score news stories, cluster them by events, and deliver via email/web. Over 30 groups use them daily to monitor human rights and civil rights violations via news reports. We co-designed the system to help, basically replacing google news alerts with a custom-built open-source data pipeline. The network is growing across multiple topics: feminicide, criminalization of pregnancy, business human rights violations, and more.

This type of work emphasizes the need to look for issues that are systematically ignored and listen to voices that are unheard or silenced. Innovate in neglected areas with higher potential for social impact.

2. Build With, Not For

Technology should not be imposed on communities but co-created with them to ensure relevance, impact, and empowerment. This participatory approach contrasts with the traditional top-down model of technology deployment.

I first heard this phrase “build with, not for” from Catherine Bracy during her time at Code for America. She used it to emphasize the idea of co-creating technologies with government and community, instead of creating top-down solutions. As technologists, even our language is embedded with the opposite approach – consider the ideas of “deploying” vs. integrating” tools.

One response to challenges around Individualized Education Programs (IEPs) in the U.S. present a good government-related example to learn from. IEPs are plans created by public schools to offer special education services to children in need of more specialized care. They are created collaboratively across the school system, teachers, and parents. These legally mandated plans are in use for around 8 million students across the US. However, many parents struggle to understand the services their child is offered in their complex IEP.

The AI-EP team used interviews to identify common concerns parents had about IEPs. Through interviews, the team identified key challenges and built an AI-powered translation engine and chatbot to help parents navigate complex documents in a personalized way. The features include translation, question answering, and more. This creates a new AI-supported method for helping parents understand a complex document that is written in legal language using complex terms.

Projects like this remind us to focus on participatory design and iterative solution that integrate feedback from the impacted community. Co-creation ensures relevance, impact, and empowerment.

3. Study Up

AI can be a powerful tool for interrogating power structures, flipping traditional methods to focus on the culture of power rather than the culture of the powerless.

Anthropologist Laura Nader’s concept of “studying up” is essential here—using AI to investigate those with authority rather than those affected by their decisions. She argued that the methods of traditional anthropology, which typically studied groups with less power in social structures, should instead be flipped to study mechanism of power like colonization. Chelsea Barabas’ work on bringing this idea to algorithmic fairness has been highly influential on my thinking here.

A striking recent example is the Landlord Tech Watch project. Landlords increasingly rely on automated screening tools that use criminal records, eviction histories, and credit scores to assess tenants—often reinforcing racial and economic discrimination. By simulating tenant reports, this project demonstrated how these technologies embed bias and provided advocacy groups with concrete evidence for policy change (So, 2022). They are studying the tools of those with power in the rental market to understand disparate impacts and work to fight against them.

This line of work studying up in AI systems identifies that we should prioritize identifying and interrogating power structures so that we can then flip traditional research methods to challenge systemic biases. We can leverage advanced AI tools to investigate those with power.

4. Look for Quick Wins

Does every problem need a complex AI solution? Often, small, targeted interventions offer more sustainable impact than complex, resource-intensive projects.

Journalists are often at the cutting edge of leveraging new technologies, and considering the ethical implications for their field… genAI is no exception. Their work serves as a model for building small, targeted, interventions that offer quick wins with lightweight AI integrations.

One compelling example is the Eva chatbot developed by the Paraguayan news outlet El Surtidor. Paraguay has many women in prison, or awaiting sentencing, for drug smuggling. The team wanted to open a door into the taboo issue, so they trained an AI chatbot based on interviews with women awaiting sentencing for international trafficking. Readers could interact with the bot to ask quesitons about their cases, how they got caught, and more (with appropriate guardrails). It quickly reached more than 15,000 interactions with the public. Eva helps reframe the public narrative around drug-related crimes by challenging stereotypes and stigma.

Quick-turn projects with tightly defined use-cases like this help us consider what kind of new stories with social impact we can create, focusing on lightweight, adaptable solutions that offer immediate impact. How can AI help you make impact on a small but well defined idea?

5. Consider Hidden Risks

GenAI technologies are built to look magical, and often produce good-looking results at first pass. Scholars are adopting them to support media research, without fully considering the very real social, financial, and ecological impacts.

More and more often I’m seeing genAI used for computational tasks that are well solved with regular ML: entity extraction, geographic ID, etc. There are a set of already defined problems that we don’t need genAI for. Ongoing work by academics and journalists are uncovering the true costs of the massive investment in LLMs. Like Bitcoin, with genAI there’s a very real future where we have blackouts because of computational power usage.

An example is a recent study I worked on with a team from The Scope. We wanted to analyze articles in the hyper-local news source to assess content diversity via geography, topic, and quotes. We could have used genAI for all of this, as others have, but for geography and entities we used existing off-the-shelf solutions because they are well validated, produce reproduceable results, are free to deploy and use, and are open source (I’ve helped build both). However, without well-performing and easy to use existing solutions for quotes, we decided to try the genAI black box. We iterated on prompts against a manually coded list of quotes and ended up using ChatGPT v3.5, which was easy to integrate with, and cost little ($6 total). Quote extraction and attribution (via pronouns) worked well in the end, allowing us to do gender-based analysis. That was only possible at scale thanks to ChatGPT. In the end we were able to do a cross-sectional analysis along those three dimensions of content to help the editors audit their own content, and plan for any changes with that data in mind.

Our work on that study shows how genAI integrations should be carefully designed to only be used where necessary. That can help reduce areas where results are not easily reproducible and justify work towards algorithmic interpretability to mitigate unintended consequences. Companies creating LLMs are deliberately obfuscating the hidden power and environmental costs, and don’t have a history of being good partners to the media industry.

The five principles outlined here—centering underserved issues, building with communities, studying up, looking for quick wins, and considering hidden risks—are foundational for using AI responsibly in the pro-social sector. If you’re interested in AI’s role in social impact work, I encourage you to explore these principles in your own projects. These kind of approaches and examples can help ensure that we model how AI technologies might serve as a tool for empowerment, rather than perpetuating existing inequalities.