Data Culture Group @ Northeastern University

Learn CARefully (keynote on teaching AI for data journalism at SciCAR)

Thu, 17 Sep 2026 00:00:00 +0000

The following is a write up of a keynote I was invited to give at SciCAR, the annual German gathering of scientists, journalists, and researchers. Consider it under-construction thoughts, aligned with the tradition of blogging to think out loud.

Everyday we see a new gaffe, book, lawsuit, or report about AI and journalism. In the practice of data journalism, like many others, this new technology is already influencing how we approach learning in newsrooms and classroom. I want to pause to reflect my own approaches and catalog some inspirational examples here to tease out how we can ensure any engagement with AI in data journalism work is in line with the fundamental principles that underlie the practice of being a journalist.

SciCAR, like NICAR and the Computation +Journalism Symposium, is one of the few places where data journalists, academics, scientists and others come together to discuss and develop our collaborative work. You’ll note that NICAR and SciCAR share a now-antiquated acronym: “Computer Assisted Reporting” (CAR). The term sounds a bit dated to the ear because there few areas of reporting that aren’t assisted by computers, but “CAR” reminds us that computing was once considered a radical and unfamiliar force in newsrooms. CAR came to popularity as a label for exploring the possibilities of how computers could help in reporting, and the ways it interfered with the regular practice of journalism.

I was honored to be invited to deliver this keynote talk at SciCAR in Dortumnd.

I think we need to return to that way of thinking in an era of chat-based large language models (LLMs), considering how and what we want computers to assist with. So CAR is the basis of my somewhat tongue-in-cheek reminder: we must teach and learn extra CAR-efully right now, committed to thoughtful consideration of how and where we use AI to assist our data reporting. See what I did there?

Some Context

Like so many of you I’ve been tracking developments in AI and journalism, helping launch a series of related news round-ups on Storybench that our Northeastern J-School grad student Vivica D’Souza continues to release every month or two. A core tension emerged for me from watching developments over the last two years: practicing journalists are both adapting and resisting. In parallel, universities that train new journalists are widely adopting AI, causing many student groups to organize and push back (arguing that “AI is adverse to the idea of learning”).

The flood of news updates about AI and journalism is almost impossible to keep up with. We’ve been writing round-ups on Storybench to highlight developments every month or two.

I interpret this resistance as centrally about power: who gets to decide how and for what these new tools are used in newsrooms and J-School classrooms? With this lens I’m immediately reminded of the Luddites of the early 1800s. Luddites weren’t opposed to the idea of industrialization, they were skilled workers fighting for the application of those new machines in ways that continued to produce high quality goods, were under the control of workers, and didn’t violate existing labor practices.

Does that list sound familiar? It sure does to me. Luddites come up often in media discussions about resistance to AI. In the field of data journalism, AI-driven work risks polluting product quality, is being forced into many learning settings, and is putting careers at risk. Yet practicing data journalists are rapidly starting to use AI to help develop stories, create interactive visuals, support investigative analysis, and more. There’s that central tension again.

So I choose to engage with AI technologies in my teaching and collaborations with learners of all types. I focus on thoughtful use of AI, critiquing misuse of AI, and principled refuse of AI. Data journalists have agency in how AI is engaged, or not engaged with, in their work. Let me run through a few central principles and help me reflect on how to do this well.

Be Specific In Your Learning

Once of my guiding principles here is to always be specific when I introduce AI for data journalism. AI companies market that their models can and will help with everything, but that’s mostly because they are unproven and untested; these are new tools that are looking for market validation and dominance. In that way they remind me of the introduction of the Apple Watch, which was pitched as a communication device (remember the idea of drawing hearts to send to your loved one?), a fashion accessory (remember the 24 karat gold watch?), a high-precision tool to tell time and a health and fitness tracker. A decade later it is clear that the last on those list is the market they found; it is primarily a health and fitness wearable.

AI hasn’t gotten there yet. We don’t really know yet which applications it will really help us with. This makes it hard to introduce in productive ways. Can AI help us brainstorm, automate workflows, edit our work, code for us, scrape datasets, make charts, add annotations, surface insights in CSVs? We don’t know yet. I find that being specific about those as separate potential capabilities helps significantly.

We’re beginning to see examples of recurring concrete use-cases for AI in data journalism, which I classify into buckets of data production, data shaping, data analysis, and data representation. One example on the data shaping side is CNN’s recent work identifying a pattern of Trump promoting companies on his Truth Social blog days after buying their stocks. Reporter Isabelle Chapman shared how they “used artificial intelligence to compare a database of Trump’s Truth Social posts with a full list of stock trades released in his annual financial disclosure”. I imagine using AI to link between the two datasets helped them with scale and fuzzy matching logic. These two tasks are well-known challenges in investigative data work, so they offer very specific (and verifiable) opportunities to leverage AI.

Another example, from the realm of data representation, is using AI to write the code that renders complex and novel interactive graphics. This past spring graphics developer (and Northeastern J-School grad) Yuriko Schumacher shared with my Data Storytelling class how she’s using AI coding in her work at the Athletic. Code generation has helped their teams iterate faster, robustly support desktop and mobile, and focus on interactive design so their readers are informed and engaged with complex visualizations that represent large sports-related datasets.

I point to these kinds of examples in my teaching to support discussion about how AI might help or hurt our ability to perform specific parts of our data storytelling. We don’t discuss “AI” as an abstract; we instead focus on concrete applications of chat-based LLMs within a list of concrete data journalism tasks.

Learn by Trying Things Out

Specific examples help push against the pervasive industry hype about AI “changing everything,” offering a path to concrete things learners can try out. Scholars from various domains have led research into how organizations working for the public good often repurpose technologies, putting them into service for purposes other than those they were designed for (including in journalism). Over and over this research shows that the process of adapting technological tools begins by simply trying something out to see if it helps.

One example from my own teaching is related to infographics. Early in my Data Storytelling course I ask students to write a short story about a small dataset with two standard charts. After this I have them put a one paragraph narrative summary, alongside the dataset, into a nano-banana model (via Gemini), asking it to turn their story into an infographic. The results are mixed and often hilarious, prompting opportunities for visual critique and sometimes sparks of symbols that could be repurposed. Most of the graphics produced last year were terrible but the point is that we practiced assessing some hype: nano-banana is supposed to be great at infographics, but the results we got by trying it out quickly were unimpressive. That’s important, and builds confidence in students to try things out and make their own judgments rather than trusting the marketing hype.

Examples of infographics created by my students using Gemini in spring of 2026.

Trying things out like this sounds great, but we have to consider the costs incurred in the playgrounds we create for ourselves and others. Legitimate and real concerns about water use, energy supply costs, training data copyright theft, big tech motivations, increasing costs, the spread of “slop” AI generated content, out-of-date models, and impact on cognition are all raised regularly and written about in depth. These can’t be ignored and have to continually inform how much we’re willing to try out AI tools that have so many negative impacts. I don’t have an easy answer here, so instead I engage with my peers and students to see how they feel vis-a-vis these concerns after trying out an AI tool.

Shaw & Nave in their oft-linked “cognitive surrender” paper found that people using AI to assist a task just kind of did what it told them.

However, I do think trying to generate code is a distinctly specific task within that space. Yes, I know and have seen the mixed results from research on learning to code with AI, but put that aside for a minute. My concerns about those costs I listed above are reduced in the case of AI for coding for a few reasons. First, much of the training material was obtained legally, scraped from online open source repositories with licenses. Second, data representation or analysis in journalism contexts is short-lived, with a ship-date after which code seldom needs to be revisited so technical debt accrued doesn’t matter that much. Third, local models are getting better quickly, meaning that cloud costs and corporate control aren’t as relevant. Finally, code is verifiable. You can see what the AI model produced, and the code you run is deterministic, unlike the usual probabilistic outputs of an LLM. Generating code to help in your data journalism feels like a specific use of AI that stands distinct from many others and I have far fewer concerns about it. To be clear, that’s only when you consider learning data journalism as the goal, not if your goal is learning to code; that’s an entirely separate discussion.

The Generative AI in the Newsroom blog from Diakopolous and Gilert at Northwestern University is a great live reflection of what the industry is trying out Regular posts from global newsrooms catalog the results of experimentation with AI; I read every single one. Their recent Agentic AI Investigative Challenge dove even deeper; the winners all showcased prototype investigative AI tools that centered provenance and reliability. These journalists are all trying things out and reflecting out loud about how it went.

Getting your hands dirty with tools, like authors for that blog do, is central to understanding their capabilities. I ask learners I work with to try things out because that is the best way to understand the opportunities and limitations of AI in data journalism.

Center Journalistic Principles

ProPublica’s AI principles page offer that they “treat AI-generated material as unverified source material.” Accountability , verification, transparency, integrity, privacy & security–their list is a pretty good set of core principles to guide your journalism. Sadly we have a growing number of AI gaffes in the daily operation of normal journalism practice that we can point to as the opposite, from the US state dept generating an incorrect map of Africa, to the New York Times correcting that a statement was an AI summary and not a quote of the Canadian opposition leader.

I take inspiration from well-written AI policy transparency like ProPublica’s, especially when it the ideas are so well-aligned with journalistic principles.

Readers see this. A study from Trusting News found that readers across 10 global news organizations overwhelmingly think AI disclosures are “very important” or “extremely important.” However, other research found that while readers value these disclosures, they still left the impression that AI connected journalism was less trustworthy. I have my students all turn in statements based on guidance from Trusting News’ “AI Trust Kit”. Writing these considered AI disclosures builds a strong reflective practice and reminds them that these core principles must guide all our work, no matter if it uses AI or not.

But how we develop these AI guidelines is also important. Most organizations start by looking at what AI is being used for and then consider what rules they need for those situations. This kind of approach, argues researcher Daniella DiPaola, is reactive and offers poor long term impact. Instead, she posits that we should start with what we care about most, and then develop AI policies that reinforce those core values. I find her values based approach to developing AI policies extremely helpful, and though it was developed for school systems I think it could be immensely valuable to newsrooms (and underlies how ProPublica presents their’s).

This kind of consideration of what principles guide engaging with AI in newsrooms came up as well in a recent Dutch thesis by van Bree. They describe how journalists engage AI tools in three ways: a colleague, an assistant, or an outsider. The colleague role cedes some authority and agency to AI while looking over it’s shoulder or helping it work. The assistant puts the human in charge of a collaboration with clearly defined tasks the AI would undertake. The outsider AI was seen primarily as a threat, devoid of verifiability and mostly needing attention as a subject of reporting. These three roles echoed my own experiences, where I move between considering what role the AI tools are playing. As noted, my CAReful approach means that I most often take the position that AI can be my assistant, following my directions and leaving the primary journalistic or design work of data journalism to me.

I appreciate van Bree’s naming of roles, and choose to highlight using AI as an assistant in my own learning and teaching. This is my own simple illustration.

Of course, this talk for SciCAR is in Germany, so I had to again laud the amazing Nazi party membership database project from Die Zeit as a well-considered inspiration in the investigative domain (which I admittedly know less well). They used AI up and down the chain of data processing for this deep investigative project. Throughout they offer strong examples of centering transparency, such as how they include an option for readers to offer a correction if some aspect of the processing of the hand-written cards led to an error or omission. Staff have shared with me that the story was a huge hit with readers, drove a major bump in subscribers and in turn created deeper investments within their newsrooms for AI investigative capabilities.

No matter what you learn or try with AI, centering journalistic principles is key. In the hype cycle and push to AI-ify everything we can’t lose sight of the core values that make data journalism what it is.

Stay Critical

My last guiding principle responds directly to the marketing and hype cycle of AI tools: we have to keep asking critical questions. In keeping with the rest of this piece, here I have more questions to offer as insights rather than answers. Sorry!

Is Big Tech a reliable partner? Remember the failed promises of a “pivot to video”? Offers of money, attention, and more from Big Tech are best viewed with skepticism. AI tools don’t cite news sources that they were trained on. AI company leaders don’t seem to view the misinformation and news-slop problem as critical. Our business models are not aligned. The recent history of journalism and Big Tech should make you constantly question if they are a reliable partner or provider of services.

Can we create alternatives? The AI moat might seem to large to cross, but if we can’t trust the California-centric monopoly perhaps we need to try. Model development is becoming more well understood, leading to public LLMs from countries from Switzerland, India, Poland, Spain and beyond. Even Reuters is getting in on the LLM-model building game, releasing their own “Thomson” model earlier this year. There are opportunities for custom AI work on specific tasks related to data journalism as well, as evidenced by the work Zetland did to build Good Tape, a revenue-generating transcription service now used by millions. Newsrooms and data journalism teams have long been an innovator in technologies (d3.js, Django, Svelte, etc.), A recent workshop I joined, led by my friend Harini Suresh at Brown, on “Collective Governance of AI in Newsrooms” left me hopeful that this energy is being directed at creating industry-led alternatives.

What of value is lost? Even when automation helps with tasks such as transcription, we have to consider the cost. Older journalists around me agree that the nuance of a speaker’s voice and repeated listening to a quote over and over build a subtle and deep relationship with a source when transcribing, one that is often critical to telling the right story. There are clear advantages to automated transcription, yet we also must introduce the loss of that relationship as a tradeoff to learners. In class I simply ask students to read a sentence, and then listen to the same thing read aloud from the source; the difference is obvious. This question is important to keep asking ourselves. For instance, what of value is lost when a robot is writing the d3.js code for your next interactive data visual?

Is AI good for learning? There is still much to see, but early evidence suggests that using AI can help in general assignments like homework assignments, but hurts in longer term learning like on exams. As I’ve already noted that I see a real gap in learning to code with AI; students who have a certain level of knowledge already can use and learn with AI coding tools quite quickly, but those who haven’t met that threshold learn significantly slower. Frank Elavsky writes about the misperception of how AI might help us wonderfully, nothing while we want AI to fill in skills needed to implement our wonderful ideas, in fact it is often used to produce high quality technical projects for poorly refined ideas. Conceptualizing and building are different and intertwined processes, requiring reflective use of tools to avoid the “slop-zone” he points to.

These four questions are the type of things I keep asking myself, and others I learn with, in regards to how AI might be used in data journalism. Asking ourselves hard questions is how we stay critical in this time.

Reflecting on Knowing

If you were expecting me to tell you to make charts with AI, or to not make chart with AI… sorry! These reflections are about the learning of data journalism in an AI era. If you know my background this won’t be surprising; I studied learning and computation for my M.S. at the Lifelong Kindergarten group at the MIT Media Lab. Yes, I went to kindergarten twice! The approach to designing creative learning experiences I learned there continues to inform all my teaching and research. More deeply, the idea of thinking about thinking continues to lead me to these kinds of moments of reflection.

It turns out that ability I was introduced to years ago is critical right now, because AI tools are changing how we think, learn and know things. The relevant academic term for this is “epistemology”: theories of how we know things. It’s a little meta but important to consider. Consider the simplistic example of ways you could know a soup is too salty: measuring the amount from bowl to compare to the recipe, tasting it yourself, hearing a trusted friend tell you, etc. I’m convinced that using AI tools to learn something changes how to you know it, though I think we don’t quite understand precisely how yet.

In older work I cited Heron’s concept of “extended epistemology” to highlight and explain that we have different ways of knowing things. I argue data journalist has a lot to learn from this.

This idea of how we know things matters in data journalism. In a talk for AoIR I argued that the off-screen arts-based approaches I work on in community data projects are a form of “epistemological resistance”, fighting the dominant formalized ways of knowing. This borrows from Turkle and Papert’s “epistemological pluralism” in software learning and theories of fighting colonialism from sociologist de Sousa Santos. AI changes how you think and learn in important ways, and you should be aware of that so you can resist it as needed. Reflection is an important part of learning, and one I’m trying ot practice right now with this piece.

Learn CAR-efully

So what does this mean for your learning and teaching? How do we apply my themes of being specific, trying things out, centering journalistic principles, and staying critical while AI technologies continue to be adapted into data journalism settings? Reflecting on their work in the Google / LSE JournalismAI initiative over the last few years, Schjøtt and Schaetz label what they saw as “domestication” of AI in newsrooms. They lay out four parallel paths the technology followed in newsrooms: demystifying AI to understand what it is, cultivating skills to use it, managing the limits, and appropriating it into practice. van Bree used the same term: “domestication.” If AI is a wild animal to tame in our data journalism, what methods are we to use?

I find those four paths of domestication useful to pause and consider for myself as a reflective learner. Sometimes you need to step away from the hustle and bustle of a tool and project to reconsider it from the ten-thousand foot level. In the end their report saw the processes led to both (1) increased confidence to try and engage with AI and (2) a growing sense of the inevitability of AI in the newsroom. There’s that tension again. That mixed outcome mirrors my own experiences on this space, and I’m sure will continue to complicate my under-construction approaches to teaching and learning AI in data journalism. Across those principles my key message at SciCAR is to learn CAR-efully. Hopefully the approaches and the examples I’ve shared inform and connect to your own experiences.

My closing side: learn CARefully.

Whose Quotes Get Cleaned Up?

Sat, 16 May 2026 15:00:00 +0000

In mid-February, The New York Times ignited a storm of criticism with a piece about Representative Alexandria Ocasio-Cortez at a security conference in Munich. The article was critical (the headline included the phrase “with some stumbles”) but the anger it provoked was caused by a verbatim quote that included repeated “ah”s, “um”s, and “uh”s. We all speak like that while thinking out loud, but even the Times’ own Guidelines on Integrity indicate that “The writer should, of course, omit extraneous syllables like ‘um’ and may judiciously delete false starts.” Was this sloppiness? A hit piece on AOC? Or was this a signal of something deeper about how mainstream news edits quotes? I dove in with some code, genAI, and theories to try and find out more about this kind of “quote cleaning.” What I found suggests that including “um” is incredibly rare and opens up deeper questions about how quotes operate in news to normalize some speakers and scrutinize others.

The quote as published by the New York Times

The Politics of Quotes in News

It’s typical, and even helpful, for journalists to tidy up a direct quote for better readability. We teach this in journalism schools and many media outlets codify this in editing policies (as those Times guidelines demonstrate). That said, the practice is largely invisible: I doubt most readers think about whether the words in quotes exactly match every sound the speaker uttered. This kind of quote cleaning is an editorial choice, and like all others, it embeds values. In this case those values are likely to reflect underlying ideas about what counts as articulate, authoritative speech. Who sounds smart? Who sounds right?

This isn’t a surprising comment in the context of our racialized politics. Black Speech and Spanglish are often used by political leaders in those communities, yet seem to show up less often in news quotes. Any multi-cultural person knows the norms of code switching in language to match the dominant audience expectations. A high-profile recent example is Kamala Harris, who was criticized as being inauthentic because she spoke using different vernaculars to differing audiences.

Consider a more contemporary counter-example: President Trump. His often-incoherent rambling statements have challenged editorial quoting norms (among so many others norms he’s challenged). A whole new term — “sane-washing” — has come into vogue as part of related media critiques. Responses have ranged from the NPR Public Editor’s explanatory engagement, to op-eds decrying the coverage, to criticism of how little what Trump actually says is quoted.

These anecdotes suggest a pattern where high-status white politicians often get their quotes cleaned up as a matter of course, but others who don’t belong to that dominant group might not receive the same treatment. AOC sits at the latter group, as a Latina progressive woman. This editing from the Times lands differently with that in mind.

Can Data Help Us Understand This?

To investigate this specific example, I decided to see how often “um” and “ah” were included in quotes in the most visited US online news sites on the same day that article was published (Feb 13, 2026). Conveniently, when we want to answer questions like these we have a wealth of technologies at hand to help. Specifically, with my colleagues Meg Heckman and Elisabeth Hadjis, I just published a paper in News Research Journal about using genAI prompts to extract quotes from news stories (something traditional machine learning doesn’t do well). I remixed methods from that work to answer this question about just how rare including those utterances was that day.

My analysis included 36,045 quotes across 8,066 articles published by 45 news sources on Feb 13th. From those quotes, just eight quotes included “um” — one was from AOC while the rest were non-politicians. That’s just 0.02%! Even fewer, just three stories included “ah”; none from politicians. I can confidently conclude that including “um” or “ah” in a quote from a high-profile politician on Feb 13th was a very statistically rare choice.

The eight quotes identified with “um” or “ah”

Of course, since I don’t have raw transcripts for all those stories I can’t tell you how many dialectical markers, code-switches, and instances of non-standard grammar were removed. I’d be curious to investigate this more for politicians who carry multiple identities, to see if there are keywords from a dialect indicative of code switching that I could track across how they’re quoted in news stories. Consider this an invitation to explore more: my little experiment suggests that we have the tools to answer these kinds of questions.

How’d I Do This?

Curious to try this kind of thing yourself? Here’s a bit more about how I found these results. I remixed our prior approach from that paper with a set of the most-visited online US news sources, the Media Cloud online news archive, a revised prompt for genAI, evaluation with the DSPy library, and a small set of duct-tape scripts to see what the data said about how normal — or not — that quote was.

I’m a hesitant generative AI user, so at each step where I used it I tried to be thoughtful about how to check the validity of the results it gave me. I call this my “genAI-last” approach: I try to use any other reasonable computational methods before falling back to genAI to help me solve some research analysis problem.

A diagram of the data pipeline used for this investigation.

I began by defining what media sources I cared about. The seed here was the Times, so I thought it made sense to focus on some reasonable set of large mainstream online news outlets. Media Cloud has a collection of “Most Visited” news websites from January 2026, based on SimilarWeb data as published monthly by the PressGazette. I can’t really speak to the full methods they use to collect their numbers, but SimilarWeb data is widely used and the list looked reasonable to me. This particular list from Media Cloud was already filtered for US-based publications, which worked well because I wanted to look at American editorial norms.

With that list of publications in hand, I used the Media Cloud web-based search tools and API to pull all stories from those websites on Feb 13, because I just wanted a one-day snapshot for this short blog post. Then I removed duplicate stories based on whether there was a story with the same title from the same outlet on that day, because often the same story is published at multiple URLs. That left me with 8,066 unique stories.

In parallel, I iterated on creating a genAI prompt and evaluation set to help me extract quotes from stories and attribute them to speakers. An earlier version of that News Research Journal work we presented at the 2024 Computation + Journalism Symposium gets more into details about why, but there isn’t really an off-the-shelf open solution for extracting and attributing quotes. In general for this kind of work with AI models it is important to (1) define your task, (2) build a ground-truth evaluation dataset, and (3) create an evaluation and scoring rubric. To start the genAI eval I pulled a handful of stories and identified quotes and speakers by hand; this would be my evaluation dataset to see how well different prompts and models worked. To test things I used DSPy, a great Python library for iterating on and evaluating genAI prompts. An existing account on the cloud-based Groq model provider let me run test prompts against this evaluation set with various genAI models. I iterated with Claude, ChatGPT, and Gemini on refining the prompt, and ended up seeing best performance on a prompt that Claude expanded for the llama-3.3-70b-versatile open-source model (recall and precision both over 0.95). If this was a more developed project, I would have evaluated against more hand-labelled ground truth data and evaluated multiple runs of the same data (because genAI models aren’t consistent, by design).

With a prompt and dataset in hand, I then just needed to run the prompt against each story. Here at Media Cloud we have an internal-for-now data pipeline tool called “Sous Chef” for this, so without too much effort I was able to get the results back as a CSV of quotes with related story metadata. I used some simple code in a Python notebook to extract and identify quotes with “um” or “ah” in them, using Spacy to tokenize the terms (see the code and data). For each of these stories I went back to read it to make sure the quote and speaker identified by the AI model were correct.

This process took me about a day of work, split across a week of divided attention. This array of tools really has put this kind of quick media investigation within reach and presents very exciting opportunities to learn about norms in the news.

So What?

Why does it matter if editors and journalists “clean up” speech like this? I spoke with long-time Boston Globe editor Mary Creane to get her take. Creane started by noting that in news it’s “not our job to make people look stupid; on the other hand our job is to present people as they are.” As a rule, she said she would “never change their quote.” Instead, if the choice was made to remove an “um” or “ah”, she would use ellipses to make clear that some words in the quote had been taken out.

Creane emphasized the difference between speaking well and making a good point. Former vice-presidential candidate and Alaska governor Sarah Palin, for instance, claimed to be a foreign policy expert because you can see Russia from Alaska, which Creane noted was “not ungrammatical, but points out a weakness.” In situations where the speaker you want to quote is speaking poorly but making a relevant point, an alternative is to paraphrase. In a very classic sense, Creane believes that as journalists “our job is to say ‘here’s all the information, make of it what you will.’” When it comes to editing quotes appropriately, the core question for her is “if we’re not fair, if we don’t have integrity, then who are we?”

For me, this integrity was lacking in how the Times quoted Ocasio-Cortez. The quote used was aligned with the critical tone of the article, but intentionally broke policy and norms to portray her as ill-equipped due to the form of her comment, not the substance.

Despite all the change driven by social media, online news is still a strong driver of public understanding and shapes perceptions of those in power. This example says something about what types of voices we amplify and showcase in society, and where and when everyone is allowed to speak as their true self. That’s important beyond just research, because it is a statement about inclusivity, which we should all be aware of. I think it’s good news that the tools we can use to study the influence of news at scale are finally within reach.

Originally posted to Storybench

Data Drums wins a Data Sonification Award

Wed, 29 Apr 2026 15:00:00 +0000

We’re thrilled to share that Data Drums has won a 2026 Data Sonification Award in the Arts category!

“Data Drums: A Rhythmic Call to Climate Action” is a live performance that brings global CO₂ emissions to life through the dynamic beats of a Brazilian-style drum ensemble. Drawing on carbon emissions data from Brazil, India, and the United States, the four-movement piece translates numbers into rhythm. At various points in the performance each drum and beat represent a different slice of the data; Volume, tempo, and pattern shift to reflect emissions levels and growth rates. Through drumming the contrasts and connections between countries becomes visceral and immediate. You feel the emissions. The piece culminates in an interactive finale where the audience joins in, embodying the collective responsibility and possibility for climate action.

The project was co-created with Marcus Santos and Lily Gabaree, and supported by the Northeastern CAMD Climate Change and Public Communication Impact Group and the Northeastern Center for Transformative Media.

We’re proud of how this project pushes what data storytelling can be — not a chart or a dashboard, but a room full of people making noise together about climate change. Get in touch if you’re interested in bringing Data Drums or other participatory data experiences to your community. You can read more the ideas behind about Data Drums in my Community Data book.

Beyond Accuracy: A Case Study of Sociotechnical ML Evaluation

Sun, 22 Feb 2026 15:00:00 +0000

The use of large language models and generative AI has significantly impacted computational social science, creating new capabilities and raising new questions for those using software to support social analysis. Key among those is how to evaluate if large models or prompt-based workflows are actually performing better than conventional machine learning. Against this backdrop, we want to propose a more sociotechnical approach to evaluation, one that includes contextual metrics and discussion beyond the standard recall and precision scores. This brief case study shares what happened when we tried this approach on a classifier we’ve built as part of the Counterdata Network’s collaboration with the nonprofit, US-based organization Pregnancy Justice.

Some Background

Building on software systems built as part of the Data Against Feminicide project, the Counterdata Network partners with groups tracking human rights violations via online news to partially automate some of the content discovery and evaluation. In practice, this looks like working together with organizations to train custom ML classifiers that score candidate news articles from online news archives (based on geographic filtering and on keyword matching) for relevance to the violations they want to track.

One of our more recent partnerships is Pregnancy Justice, a national legal advocacy organization that advances and defends the rights of pregnant people, no matter if they give birth, experience a pregnancy loss or have an abortion. Pregnancy Justice’s research team tracks pregnancy criminalization cases across the U.S. in collaboration with academic partners. These are cases charging pregnant people with crimes related to pregnancy, pregnancy loss, or birth. Their extensive and rigorous work is documenting the impact of the Dobbs supreme court ruling, and supporting people brought into the criminal system because of their status as pregnant. Our software system surfaces relevant news articles about cases that fit their focus each week.

The platform where the Pregnancy Justice team reviews retrieved articles and labels them as relevant or not relevant.

Improving the Model

After collecting and annotating a sample training set and experimenting with ML models, the first version of the model was deployed into the email alerts system to retrieve articles in November 2024. The Pregnancy Justice team reviewed retrieved articles each week and labeled each as Relevant / Not Relevant. When our team analyzed the results, we found the false positive rates were near 40%, meaning that every 2 out of 5 articles returned to the team was not relevant to their search.

From a purely technical standpoint, this was alarming. The model seemed barely better than random chance. As ML researchers, our instinct was to fix the model. We designed three parallel improvement experiments to reduce false positives and boost conventional ML performance metrics. What we learned through the process challenged our concepts about what better even means.

Experiment 1: Should we incorporate better data?

First, we updated our TF-IDF model using a subset of newly annotated data from PJ’s weekly reviews. After incorporating the new data, the model’s false positive rate decreased from 55% to 10%. Our analysis showed that much of the original misclassification was a result of domain-specific terms, such as “midwives,” which appeared frequently in real-world news cases but were absent from our initial manually sourced training set. By incorporating the team’s feedback, the dataset better matched the real-world linguistic context, allowing the simple model to more accurately generalize.

Experiment 2: How about more complicated models?

Our first model used TF-IDF, a method that identifies important words that are common in documents we care about but relatively rare in the entire collection. It’s a relatively simple approach by modern NLP standards, as it counts words rather than attempting to understand semantic meaning.

We hypothesized that more sophisticated embedding models might perform better. Unlike TF-IDF, large language models (LLMs) like BERT, Universal Sentence Encoder, and MPNet are designed to understand the semantic context—the meaning behind sentences. Theoretically, they should capture the nuanced relationships inherent in complex social issues like pregnancy criminalization.

We tested several state-of-the-art embedding models against our current TF-IDF baseline. However, given the ongoing debates about the ecological impacts of LLMs, we decided to evaluate them on more than just accuracy. We also evaluated their environmental and computational costs.

The Trade-off: Accuracy vs. Environmental Impact

The AI community is increasingly recognizing that model performance cannot be divorced from its environmental cost. Extensive research has emphasized the nontrivial energy consumption and associated CO2 emissions from training and deploying LLMs. This environmental footprint —through pollution, resource extraction, and climate change— disproportionately affects marginalized communities, many of the very communities Pregnancy Justice serves. As a result, our team was committed to prioritize environmental sustainability. With this in mind, we evaluated not just accuracy metrics but also running time and CO2 emissions (estimated using the codecarbon package).

Embedding Model	FPR	Time	CO2 Emissions
TF-IDF (w/ new data)	0.12	1x	<1 mg
MPNet	0.07	105x	>10 mg
USE	0.07	31x	>5 mg
Sentence-BERT	0.09	18x	>10 mg
mxbai-embed-large	0.23	353x	>100 mg

Model performance (FPR) and computational cost (time, CO₂ emissions) across embedding approaches. FPR = false positive rate (lower is better);

Our results show that while some embedding models achieved low false positive rates, the computational costs were staggering. The mxbai-embed-large model, for example, took 353 times longer to run and emitted significantly more carbon than our original model. The accuracy gains were small, especially compared to the retrained TF-IDF model, with substantial environmental costs. This matches findings from other research, where simpler models such as static embeddings have been shown to perform comparably on digital humanities tasks while requiring significantly less compute (Ehrmanntraut et al., 2021).

After discussing these results with the Pregnancy Justice team, we reached a critical conclusion that the marginal gain in accuracy does not justify the increased environmental and computational harm.

Experiment 3: How confident should the models be?

Our final experiment involved adjusting the model’s decision threshold. Our model outputs a probability score between 0 and 1 for each article, indicating the model’s confidence that the article is relevant. For the first model we used a threshold of 0.5: articles scoring above 0.5 were classified as relevant.

Histogram showing the distribution of scores on stories from the first model. Note the majority after under 0.5, our threshold for including in results sent to the team.

We hypothesized that If the threshold were raised, e.g. to 0.7 or 0.8, the model would reduce false positives by only returning articles it was very confident about. Our experiments confirmed this, but also showed a tradeoff: a stricter threshold means missing some truly relevant articles.

When we presented this tradeoff to the Pregnancy Justice team, they offered a surprising insight: they wanted the false positives.They explained that reviewing even the false positive articles, those not directly about pregnancy criminalization, was actually valuable. These articles helped them understand the broader landscape: adjacent policy discussions, related criminal justice issues, and the wider context in which pregnancy criminalization occurs. What we labeled as irrelevant from a technical standpoint was often relevant from a strategic advocacy standpoint.

This insight fundamentally reframed our evaluation. The model wasn’t just a filter to remove noise—it was a discovery tool that exposed the team to a broader information ecosystem. The “noise” contained signals we hadn’t anticipated.

Towards Sociotechnical and Community-Centered ML Evaluation

This case study highlights three critical lessons for evaluating ML systems with stakeholders and communities:

Bigger isn’t always better: Improving the training data might yield far significant improvements than increasing model complexity.
Let communities define success: Technical metrics like accuracy and false positive Rates are proxies for utility. What counts as correct output depends on contexts, workflows, and values that these metrics cannot capture. We recommend co-designing metrics and success criteria with the people who will use the system.
Evaluate the System, Not Just the Algorithm: Efficiency is an ethical metric. By quantifying the time and CO2 costs, we made a value judgment that aligned our technical infrastructure with our project’s justice-oriented values.

Data Journalism in Dark Times: notes from the C+J keynotes

Sun, 14 Dec 2025 15:00:00 +0000

I just returned from the Computation + Journalism Symposium in Miami, and I’m still processing the weight of what I heard. The four keynotes this year painted a sobering picture of data journalism operating under pressure—from authoritarian governments, algorithmic complexity, and information warfare. But they also offered glimpses of resilience and innovation that feel essential right now. Here are short summaries of each keynote, drafted with some AI assistance based on my live notes.

Me ready for Miami in December ☀️

When Data Journalism Isn’t Enough

Attila Bátorfy opened with 15 years of experience doing data journalism in Hungary under Viktor Orbán’s increasingly authoritarian government. His message was blunt: sophisticated data journalism hasn’t stopped democratic erosion. Despite producing groundbreaking investigations–including tracking private jets to predict and photograph secret meetings of oligarchs–the work had limited impact on political reality.

What struck me most was his catalog of how governments actively undermine data journalism: releasing low-resolution data, commodifying public information, creating datasets with bad methodology, and even sponsoring “independent” counter-data. His COVID dashboard became a trusted source (5 million uniques in 2 years, cited in dozens of scientific journals) precisely because government data was so unreliable. But he’s now moving away from data journalism, noting that audiences “don’t want facts” and that Hungarian newsrooms see it as expensive with bad ROI. The one exception? Sports and health data still cut through the noise. Attila’s upcoming work focuses on historical representatos of data, and I’m excited to learn from what he shares.

Data as Weapon and Shield

Kae Petrin’s talk on anti-LGBTQ+ data policies brought this theme of governmental manipulation home to the US context. The Trump administration’s systematic removal and alteration of CDC data—changing “gender” to “sex,” taking down youth risk behavior data, adding warnings that datasets are “threats to society”—isn’t just censorship. It’s erasure through datafication.

Petrin highlighted a crucial methodological problem that journalists fumbled: when the CDC improved its estimation methods for trans youth populations, the numbers roughly doubled. News coverage reported a “sharp rise” in trans youth, playing into “social contagion” narratives, when the real story was that better measurement revealing what was always there. Meanwhile, the demographic data showing far more trans young people than older adults raises haunting questions about where trans adults over 24 are. These are questions we can’t answer if the data disappears.

But Petrin also showed data’s dark side: data can make people legible and vulnerable.

User Agency vs. Algorithmic Power

Homa Hosseinmardi’s work offered a different kind of complexity, and reveals that the villain might not be who we think. Her research on YouTube radicalization used an innovative “counterfactual bots” method to separate user intention from algorithmic influence. The finding? Users who relied exclusively on YouTube’s recommender actually consumed less partisan content than those who actively searched and navigated themselves.

This doesn’t absolve platforms of responsibility, but it challenges the narrative of algorithms as “great radicalizers.” The real picture is messier: a small group of users consuming vast amounts of far-right content, driven largely by external links and their own preferences. When her team simulated users without agency (random video selection, first recommendation only, etc.), those bots were showng content that drifted towards the politcal center over time. The algorithm appears to reflect and amplify existing preferences rather than creating them from scratch.

The implications for journalism are significant: we may be overexamining algorithms while overlooking user agency and the broader information ecosystem that drives people to platforms in the first place.

Navigating Narrative Warfare

Yevheniia Drozdova closed with work from texty.org.ua, showing what data journalism looks like when your country is literally at war. Working from Kyiv, her team has moved beyond fact-checking individual claims to mapping entire narrative ecosystems. They train models to identify manipulation techniques, track coordinated bot networks across Telegram (where 50%+ of Ukrainians get news), and detect patterns in TikTok campaigns using shared filters and timing signatures.

Her team found 2,000 TikTok bot accounts and documented coordinated Telegram networks. They trained models to detect emotional manipulation techniques, finding that 90% of messages from Russian accounts use fear and doubt. But their work isn’t just technical detection, it’s about providing context before narratives go viral. As she put it: “Fast news only satisfies the hunger for information, while slow journalism addresses a different deficit—deficit of understanding.”

What Stays With Me

These four talks trace a dark arc: data journalism struggling against authoritarian data practices, vulnerable populations made more visible and endangered through datafication, algorithmic explanations that may miss human complexity, and full-scale information warfare requiring new methodological approaches.

But I’m also struck by the practitioners’ persistence. Bátorfy’s COVID dashboard becoming a trusted institution. Petrin tracking articles through the Trans News Initiative to understand coverage patterns. Hosseinmardi’s methodological innovation in creating counterfactual bots. Drozdova’s team prebunking narratives in real-time during a war.

These keynotes show various ways data journalism remains essential for documentation, for providing sources of truth, for understanding complex systems, and for serving communities when governments fail them. C+J’25 offered a critical moment of trans-national solidarity that I won’t soon forget.

Notes from MISI Summit on Building Trust in Climate and Science Journalism

Mon, 10 Nov 2025 15:00:00 +0000

I attended the inaugural Boston University Center for Media Innovation & Social Impact (MISI) Summit recently, where a diverse group of leaders came together to discuss the theme of “Communicating Climate.” Hosted by the Center’s Directory Eric Gordon, the event included sessions on related topics. Here’s a quick blog post about key points from one session that focused on how journalism, academia, and public media can restore trust in an era of economic instability and eroding credibility.

The start of the “Business of News” Panel

The Business of News: Audience Trust and the Future of Science and Climate Policy Journalism

This panel included Brian McGrory (BU Journalism Dept Chair, formed Boston Globe editor), Sue Robinson (Helen Firstbrook Franklin Professor of Journalism at the University of Wisconsin–Madison), and Dan Mauzy (WBUR’s Executive Editor for News).

It was moderated by Meghan Irons (investigative journalist and Professor of the Practice at Boston University), who opened with the fundamental question: how do we restore public trust when newsrooms face economic instability, eroding credibility, and an increasingly complex information landscape?

Local News, Attention, and Transparency

Dan Mauzy opened with an optimistic take: local news has a leg up compared to national outlets, and public media has an advantage among those familiar with it. He focused don transparency as a tool for building trust, particularly related to showing how journalism happens and showing your work. Dan noted a sobering statistic: “8 or 10 people have never spoken to a local reporter”—a problem that needs solving. In parallel, to ensure they are representative WBUR conducts audits on their source demographics, comparing them to local census data to ensure representative coverage.

For climate coverage specifically, the answer is making it local: to “literally bring it home to people.” Instead of abstract global warming statistics, WBUR focuses on local watering holes, jobs, heat pumps, energy bills, and solutions. Dan pointed to their Cape Cod plumbing contamination story about nitrates as an example of making climate impacts tangible. WBUR now has 3 reporters plus an editor dedicated to their climate team.

Brian McGrory acknowledged the widespread distrust across institutions, but noted an interesting pattern: crises drive attention. The Boston bombing, Trump’s first presidency, and COVID all led to subscription bumps for the Globe. Digital subscriptions actually doubled during the pandemic. “These are votes of confidence on things we do and how we do it,” Brian said. But he expressed uncertainty about how to reach people who aren’t actively seeking out journalism without alienating current loyal readers. His philosophy: “when readers see things they like, admire, trust, they will stay, otherwise they will leave.” The answer for him? “Focus on interesting, important, consequential stories and the business model will follow.”

Sue Robinson opened arguing that journalism education and newsrooms aren’t teaching key skillsets: radical transparency vis-à-vis audience engagement and restoring trust. She pointed to organizations modeling this approach: ProPublica, Seattle Times, WITF, Baltimore Banner. WITF’s 2024 election coverage particularly stood out to Sue as a model, with Q&As, pledges about misinformation and accuracy, and messaging about what’s happening behind the story.

She pushed back on Brian’s faith in the story, pointing out that “The audience doesn’t come to you—they’re being told to hate us.” Instead of waiting for audiences to find them, news organizations need to actively work on finding common values with communities.

What’s Actually Working

Sue shared how listening sessions can change hearts and minds of those reaaders who are being told to disengage from local news. Through the Trusting News project, she works with journalists who talked to people disengaged from news brands. The sessions were challening for journalists who had to listen with “devastating results that traumatized [them].” But here’s the remarkable part: after those conversations, over half of participants said it built confidence and one-third said they’d probably subscribe. The takeaway? “Bring people together around a shared topic of concern—listening and convening can change hearts and minds.”

Dan echoed this from WBUR’s experience hosting listening sessions on various topics, particularly in response to parachute journalism in the Merrimack Valley. It’s “enriching for reporters to hear what people think is overlooked, what makes them proud.” The challenge? How to scale these personal interactions.

Brian agreed that focus groups are great, creating opportunties where meeting the person behind the reporting can “change the tune of the haters”. He pushes to have journalists that get out into community regularly: “in local news we can do that.” Brian also emphasized inviting readers into the process itself—not just the meta aspects like ethics policies, but the exposing the actual work of reporting. When people understand the process, trust follows.

Sue highlighted that public media is out front on innovative approaches. She pointed to Lookout Santa Cruz as a leading example. They partner with influencers in the community itself to scale their reach, working with people who have huge groups of followers not already connected to their local media organizations. Lookout is even fundraising with non-profits together. We “need more creativity like this,” Sue urged.

Dan shared that WBUR is doing more vertical video now and reaching new audiences. They’ve done partnerships with trusted community members on niche topics, which has expanded reach and built trust. The catch? “Platforms are not our friends, so how to build loyal communities developed within those platforms is challenging.”

The Hard Questions

Are Some People Unreachable? Brian posed a difficult question about major outlets like the NYT, WSJ, and Globe: while subscribers are growing, they’re “not reaching a critical part of the country.” His question: “BUT are those people reachable, and what might be the sacrifice to reach them?”

The Talent Pipeline is Broken Sue ended on a sobering economic note: pay rates aren’t good enough, so students coming out can’t afford to go into traditional newsrooms. Instead they’re going to NGOs and corporate communications to exercise similar skills. This threatens the entire pipeline of journalism talent.

The Continually Shifting Media Ecosystem Dan raised another challenge for the future: “how to create loyalty in a fractured media space? Discovery is harder in an AI-mediated internet search world.” As platforms and AI change how people find information, news organizations need new strategies for building direct relationships with audiences.

I left with a sense of both sobering challenges and promising experiments. While there’s no silver bullet for restoring trust, the path forward Sue, Brian, and Dan pointed at focused on doubling down on transparency, community engagement, and creativity. I can get behind that.

The views out the new-ish BU Computer and Data Science building are alwasy amazing 🍁☀️💦

Notes from csv,conf,v9: Community-Centered Data Stories

Mon, 29 Sep 2025 15:00:00 +0000

The University of Bologna graciously hosted csv,conf,v9 in Palazzo Malvezzi. Accordion music drifted in through the windows while we gathered, echoing the mix of technical innovation and creative application that folks shared. Speakers from around the world shared how they’re using data not just to inform, but to transform communities. Here are gently editing notes from the sessions I attended (edited with some help from AI); any errors are probably my fault.

Photo of all the attendees at csv,conf,v9 (source: John Chodacki)

Filling Information Voids with Chatbots: Using LLMs on WhatsApp for News Access

Mathias Felipe (he/him), InfoAmazonia

In the Amazon forest, communities often face critical information voids around environmental issues, while relying heavily on WhatsApp as their main communication platform. This talk presents BOTO, an open-source chatbot developed by InfoAmazonia that delivers localized, thematic environmental information—such as deforestation alerts and wildfires—through WhatsApp. Powered by Large Language Models (LLMs) and designed with a low-bandwidth architecture, BOTO aims to make local news information more accessible in under-served areas. In this talk, we’ll discuss how BOTO was created with participatory design to empower communities, bridge information gaps, and put news directly in the hands of those who need it most.

Felipe showing what Boto looks like in action

The most striking thing about BOTO isn’t its technical architecture, others are also exploring WhatsApp Business API integration with RAG-based article search; it’s how InfoAmazonia approached the design process. Focus groups with different indigenous communities shaped every decision, from interface choices to content summaries. The result? A system where people can select their region and themes, then receive data summaries and links directly to their phones.

Felipe emphasized something crucial: their audience doesn’t have a problem with chatbots. The hesitation comes from outside assumptions about what “non-technical users” want. When you actually ask communities what they need, the solutions become clearer. BOTO soft-launched recently with official rollout planned for next month, sponsored by ICFJ. The real test will be sustained engagement, but the co-design foundation suggests they’re building something people actually want to use.

Blazing Fast Automagical Metadata

Joel Natividad

With the pervasiveness of low quality, low resolution metadata in existing Data Catalogs, and new metadata standards like DCAT3 and Croissant demanding even more FAIR metadata, how do you make it easier for Data Stewards to maintain High Quality Data Catalogs?

Natividad giving his talk in a somewhat distractingly ornate room at the palace

Natividad’s (from datHere) talk hit on every data steward’s nightmare: the metadata maintenance burden. New standards like DCAT3 and Croissant (for machine learning) demand even richer metadata: properties, data dictionaries, summary stats, frequency tables. The solution? Flip the workflow. Instead of forcing metadata entry before data upload, let people upload first and generate suggested metadata automatically. Their qsv and describegpt tools infer data dictionaries, descriptions, and 40+ summary stats. It creates a “data steward in-the-loop” approach—human oversight with automated assistance.

Most intriguingly, they’re moving toward “chat with your catalog”—custom chatbots that can surface relevant CKAN datasets when answering questions. It’s RAG applied to data discovery, potentially transforming how people find and use datasets that are published and publicly available.

Guillaume Plique (he/him), médialabSciencesPo, Paris

CSV is ubiquitous in social sciences and in the humanities. CSV data is indeed the perfect bridge between social scientists, accustomed to dealing with tabular data, and research engineers needing to process the same data. That is why SciencesPo’s médialab has been building many of its Open-Source tools around CSV files, from well-designed web apps such as Table2Net to convert tabular data into graph data, down to powerful CLI tools such as minet to collect data from the web or xan to process tabular data using constrained resources.

Plique, and the lab where he works, are _all_ about CSV

Plique delivered a love letter to CSV. His argument is compelling: CSV is affordable, understandable, and free. It functions as a bridge between researchers, students, and engineers. It doesn’t require complex compute resources and naturally encourages data sobriety. The médialab’s tool ecosystem is impressive—table2net for web-based CSV-to-graph conversion, takoyaki for clustering CSV data to find clerical errors, minet for CLI-based CSV processing, and xan for visualization. All built on the principle that tabular data is inherently more comprehensible than JSON hierarchies. His closing advice: “If you need random access, use SQLite. Otherwise, CSV.” It’s a reminder that sometimes the simplest tools are the most durable—you’ll still be able to open a CSV file in 50 years.

Keynote: Developing tech with community: the example of Open Data Editor

Sara Petti, Open Knowledge Foundation

Open Data Editor is a desktop application specifically designed to help people detect errors in tables. It has been developed in constant interaction with the community from a very early stage. These interactions helped us understand what was really helping the community and what not, and especially made us aware of how much the use of such a tool could actually be helpful in increasing data literacy.

Petti dove into how they built a data cleanup and analysis tool with partners

Petti’s keynote was a masterclass in community-centered design. OKFN initially tried to build “an app to do it all”—review data, prepare it, map it, write articles. The scope was overwhelming and misaligned with user needs. The breakthrough came through extended pilot testing with organizations like ACIJ in Argentina and StoryData in Spain. This led to dramatic simplification and focus. The second round of pilots with five different organizational types revealed something crucial: researchers worried about changing metadata, navigation and design were hugely impactful, and people don’t read documentation.

Key lessons that resonated: language matters enormously (“validation,” “execute” carry technical baggage), there’s only so much you can do to make highly technical tasks accessible, and many challenges are fundamentally non-technical. Most importantly—iterate based on feedback, keep things simple, and don’t be confused by the latest tech. The project pivoted from React to a more sustainable framework, emphasizing the importance of maintainable code over popular technologies.

Democratizing data: data literacy for community action

Emily Zoe Mann (she/her/hers), University of South Florida Libraries

This presentation will share the results of Data Literacy for Community Action, a year-long pilot in the community of St. Petersburg, Florida, USA that provided data literacy to non-profit groups and interested members of the community, with a focus on using local data sets and data related to social determinants of health.

Mann shared lessons from hands-on data workshops with library patrons

Mann’s project exemplifies academic-community collaboration done right. The partnership between University of South Florida, the public library system, and a philanthropic health equity organization created something none could achieve alone. The workshop structure was brilliantly simple: introduce a concept, share a local dataset demonstrating it, bring in a local person to discuss real-world application, then run group activities. Her red tide sonification example from South Florida shows how hyperlocal data becomes immediately relevant when paired with familiar environmental experiences.

One insight that stood out: attendees came with vastly different question types, from basic to nuanced. Mann’s library background prepared her for meeting people at all skill levels, but it reinforced how “data literacy” isn’t a single skill—it’s a spectrum of capabilities that communities develop over time. The flexibility lesson is crucial for similar pilots: stay responsive to what people actually need rather than predetermined curricula.

Keynote: Clara Jiménez & David Fernández Sancho

Maldita.es Foundation

Clara Jiménez Cruz is the co-founder and CEO of the Maldita.es Foundation, a leading organization in the fight against misinformation… David Fernández is the current CTO of the Maldita.es Foundation, where he leads the development of innovative digital tools to combat disinformation.

The team shared a timeline of work they’ve done

This keynote hit hardest, perhaps because of recent events. Jiménez and Fernández positioned disinformation not as isolated false information, but as structured narratives—emotionally engaging, rationally grounded, amplified by trusted figures, global in scope. Their Valencia floods case study was chilling: the same narrative emerging simultaneously from Russian sources, far-right groups, and anti-vaxxers, all attacking institutional trust. The message was consistent: “You can’t trust government, trust me.”

Their AI-assisted approach uses Wikidata entity extraction to cluster claims, with human-in-the-loop narrative association. They’re tracking 600+ claims monthly—a scale that demands automation while recognizing AI’s limitations in understanding social context.

The most powerful moment was Jiménez’s declaration: “data as a means of resistance.” In an environment where “the internet is becoming more dangerous on purpose,” systematic documentation and analysis of disinformation patterns becomes a form of civic defense.

Data without borders: Straight out of spreadsheets and into the streets

Michael Brenner (He/Him/His), Data4Change

At Data4Change, we work with communities around the world to explore how data can be collected, understood and shared in radically different ways. This talk takes you through a series of surprising, community-led projects that didn’t begin with data, but with burning questions our community partners wanted to find answers to.

Brenner’s examples were extraordinary—from addressing intimate partner violence in Uganda’s LGBTQ community, to flute music rendering Ethiopian census data audible to the blind. Each project started not with data, but with burning community questions. I mention many of them in my Community Data book.

The data murals project with Haki Data Lab in Mathare particularly resonated. Rather than imposing external visualization methods, they recognized murals as a standard local communication mode, then created input visualization murals with string and stencils. The “power portraits” methodology for the Equal Victims project was brilliant—co-created visual representations that function simultaneously as art and data points about lived experience, avoiding retraumatization while enabling systematic documentation.

Brenner’s language shift is important: moving from “data literate” toward “data curious, data critical, and data confident.” It reframes the relationship from deficit-based (lacking literacy) to asset-based (developing capabilities).

Keynote: Fifteen Years into the Open Data Movement

Giorgia Lodi & Andrea Borruso

Fifteen years into the open data movement, we take the stage as an unusual duo to reflect, provoke, and laugh about where we’ve landed. A dialogue, shaped by friendship, a few successes, many failures, and a bit of fun—think less keynote, more late-night talk show.

This closing keynote was part celebration, part wake-up call. Lodi and Borruso’s Italian open data examples were both hilarious and heartbreaking—ministry electric charging station data published with beautiful graphs but completely locked down, forcing community scraping to make it actually useful. Their analysis cuts deep: following the letter of the law isn’t enough when datasets lack description, remain readable only by specialists, or simply don’t exist (particularly around gender-based violence). Communities get pushed out by technicalities, formalism, and missing data.

A case study about electric car data exemplifies the problem—officially “open” data published as unusable pivot tables instead of raw, machine-readable records. When the ministry finally released proper data, it came with semicolon separators and complete messiness. But their vector geography data example shows what’s possible: despite delivery delays and nested zip file complexity, raw land ownership data triggered new software, plugins, and services precisely because it was genuinely machine-readable.

Their conclusion resonated throughout the conference: exploit technology to support data management, think about machine-to-machine AND machine-to-human AND machine-to-community interfaces, and recognize that technical solutions alone can’t address fundamentally social challenges.

Reflections: Data as Community Practice

Walking through Bologna after two days of talks, I kept thinking about the accordion player who inadvertantly soundtracked the opening day. There’s something fitting about that mix of structure and improvisation, tradition and innovation. These presentations collectively argue for a fundamental shift in how we approach data work in and with community. We must shift from extraction to engagement, from literacy deficits to community assets, from technical solutions to socio-technical systems. Whether it’s BOTO’s co-designed news delivery, Data4Change’s participatory data projects, or Open Data Editor’s community-driven development, the most compelling projects start with community questions rather than data capabilities.

I gave a keynote related to my _Community Data_ book (source: Marco Cortella)

The CSV theme threading through many talks isn’t just about file formats—it’s about accessibility, sustainability, and democratic participation in data practices. In a world increasingly dominated by complex AI systems and proprietary platforms, there’s something radical about insisting that the most important data work can happen with simple, open, maintainable tools. As disinformation accelerates and institutional trust erodes, these community-centered data practices feel more urgent than ever. The question isn’t just how we make data more accessible, but how we make data work serve community needs rather than institutional convenience.

ACH Keynote on "Civic Jugaad - Reclaiming Digital Innovation for Social Good"

Fri, 01 Aug 2025 15:00:00 +0000

The following is a condensed write-up of a keynote talk I gave at the ACH 2025 conference (Association for Computers and the Humanities)

I grew up as an 80s kid watching MacGyver use some duct-tape to save a city from destruction. Frequent visits to India shows me real-world street equivalents, where this kind of hacked-together solution has a name: jugaad (pronounced “jugaar”). Jugaad is the Indian culture of improvised problem solving. This can range from repurposing a motorcycle to run an irrigation pump to mashing an air-conditioning unit into a wall it was never designed for. Looking back, it is obvious that these experiences played not small part in my decision to study engineering in college.

Jugaad in action in India.

The term was briefly taken seized on by business innovators in the late 2010s, confusingly tied to both orientalism and politics of self-emancipation, and then faded again in the 2020s under criticism as “shortcuts”. However, I believe this jugaad mindset has something urgent for those of working in digital humanities, public interest tech, and journalism right now. Big Tech is consolidating and concentrating power and resources in the hands of the few, using “innovation” to describe tired attention-based monetization strategies. In parallel, the current U.S. regime is attacking universities and their funding streams. We live in disorienting times for those like, who grew up with relatively stable patterns of public investment in knowledge for the common good.

Local needs, contexts, and constraints drive jugaad-style innovation; those operating under harsh situations figure out a way to make things work. They connect people, amplify resistance, build with not for, and study systems of power. I want to tease those four points to structure my argument about the “civic jugaad” we need right now, bringing the improvisational, community-rooted ethos of jugaad into digital humanities and public interest technology practice.

1. Connect people

Facism and authoritarianism thrives on isolation and “othering.” One small counter-move is building tools that cross boundaries instead of reinforcing them.

Journalists are at the forefront here, constantly connecting their readers to each other via the stories they choose to highlight. A lightweight example from my own work on interactive journalism is Our Cup, a static website I rebuild for every World Cup that recommends matches to watch based on the largest local immigrant populations in your area, then connects you to food, music, and news from those communities. It’s genuinely a “quick and dirty” project, deployed as a static site on GitHub Pages. There’s little infrastructure and no grant funds, just an attempt to use a shared cultural moment to reduce distance between neighbors.

El Surtidor built a chatbot letting readers interact wth a women in prinson.

A more inspiring and ambitious example comes from El Surtidor, a Paraguayan news outlet that built a chatbot named Eva out of hours of interviews with a woman awaiting sentencing for international drug trafficking. With Eva they are trying to open up public conversation on a stigmatized issue: women incarcerated for smuggling. It has logged more than 15,000 interactions with the public, all built on a modest $5,000 grant. The team even considered a WhatsApp bot before landing on something cheaper to build and maintain. That’s the jugaad approach in action: focus on the local context, and just make it work with what you have.

The key lessons for practicing civic jugaad to connect people?

Design for local contexts, where the problems at hand and resources are well understood.
Make it work – if you have multiple approaches, just use the least complex options.

2. Amplify resistance

There are groups around you already fighting against marginalization they experience. You can innovate with groups in neglected areas to create higher potential for social impact.

Some of the most important public interest technology happens in areas mainstream institutions have simply neglected. My good friend and colleague Catherine D’Ignazio’s research with anti-feminicide activist groups is a good example. Official records systematically undercount gender-related killings of women and girls, so activist groups around the world build their own “counterdata” to fill the gap. Our Counterdata Network grew directly out of watching that struggle: we build bespoke, lightweight machine-learning classifiers, in consultation and collaboration with each group, that score and cluster news stories and deliver alerts by email or web dashboard. It’s essentially a custom-built replacement for Google News Alerts, tuned to each community’s specific needs. Over the last five years almost 50 groups have used it regularly to monitor human rights and civil rights violations.

System diagram and screenshot of Data Against Feminicides tool created by the Counterdata Network.

A smaller-scale version of the same instinct is a local protest mapping tool I built so newsrooms can see and embed maps of nearby demonstrations in their coverage. Many are criticizing the coverage of mass protests in response to Trump’s destruction of long-standing government programs as underwhelming. This website helps local journalists create compelling visuals to discuss incidents and patterns of resistance, perhaps pushing back on episodic vs. systemic framing issues so often encountered in this type of reporting.

The key lessons for practicing civic jugaad to amplify resistance?

Rely on and respond to problem statements the local community has developed.
Build by putting together existing pieces.

3. Build with, not for

Digital tools for the public good can’t be built with standard deploy-oriented mindsets. Co-creation gives us the best chance at relevance, impact, and empowerment

I first heard the phrase “build with, not for” from Catherine Bracy during her time at Code for America. It has stuck with me ever since, in part because it focuses on the language technologists use. We talk about “deploying” solutions, as if communities were terrain to be occupied rather than partners in the work. Recent research on “AI for good” partnerships bears this out: funders’ agendas routinely override community needs, technical metrics don’t align with social goals, project don’t meet expectations of the target audience, and technology gets centered ahead of the actual social.

One counterexample in the same space is my Northeastern colleagues’ AI-EP project. They brought together school systems, students, researchers, and parents to identify points of confusion around their children’s Individualized Education Programs (IEPs). These are legally mandated documents detailing education supports their children can receive, but are often dense and hard to understand. Only after hearing those concerns did the team build a translation engine and chatbot to answer parents’ specific questions. It’s a small pilot, still iterating, but it’s a model for sequencing: listen first, build second.

The AI-IEP team and some of the artifacts they created.

An off-screen example comes from my own work on creating participatory data murals I’ve helped communities paint in public spaces with my wife, Emily Bhargava. Data is increasingly front and center in civic participation, but spreadsheets and charts aren’t universally accessible forms; the materials and forms of data change who is able to work with it. Borrowing from rhetorical power of community art, and related participatory processes, we facilitate communities finding a story in the data about them, designing a visual to tell that data story, and painting it together in a local setting as a public mural. This builds on the longstanding power of murals to impact public conversation and policy. Similar ideas are in my recent Community Data book.

The key lessons for practicing civic jugaad to build with, not for?

Engage people with participatory design methods and distributed ownership of tools and processes.
Iteratively learn through real work experimentation, while acknowledging the potential for harms.

4. Study systems of power

Communities trying to use technology for civic good are often subject to the power structures those contain. Peeling back to the layers can present opportunities to reclaim power.

The last principle borrows a phrase from anthropology: “studying up.” Anthropologist Laura Nader argued back in 1972 for turning ethnographic attention toward the culture of power and affluence, rather than only studying the powerless. In technology research, that means training our tools on owners, corporations, and institutions instead of only on the communities affected by their technologies.

A wonderful example from my colleague Wonyoung So is Landlord Tech Watch: the team builds simulated tenant screening reports to reverse-engineer how algorithmic scoring systems embed racial bias into rental decisions. These systems, relying on criminal records, eviction histories, and credit scores, disproportionately exclude people of color as tenants. Their findings have directly supported tenants’ rights advocacy and a 2024 legal settlement with a major screening vendor.

The Landlord Tech Watch team created explanatory websites to show their findings.

With another research hat one I’m co-PI of the Media Cloud project, which offers a directory of global online news sources and a searchable index of almost 2 billion stories. Our tooling operates as public data infrastructure, supporting study of media narratives pushed out by various parties. It allows anyone to dig into media framing and shifts over time.

The key lessons for practicing civic jugaad to study systems of power?

Identify any existing oppressive power structures around your innovation and work to dismantle them.
Think about how your methods can help other “study up.”

Civic jugaad as a response

I offer this concept of “civic jugaad” as a hopefully helpful way to think about aligning your/our work to respond to the challenge of the moment (summer 2025). This is a model that western research settings can learn from. We’re not going to out-resource the platforms consolidating power around us, and we can’t wait for research funding to return to its old levels. But we can borrow from a tradition built for exactly this kind of scarcity: pay close attention to local needs and constraints, connect rather than isolate, amplify what’s been neglected, build alongside the people affected, and turn our tools toward studying power rather than only the powerless.

Civic jugaad is still very much a work in progress; more provocation than framework at this point. But if any of these examples resonate with your own work, please reach out and let me know.

Context, needs, and constraints drive jugaad innovation.

Protest Mapper: a new tool for journalists covering protests

Thu, 08 May 2025 15:00:00 +0000

As protests against the Trump administration spread, journalists are covering local rallies — but putting them in context of larger movements can be a challenge. To help, I built a simple tool that lets you search for recent protests near you and create an embeddable map you can use in your reporting. Try out Protest Mapper.

A map of the more than 400 protests in eastern Massachusetts since the start of the year.

Media Attention to Protests

One of the responses to the nationwide April 5 “Hands Off” and May 1 Mayday rallies was shock at the perceived lack of coverage from mass media outlets. Critics pointed to a lack of both print and digital coverage, arguing that it didn’t reflect the sheer volume of people that came out on the streets.

A sample of headlines critiquing media coverage of the April 5 “Hands Off” protests.

At the same time, local and regional media did show a noticeable bump in coverage. A query for articles mentioning both “hands off” and “protest” in the Media Cloud online news archive shows almost 400 matching stories on the 5th.

A Media Cloud query for “‘hands off’ AND ‘protest’” shows a peak of coverage in state and local papers on April 5.

Yet it appears that local reporters are covering protests in their area, but not often connecting them to larger movements. That might be because coalitions like #50501 aren’t as well known as unions and long-standing activist groups; they don’t have communications people with long-standing relationships to journalists.

Finding & Evaluating a Public Protest Dataset

One approach to help reporters make those links for readers, and put individual events in a broader context, is to use data about local protests. Connecting this weekend’s rally to events over the last few weeks might connect dots for audiences that are seeing public displays of resistance. I wondered if I could quickly map protests in my area based on existing data sources.

I first reviewed existing data sources to see what was available. A quick summary of what I found:

The GDELT project appears to have massive scope and has a “Protests” event category, but I found it too hard to access and use, with many broken links and inconsistent information.
The Carnegie Protest Tracker seems to focus on broad movements, not individual events.
LiveUaMap seems well established, with lots of social media results, but I found the interface challenging to use and they charge for access.
Amnesty International’s Protest Map focuses on police violence in response.
ACLED (Armed Conflict Location & Event Data) appears to be a well staffed non-profit that makes data easily available, updated once a week.
The Harvard Ash Center Crowd Counting Consortium (CCC) appears to be an academic project that is updated once a month.

This led me to focus on ACLED and CCC, since they are both regularly updated and easily accessible/usable. Each includes a list of protests and related manually curated metadata. They include protests about any topic.

My next goal was to evaluate these datasets. One metric that came to mind was to search for media coverage about protests in a fixed place and time, and then to compare that to the datasets at hand. I decided on Massachusetts in February, a context I know well, and pulled datasets from both Media Cloud and ACLED. To find media stories I queried Media Cloud for relevant anti-Trump/Musk protests with this query:

article_title:(
(protest* OR rally OR strike OR boycott OR walkout OR demonstrat* OR vigil)  
AND 
(Musk OR Trump OR Tesla OR closure OR federal OR university OR Israel OR Palestine OR cuts OR speech OR student OR research OR congress OR ICE OR agency OR USAID OR NOAA OR NSF OR NIH)
) and language:en

This discovered 76 related stories. I then coded them by hand to indicate if they were about protests in Massachusetts, identifying 41 stories about anti-Trump protests in the state from Media Cloud. In parallel, I pulled February data for the US from ACLED and filtered by the admin1 column to just Massachusetts events, identifying 51 entries in their database. To evaluate the two, we then reviewed every story in the Media Cloud corpus to identify which ACLED event it was about (linking it by the unique event_id_cnty provided from ACLED). Just two protests were in the Media Cloud dataset that did not have a related ACLED entry, both small small events reported by university student-run news sites. 95% of the sample I pulled from the basic Media Cloud query were already in ACLED, leading me to believe it was a trustworthy source for a list of protests. You can review the coded evaluation data.

A Mapping Solution

Dataset in hand, I decided to create a simple tool that would allow journalists to embed maps of recent local protests in their stories. The map at the top of this article is one example. Visit https://dataculture.northeastern.edu/local-protest-map/ to try it out yourself. You can filter by location, timeframe, and customize the map a little.

Tools like these require ongoing support, so I’ve made some decisions to try and minimize maintenance while maximizing utility. Here is a list of design decisions, concerns, and commitments:

I’ve mimicked the well known DataWrapper workflow to try and make it familiar and usable for data journalists. At the end you get an embed code (HTML iFrame) you can copy and paste into your webpage, just like I did above
This is deployed as a static HTML/CSS/JS site with no database or server compute power required, written with the open-source Svelte UI framework.
Some protests are marked in the exact same spot, for instance in the center of a city. This makes them very hard to see on the map. For these events I’ve a small random bump to the location in order to make them more visible. That is noted in the popup for impacted markers.
The code is open source on GitHub for full transparency.
I’m hosting the tool on GitHub Pages so I don’t have any ongoing expenses.
IFrame embeds can be tricky and limited, so the tool also offers the option to download a static image of the map in .png format.
You can pick between the ACLED (updated weekly) and CCC (updated monthly) data. For now I plan to manually pull the updated data each week, but hope to automate that later.
I’ve used a light background map base layer from Stadia Maps. This means it can’t support more than 200,000 hits across all uses. If this somehow magically becomes very popular I can easily change that via the Leaflet mapping library.

Give it a try, share it, and let us know what you find in the comments. Hopefully this tool and explanation will help you and others include data about the wave of protests to your reporting and add context to individual events you’re covering.

Originally published on Storybench

AI for Good: Five Key Principles for the Pro-Social Sector

Wed, 26 Feb 2025 02:00:00 +0000

Artificial Intelligence (AI) is reshaping industries across the board, but its impact in the pro-social sector is worth digging into more to understand emerging norms and challenges. Our typical examples stem from years of work on “civic tech”, “public interest tech”, “data science for good”, and related efforts. Most of these focus on challenges like species conservation, disease outbreak surveillance, deforestation tracking. Who doesn’t want to save threatened zebras? I describe these as accepted good: problems that most people agree need to be solved.

But there are problems less people agree are for the common good. What about AI projects that support for gender equity? Or projects working against surveillance capitalism? Not everyone agrees that these are projects that support the common good (though I do). These could be collectively called AI for contested good, and that’s the concept I want to dig into. Specifically, what are some key principles and methods to use when designing AI projects that support work on contested good in the pro-social sector.

This post introduces that thinking and some related examples. This line of thinking stems from an invited talk I gave recently at the UTSA MATRIX consortium’s AI Spring School, which offered an opportunity for me to spend a little more time conceptualizing and formalizing thoughts about these issues for the highly technical audience.

I took a quick selfie from the stage while the audience was doing a quick chat about projects they would describe as “AI for good”

1. Center Underserved Issues

Instead of focusing on well-studied and understood problems, turn your attention to innovate in neglected areas. This presents an opportunity for significant social impact.

This is particularly relevant in the context of feminicide, where mainstream institutions fail to collect and analyze data effectively. As Catherine D’Ignazio reminds us in “Counting Feminicide”, “What isn’t counted doesn’t count.” Over the last few years she’s been working with groups that create “counterdata” on feminicide to fill gaps in official records on gender-related killings of women, girls (cisgender and transgender). The work of these civil society groups to produce datasets about this violence supports advocating for policy, enforcement, and garnering public attention.

Producing this data is hard, both logistically and in regard to the human toll. Global groups struggle to create datasets of gender-related killings with ill-fitting tech. This creates significant data gaps and emotional trauma. As one partner noted,

It is not easy to read ten cases of feminicide and put them on a table, disaggregate them, have to put a name, age, circumstances and all that detail, without it affecting you emotionally.

In response, I’m part of the the Counterdata Network that works with groups to create bespoke machine learning classifiers that score news stories, cluster them by events, and deliver via email/web. Over 30 groups use them daily to monitor human rights and civil rights violations via news reports. We co-designed the system to help, basically replacing google news alerts with a custom-built open-source data pipeline. The network is growing across multiple topics: feminicide, criminalization of pregnancy, business human rights violations, and more.

This type of work emphasizes the need to look for issues that are systematically ignored and listen to voices that are unheard or silenced. Innovate in neglected areas with higher potential for social impact.

2. Build With, Not For

Technology should not be imposed on communities but co-created with them to ensure relevance, impact, and empowerment. This participatory approach contrasts with the traditional top-down model of technology deployment.

I first heard this phrase “build with, not for” from Catherine Bracy during her time at Code for America. She used it to emphasize the idea of co-creating technologies with government and community, instead of creating top-down solutions. As technologists, even our language is embedded with the opposite approach – consider the ideas of “deploying” vs. integrating” tools.

One response to challenges around Individualized Education Programs (IEPs) in the U.S. present a good government-related example to learn from. IEPs are plans created by public schools to offer special education services to children in need of more specialized care. They are created collaboratively across the school system, teachers, and parents. These legally mandated plans are in use for around 8 million students across the US. However, many parents struggle to understand the services their child is offered in their complex IEP.

The AI-EP team used interviews to identify common concerns parents had about IEPs. Through interviews, the team identified key challenges and built an AI-powered translation engine and chatbot to help parents navigate complex documents in a personalized way. The features include translation, question answering, and more. This creates a new AI-supported method for helping parents understand a complex document that is written in legal language using complex terms.

Projects like this remind us to focus on participatory design and iterative solution that integrate feedback from the impacted community. Co-creation ensures relevance, impact, and empowerment.

3. Study Up

AI can be a powerful tool for interrogating power structures, flipping traditional methods to focus on the culture of power rather than the culture of the powerless.

Anthropologist Laura Nader’s concept of “studying up” is essential here—using AI to investigate those with authority rather than those affected by their decisions. She argued that the methods of traditional anthropology, which typically studied groups with less power in social structures, should instead be flipped to study mechanism of power like colonization. Chelsea Barabas’ work on bringing this idea to algorithmic fairness has been highly influential on my thinking here.

A striking recent example is the Landlord Tech Watch project. Landlords increasingly rely on automated screening tools that use criminal records, eviction histories, and credit scores to assess tenants—often reinforcing racial and economic discrimination. By simulating tenant reports, this project demonstrated how these technologies embed bias and provided advocacy groups with concrete evidence for policy change (So, 2022). They are studying the tools of those with power in the rental market to understand disparate impacts and work to fight against them.

This line of work studying up in AI systems identifies that we should prioritize identifying and interrogating power structures so that we can then flip traditional research methods to challenge systemic biases. We can leverage advanced AI tools to investigate those with power.

4. Look for Quick Wins

Does every problem need a complex AI solution? Often, small, targeted interventions offer more sustainable impact than complex, resource-intensive projects.

Journalists are often at the cutting edge of leveraging new technologies, and considering the ethical implications for their field… genAI is no exception. Their work serves as a model for building small, targeted, interventions that offer quick wins with lightweight AI integrations.

One compelling example is the Eva chatbot developed by the Paraguayan news outlet El Surtidor. Paraguay has many women in prison, or awaiting sentencing, for drug smuggling. The team wanted to open a door into the taboo issue, so they trained an AI chatbot based on interviews with women awaiting sentencing for international trafficking. Readers could interact with the bot to ask quesitons about their cases, how they got caught, and more (with appropriate guardrails). It quickly reached more than 15,000 interactions with the public. Eva helps reframe the public narrative around drug-related crimes by challenging stereotypes and stigma.

Quick-turn projects with tightly defined use-cases like this help us consider what kind of new stories with social impact we can create, focusing on lightweight, adaptable solutions that offer immediate impact. How can AI help you make impact on a small but well defined idea?

5. Consider Hidden Risks

GenAI technologies are built to look magical, and often produce good-looking results at first pass. Scholars are adopting them to support media research, without fully considering the very real social, financial, and ecological impacts.

More and more often I’m seeing genAI used for computational tasks that are well solved with regular ML: entity extraction, geographic ID, etc. There are a set of already defined problems that we don’t need genAI for. Ongoing work by academics and journalists are uncovering the true costs of the massive investment in LLMs. Like Bitcoin, with genAI there’s a very real future where we have blackouts because of computational power usage.

An example is a recent study I worked on with a team from The Scope. We wanted to analyze articles in the hyper-local news source to assess content diversity via geography, topic, and quotes. We could have used genAI for all of this, as others have, but for geography and entities we used existing off-the-shelf solutions because they are well validated, produce reproduceable results, are free to deploy and use, and are open source (I’ve helped build both). However, without well-performing and easy to use existing solutions for quotes, we decided to try the genAI black box. We iterated on prompts against a manually coded list of quotes and ended up using ChatGPT v3.5, which was easy to integrate with, and cost little ($6 total). Quote extraction and attribution (via pronouns) worked well in the end, allowing us to do gender-based analysis. That was only possible at scale thanks to ChatGPT. In the end we were able to do a cross-sectional analysis along those three dimensions of content to help the editors audit their own content, and plan for any changes with that data in mind.

Our work on that study shows how genAI integrations should be carefully designed to only be used where necessary. That can help reduce areas where results are not easily reproducible and justify work towards algorithmic interpretability to mitigate unintended consequences. Companies creating LLMs are deliberately obfuscating the hidden power and environmental costs, and don’t have a history of being good partners to the media industry.

The five principles outlined here—centering underserved issues, building with communities, studying up, looking for quick wins, and considering hidden risks—are foundational for using AI responsibly in the pro-social sector. If you’re interested in AI’s role in social impact work, I encourage you to explore these principles in your own projects. These kind of approaches and examples can help ensure that we model how AI technologies might serve as a tool for empowerment, rather than perpetuating existing inequalities.

Nebraska Data Art Jam: Beyond Visualization

Tue, 14 Jan 2025 02:00:00 +0000

I’m just back from delivering the keynote talk at the Data Storytelling Workshop and Data Art Jam, which brought together a fascinating mix of creative thinkers, from scientists to journalists to artists. Hosted by Dan Novy at the Johnny Carson Center for Emerging Media Arts, this interdisciplinary event highlighted new ways of communicating through data, bridging art, technology, and storytelling. Here’s a look at the insights and projects from the other speakers at the event:

Novysan introducing the event by showing how the R Studio hammer makes everything look like a nail, leading to cookie cutter data vis results

Dr. Laura Guertin’s Data Quilts

Dr. Laura Guertin from PennState Brandywine opened the event with her deeply personal and innovative approach to data storytelling. As a trained marine geologist, she had spent much of her career in STEM education. But it was an object from her grandmother—a Kenmore sewing machine—that set her on a new path. Laura began quilting and soon realized that quilts could tell stories, including those based on data. In 2018, she joined a team working on “coastal optimism,” an initiative that aimed to counter the often bleak narratives of environmental decline (for me this evoked a solutions journalism approach). Instead of focusing solely on doom and gloom, she highlighted solutions: open levees in Louisiana that balance storm protection with environmental exchange, and offshore Christmas tree pens used to reduce coastal erosion.

Dr. Guertin speaking about her data quilts

Inspired by these stories, Laura returned home and created “Stitching Hope for the Louisiana Coast,” her first data quilt. The response was overwhelming, and she began exploring more ways to translate climate data into fabric arts. This relates to now-popular trends like temperature scarves—knitted or stitched rows of fabric, each representing a day’s temperature—and expanded the concept to include themes like air quality and rainfall.

Her most recent work connected to a research expedition on the open ocean. Serving the role of education and outreach on the boat, Zoom interactions with kindergarteners about measuring cloud cover inspired a quilt about the related data they were producing on-board. She shared daily photos of the sky with the kids, and the kids collected their own data to compare. The quilt she later made when back on shore, using local South African fabrics, visually represented cloud cover with colorful trapezoids, connecting scientific observation to physical materials and local culture. Laura’s quilting projects demonstrate how creative approaches to data can evoke curiosity and make complex information more accessible to a wide variety of audiences. As she noted, quilts have a unique power to draw people in—they feel warm and inviting, not intimidating. Explore more of her work on Dr. Guertin’s website.

Matt Waite on Data Journalism with AI

Later, Matt Waite, a journalism professor at the University of Nebraska-Lincoln, took the stage with a compelling argument for using artificial intelligence in journalism—but only for the boring stuff. He described AI as “the dumbest intern you could have,” perfect for automating repetitive tasks while leaving the creative work to humans. During a recent sabbatical, Matt explored how generative AI could support data journalism. His project focused on using R to integrate with Google Gemini, automating the summarization of county-level data to create personalized, map-based visualizations. He emphasized how this could make journalism more efficient, particularly for local stories.

Prof. Waite showing a demo of his work in R Studio on data journalism augmentation with AI

However, as he demonstrated, genAI isn’t foolproof. When he added a fact-checking bot to audit the summaries it produced, it caught errors, such as incorrectly describing a zero value as “no change.” Asking it to add “local context” to headlines introduced even more opportunities for inaccuracies. For Matt, these missteps underscored the importance of starting small with AI. Rather than aiming for big, transformative projects, he advocated for using AI to handle the mundane tasks that free up human journalists to focus on what truly matters: creativity and critical thinking.

Robert Twomey on Data and Art

The final presentation of the morning came from Robert Twomey, an artist and engineer whose work explores the intersection of data and art. Robert shared how his background in both science and painting informs his practice, with a focus on observation and representation. He drew inspiration from his MFA advisor, Natalie Jeremijenko, and projects like the “Suicide Box,” which used motion detection to document unquantified data on the Golden Gate Bridge. Robert pointed to how Jeremijenko’s work asks thought-provoking questions that are still core today: What counts as data? How can art challenge the ways we encounter and understand it?

Prof. Twomey connecting the dots between traditional data representation and painting as observation

He showcased projects that ranged from data physicalizations to generative art. One example was Sabrina Raaf’s Translator II: Grower, which visualized CO₂ levels by “painting” them on a gallery wall. Another, Joel Ong’s Aeolian Traces, used human migration data to generate fan-driven wind in a physical installation. Robert’s own work included a chatbot he created in 2005 based on interactions with his grandmother, who had been diagnosed with Alzheimer’s. The chatbot became an interactive data portrait, offering a poignant exploration of memory and technology. In another project, Garbage Cubes, Robert compressed physical artifacts into sculptural representations of “irreversible data compression,” challenging traditional notions of how we store and interpret information.

Throughout his talk, Robert emphasized the role of artists as experts in making things experiential. He pondered genAI’s ability to have explanatory power at all.

My Own Take on “Data Beyond the Visual”

As the opener before these brilliant folks, I took the opportunity to challenge the audience to rethink how we engage with data in community settings. I laid some groundwork by focusing on how now-traditional data visualization methods, developed in fields like science and business, often fail to meet the needs of pro-social domains like museums, libraries, and grassroots activism. These settings require tools that prioritize participation, empowerment, and justice.

Pulling from my book, I highlighted examples like participatory data murals and civic data sculptures, to demonstrate how creative approaches can reflect data back to communities, fostering deeper connections and driving action. I also previewed work on Data Drums, a climate change sonification composed for a Brazilian drum troop, which is set to debut this spring (more to come on here soon).

Through evocative examples and practical insights, I urged the audience to go beyond default charts and graphs, activating the latent creative capacity within communities to make data work more inclusive, impactful, and joyful.

Those talks set the scene for a hands-on 2nd half of the day, where groups spent hours designing and building data sculptures with copious amounts of craft materials, large and small, that Novy had supplied. The open space made collaboration easy, and groups dove in with impressive energy to the new activity.

Collage of teams working on their data sculptures.

Overall, the Data Art Jam was a testament to the power of interdisciplinary collaboration. From quilts that tell stories of resilience, to AI tools that simplify journalism, to art that questions the very nature of data, the event showcased the endless possibilities for reimagining how we communicate information. It left attendees inspired to experiment, adapt, and push the boundaries of what data storytelling can achieve.

Note: this post was drafted from my personal notes into a narrative using genAI tools and then reviewed and editing by hand

Journalism papers at IEEE Vis 2024: selected highlights

Fri, 03 Jan 2025 05:09:00 +0000

IEEE Vis is the premiere global academic computation-focused visualization conference. Each year about a thousand people attend, presenting around 100 papers. Some of those touch on data visualization and storytelling in journalism settings, bringing deep technical thinking and research into a domain that connects to the practicing newsroom. In this post I thought I’d summarize a few selected papers I saw at IEEE Vis 2024 that might be relevant to data visualization experts and data storytellers who work in journalism.

Structural influce model from Fu & Stasko, 2024

In “More Than Data Stories: Broadening the Role of Visualization in Contemporary Journalism”, Yu Fu and John Stasko identify ways data visualization can expand its impact in journalism beyond storytelling. They helpfullly lay out a research agenda to push other academics to work at this intersection. It might be useful to data journalists to look at their list of newsroom values data visualization can support, and offer some language for working with academics (like me) to develop co-created research projects.

An example Brazilian news data visualization Brito & Ferreira, 2024

In “Visual and Data Journalism as Tools for Fighting Climate Change” authors Emily Brito and Nivan Ferreira use media coverage of the terrible floods in Rio Grande do Sul (Brazil) as a case study in how data journalism can be a tool for fighting disinformation and climate change. Key elements they focus on include pursuasive communication, risk communication, and challenges realted to literacy and misinformation. Their paper offers valuable insights and examples for data journalists working on climate catastrophes.

A visual representation of the protocol used in the study by Rogha et al, 2024

In “The Impact of Elicitation and Contrasting Narratives on Engagement, Recall and Attitude Change with News Articles Containing Data Visualization” authors Milad Rogha , Subham Sah, Alireza Karduni, Douglas Markant, and Wenwen Dou explore the “you draw it” approach to interactive data storytelling. They experimentally explored the impacts of asking readeres for their opinions on attitude change, recall and overall engagement. They found that while showing showing contrasting visual data narratives did increase engagement, they didn’t improve recall nor change attitudes more. This kind of experimental research is critical to understanding how to build more effective data visuals to engage and impact the public.

Intro slide about Wang et al, 2024

The paper “Visualization Atlases: Explaining and Exploring Complex Topics through Data, Visualization, and Narration”, by Jinrui Wang, Xinhuan Shu, Benjamin Bach, and Uta Hinrichs, offers a taxonomy of “atlases” of data. This might be particularly useful to data journalists creating exploratory interfaces that let the reader browse through large datasets related to (more explanatory) stories. They found that atlases tend to combine data, narration, and structured navigation to make complex topics accessible. Speficially, many use design templates and careful data curation for consistency and clarity in visual storytelling. Their catalog of these templates and approaches in the paper might help others who are launching projects that create atlases like they describe.

I’m sure there were others that have relevant findings for folks working with visualizaton in newsrooms, but these were the ones that jumped out to me.

Talk Summary: Dr. Vishwanath on Science Communication & Misinformation

Wed, 18 Dec 2024 05:09:00 +0000

Notes from a talk By Vish Vishwanath (Harvard School of Public Health; Dana-Farber Cancer Institute), hosted by David Lazar and the Internet Democracy Initiative, December 18, 2024.

The surge of misinformation in science and health communication is not just a problem of individuals making poor choices or failing to fact-check. It’s a systemic issue deeply embedded in the structures of our modern information ecosystem. In his talk, Vish Vishwanath, chair of the National Academies committee on “Understanding and Addressing Misinformation in Science,” argued that we need to move beyond individual blame and examine the structural forces that create and amplify misinformation. Here’s a quick summary of how he unpacked the problem—and what we can do about it.

Vishnawath speaking at the event

The Problem: An Overloaded Information Ecosystem

The modern information landscape is overwhelming. With more channels and a greater volume of content than ever before, even science communication and translation experts struggle to make sense of the noise. This crowded ecosystem provides fertile ground for disinformation and misinformation—false information spread with or without malicious intent. Vishwanath broke down the key players shaping today’s science and health communication landscape:

Journalists: Traditional media outlets, along with local and minoritized press, play a major role in communicating science. The minoritized press, Vishwanath noted, often takes on an advocacy role, contrasting with the gatekeeping tendencies of mainstream media.
Activist groups and government officials: These actors produce content to inform, persuade, or advocate, often serving specific agendas.
Social media platforms: Platforms like WhatsApp allow anyone to reach massive audiences, often through viral video content. These low-barrier tools can amplify both accurate and false information.
The private sector: Commercial industries, like tobacco or sugary beverages, spend billions to shape public perceptions, often promoting narratives that align with their profit motives. For instance, tobacco companies spend $10 billion annually on marketing, creating a communication imbalance compared to public health efforts.

Amid this sea of content, making informed choices about what to trust requires effort, which many individuals may not have the time, tools, or skills to devote.

Misinformation spreads rapidly in this interconnected world, driven by factors such as:

Social media algorithms that amplify sensational content, regardless of accuracy.
The rise of AI-driven deepfakes that blur the line between fact and fabrication.
Declining trust in institutions, which leaves people more susceptible to alternative, often incorrect, narratives.

One illustrative example Vishwanath shared related to cancer prevention. Despite robust evidence that lifestyle changes can reduce cancer risk, many people believe that cancer is inevitable and that “everything causes it.” These misconceptions are perpetuated by social media spirals, where inaccurate ideas are reinforced and mainstreamed.

Inequalities in Health Communication

Structural influce model from a related academic paper (CC BY 4.0)

Inequalities in how people access, interpret, and act on health information are deeply rooted in education, race, and socioeconomic status, with significant consequences for public trust and health outcomes. Education levels and racial identity intersect to shape science literacy, with certain groups, like white evangelical populations, being among the least likely to get vaccinated due to cultural and informational divides. During the COVID-19 pandemic, reliance on online tools for vaccine scheduling highlighted the digital divide, leaving those without internet access at a disadvantage. Marginalized groups are often labeled as “hard to reach,” but Vishwanath reframed this as “hardly reached,” pointing to systemic failures in outreach and inclusion. These data gaps and inequities perpetuate disparities in health communication, reinforcing a cycle of exclusion and misinformation.

Structural Solutions: Rebuilding Trust Through Community

Vishwanath called for a shift from individual-focused solutions to systemic, community-driven approaches to combat misinformation effectively. By adopting an ecological perspective, efforts should target structural barriers such as limited access to technology and educational disparities, rather than placing the onus solely on individuals to “read critically.” Community-engaged research emerges as a key strategy, involving collaboration with communities to co-define problems and co-develop relevant solutions. Examples of this approach include the MassCONECT Portal in Massachusetts, which provides localized resources and training to help communities adapt public health messages, and SANCHAR in India, which has trained over 200 journalists to critically assess and communicate scientific information. These initiatives illustrate how building structural and community capacity can amplify accurate, evidence-based science communication.

Ultimately, Vishwanath called for more participatory and inclusive approaches to science communication. Misinformation isn’t just an individual problem—it’s a structural challenge that requires systemic solutions. By leveraging community knowledge and resilience, we can create alternatives to misinformation and foster trust in evidence-based information. Vishwanath’s talk highlighted the urgency of rethinking how we produce, share, and consume science information. Only by addressing the root causes of misinformation can we build a healthier, more equitable information ecosystem.

Note: this post was drafted from my personal notes into a narrative using genAI tools and then reviewed and editing by hand

Reflections on 'UN Human Security in the AI Era' Event

Sun, 01 Dec 2024 05:09:00 +0000

I was honored to attend and speak at a recent UN event about how AI might be reshaping the contours of human security: Human Security at 30: New Security and Development Horizons in the Artificial Intelligence Age (hosted by the UN Human Development Report Office). It was fascinating to bring my experience with participatory AI into this setting and it’s focus on human security, which was new to me. It was even more intriguing to be cast as an “AI expert” in the room of development experts, many of them working on next year’s Human Development Report. I left feeling both buoyed by the amazing work and energy, but also dismayed by the feeling of inevitability in the room, and associated embracing of corporate AI narratives. New technologies we are calling “AI” pose ongoing and future threats to human security, but we get to decide who, how, and for what these tools are built and used; we can’t cede the space to Big Tech’s corporate cloud solutions. I want to share some reflections on the public morning session in this blog post.

Pedro Conceição, head of the UN Development Report Office, introducing the event

The event was one in a series of “listening session” style events that inform the production of the annual UN Human Development Report. Many of these center concepts from “human security”, which focuses on protecting individuals (rather than states) from threats such as poverty, inequality, and environmental risks. The idea emerged in the 1990s, driven by definitions from Japan and Canada. Unlike state-level security, which prioritizes borders and military strength, human security emphasizes freedom from fear, want, and indignity, tackling root causes of insecurity, and more. The key idea is that a people-centered approach is essential in addressing global challenges that don’t stop at national boundaries; a category that more and more of our growing challenges fit into, hence the event on human security and AI.

Existential Risks in the AI Age

Toby Ord, senior researcher at Oxford University and author of The Precipice, kicked off the first panel with a sobering keynote focused on the question of AI as an existential risk to our species. He argued that humanity’s greatest risks stem not from natural forces but from our own tools. Ord stressed that AI development is accelerating far beyond our readiness to manage its consequences. To him, industry leaders and international figures have sounded the alarm, yet policy and public discourse lag behind. For Ord, safeguarding the future necessitates treating global AI regulation as a public good—a commitment as enduring and robust as the international frameworks governing nuclear weapons or climate change. I found his commentary to be provocative, because so often I find the “existential threat” narrative around AI to be a corporate smoke-screen that pushes attention away from concrete ongoing harms. I’m not worried about robot overloads controlling my fate; I’m worried about encoded discrimination in decision making, environment harms and exacerbation of power scarcity, impacts of the rampant spread of generated disinformation, and so on.

Tory Ord’s summary of human security

The associated panel of experts highlighted AI’s dual potential to empower and endanger, emphasizing the urgent need for ethical governance and equitable development. Laura Chinchilla, former President of Costa Rica, noted the rise of AI-enabled cybercrimes like phishing and fraud, which have outpaced legal frameworks, and stressed the importance of ethical standards and human rights education to mitigate these harms. Turhan Saleh (UNDP) framed AI within a rapidly evolving global risk landscape, cautioning that while its benefits remain hypothetical and unequally distributed, its costs are already exacerbating inequalities. His arguments strongly reflected some of my own short-term concerns and priorities in this space. Daniella Darlington, a responsible AI advocate, called for investments in data governance, community stewardship, and workforce education in the Global South. She pointed to barriers like limited computational resources, insufficient training, and low public trust—challenges that align with my perspective on addressing present risks rather than speculating on future scenarios.

Restoring Trust in an Age of Worry

The second keynote was from Prof. Rita Singh (Carnegie Mellon University), who took a researcher’s lens to the opportunities and unsolved problems of AI development. Singh works on advanced tools that use voice in numerous ways, including profiling. She called for intentional, human-centered design principles that prioritize societal needs over purely technical ambitions, including calls for society to encode incredibly complex things such as social norms and ethical behaviors into constructs that AI researchers could then implement. I personally found this take quite challenging to engage with, full of language I’d label at first glance as techno-centrist and reductivist. Encode ethical rules in some machine understandable way is incredibly complex and daunting, hardly a simple pre-condition to check-off on the way to building ethical bots.

Others on the panel shared how AI is increasingly embedded in everyday life, leading to concerns about disinformation, job displacement, and online safety. These feelings of distress and distrust are pervasive, with six out of seven people worldwide reporting insecurity in their lives. The discussion explored how the human security framework, which emphasizes freedom from fear, want, and indignity, could be reimagined to address AI’s challenges and restore trust.

I was a participant on this panel, asked to address how citizens and community-based initiatives in data collection and technology development might help restore trust and enhance agency. I emphasized that AI’s future is not inevitable—we have the power to choose what to build. By fostering empowerment and ownership in technology development, we can shift the narrative of insecurity toward one of control and agency. Technology systems are inherently socio-technical, and AI is often a marketing term masking this complexity. Centering local expertise and engaging stakeholders in problem definition is crucial for creating public-interest technologies that address meaningful challenges in particpatory and empowering ways.

I shared examples from the Counterdata Network, where we work withcivil society organizations to develop AI tools for monitoring news flows that surface and document human rights violations. This model demonstrates that AI doesn’t need to rely solely on cloud computing solutions in the Global North; it can be localized and aligned with grassroots needs and capacities. I sought to echo and amplify what I learned from my colleague Alessandra Jungs de Almeida on rethinking “securitization” through feminist perspectives, framing security not as abstract defense but as safeguarding the lives and dignity of women and girls. This reframing, I argued, could profoundly reshape how we approach AI’s role in supporting or threatening human security.

Towards Equitable AI Futures

The event underscored AI’s power to exacerbate global divides if left unregulated. It risks deepening inequalities between the Global North and South, with wealthier nations monopolizing AI’s benefits while poorer nations grapple with its harms. Human security, as defined over the past three decades, must now adapt to include AI’s promise and perils. From empowering women in AI development to addressing the digital gender divide and ensuring equitable access to AI resources, the path forward demands global solidarity and a renewed commitment to ethical governance.

The juncture that this event focused on offers an opportunity: to redefine security not as freedom from fear alone, but as the ability to thrive in an interconnected, AI-driven world. Will we seize the opportunity to take ownership of who gets to build AI, for which purposes, and with what processes? I was left believing that a human security-inspired approach to AI demands that of us.

Sculpting Data with Clay – A New Take on Data Physicalization

Tue, 19 Nov 2024 05:09:00 +0000

I’m a big fan of sculptural forms of data, particularly because I think they offer novel opportunities to think harder about the material as an interface to, and representation of, the data. Two projects have popped onto my radar while researching for my forthcoming Community Data book, both related to 3D printing clay data sculptures. “ListeningCups” and “Rain Gauge” transform everyday data into tactile, functional objects. Both projects highlight how clay, as a natural material, can foster an emotional, sustainable, and hands-on interaction with data.

ListeningCups: Tactile Data Stories

Collage showing the ListeningCups, extracted from their academic paper.

The ListeningCups project is the result of a residecy with designers and artists Audrey Desjardins and Timea Tihanyi trying to represent the ambient sounds of city in mug form. Presented at DIS 2019 the paper covers their motivations to bring data closer to people’s lives via tactile approaches that might encourage reflection and personal connection. Physically, they’ve encoded soundscapes in various places around Seattle. Each bump represents a the volume of snippet of ambient sound, allowing users to “feel” the sound landscape of these everyday spaces. This concept of “data stories” showcases how art and data can blend to evoke imagination and memory.

I particularly like their work here because it allows for casual interaction as an “ambient data display”; you can just use the mug as a mug with an interesting texture, or you could choose to try and interrogate it via tactile and visual ways to dig into the data more. This kind of “craft data sculpture” presenting an intriguing form of bringing people and data together, allowing for more serendipitous data exploration. An intriguing wrinkle is the “data accidents” they point to from the physical printing process, reminding us that data representation isn’t always as exact as mythology and hype tells us. Their reflections particularly highlight their own relation to the object in cool ways, as a talking piece and as a mug. As a drinker of copious amounts of tea, I’d enjoy having one of these around to explore and reflect on.

Rain Gauge: Visualizing Climate Data with Sustainability in Mind

Collage of test prints evaluating data encoding onto extrusion rate, pauses, and size of symbols

Earlier this year a team of University of Minnesota researchers wrote a paper for IEEE Vis 2024 about explorations in creating “ceramic data physicalizations” depicting rainfall levels over the last 80 years – the Rain Gauge project. They had a particular focus on the circular use of the clay material itself. The project practiced sustainable 3D printing by reusing clay repeatedly, making it an eco-friendly alternative to plastics commonly used in 3D printing. This method supports a circular, zero-waste process.

Their paper explores objects geometries, physical encoding approaches, and logistics constraints of the current clay printing machines they were using. Amusingly they documented the slow decay of the object itself when exposed to rain. Their conclusions focus on the circular use of the clay itself, beyond considerations related to the material as a medium for data sculptures.

Clay as a Medium for Data

Why clay? Unlike digital screens or synthetic materials, clay is flexible, recyclable, and rooted in a rich history of craft and art. With each project, the inherent imperfections and natural qualities of clay make data physicalizations not only informative but also deeply personal and reflective of the specific tools used to manufacture them. This innovative use of clay adds a physical and emotional layer to the data that might function to enhance comprehension and personal connection.

As data becomes more central to our lives, these clay-based projects remind us that data storytelling can be both meaningful and sustainable. Their work reminds me that I’d like to run a study comparing the same data representation in various types of mateiral, to hopefully learn about the affordances and limitations of some of these approaches for varying audiences.

AI-Powered Content Audits for Local News

Mon, 18 Nov 2024 05:09:00 +0000

Organizations across the news industry are exploring ways to use generative AI to support their storytelling (many shared on this very blog). A growing trend is to use these new tools on the other end of the production cycle, to support content audits that assess a large set of stories against the mission and goals of your newsroom. With Meg Heckman and Liz Hadjis, I conducted a test of this approach in partnership with a hyper-local outlet (The Scope Boston), and I want to share some of our methods and higher-level critical questions. Hopefully these help you come up with a thoughtful strategy for how you might integrate AI-assisted content auditing into your editorial workflows.

More and more newsrooms are performing content audits and publishing them online.

A growing number of newsrooms are taking on content audits, from the Philadelphia Inquirer to the City NYC, measuring alignment between their stories and editorial goals. The Scope’s audit was motivated by their mission to highlight under covered places and peoples in Boston. They wanted to measure whether their content matched that goal by analyzing (1) which locations got the most coverage, (2) what topics dominated their reporting, and (3) whose voices were most quoted.

What We Did, and What We Found

We worked with the Scope’s editor to gather all their content and then report back on those three guiding questions: places, topics, and quotes. The first two were relatively easy to do with off-the-shelf open-source technologies for extracting places mentioned in text and assigning classic news topics to a story. The third was harder, because we found no ready-to-use tool for news articles that could extract quotes, attribute them to speakers, and tell us which pronoun was used about their speaker (so we could determine their gender ethically). For that one we turned to generative AI and after iterating on prompts we were surprised to see that it performed quite well.

A block diagram of our prototype technology pipeline.

As with any project, data cleaning took a big chunk of the time. However, since we started with a machine-readable XML export from the Wordpress-hosted site the cleaning was mostly on the results. For instance, we had to filter geographic places for only local neighborhoods and do things like convert “Jamaica” into “Jamaica Plain”. For the quotes, because we used generative AI, we had to validate that each and every quote was actually from the story, and that the attributed speaker’s name was also in the text. To my surprise ChatGPT didn’t invent any quotes or speakers and only ended up costing about USD $6 in total (via API calls to their “gpt-3.5-turbo” model). See the pre-print we presented at AEJMC earlier this year for details on the prompts and related methods.

At a high level, we found some validation for the Scope’s existing practices and some opportunities for improvement in line with the mission statement. Geographically, coverage was dispersed across the city and the neighborhoods editors hoped to focus on. However, while some Boston neighborhoods, like Mission Hill (17% of stories), were covered extensively, other areas like Mattapan and Roxbury received less attention (1% each) despite being very close to the institution that operates the publication (Northeastern University).

The Boston neighborhoods covered most often in stories. Longer bars mean more stories mentioned that place.

In terms of topics, culture (26% of stories), politics (18%), and economics (17%) dominated The Scope’s coverage, which made sense given their mission. However, we found that other issues the editors deemed important like transportation (2%) and technology (<1%) received little attention. This was especially true when we dug into the topics covered in neighborhoods; note the absence of transportation coverage in Mattapan, Quincy, and Roxbury despite it being an important local issue.

A heatmap-style diagram showing how much each topic was covered in each neighborhood. A darker blue square means more coverage of that topic in that place. A blank square means no coverage.

For quotes, speakers were identified by “she” (57%) more than “he” (38%), perhaps because many of its reporters identify as women. In addition, very few of the top quoted people were politicians or other public figures. Just 37% of those quoted overall fit into this “social elites” category. These findings counter results from research across mainstream media sources.

A chart showing quoted speaker gender based on the pronoun most-often used to refer to them in the stories. Bigger bars mean more unique speakers were referred to with that pronoun.

These results matched some of what the publication suspected, but also introduced some surprises and ideas for changes to consider for the upcoming year of coverage. You can read our full academic paper from the 2024 Computation + Journalism Symposium to see more detailed results about these three approaches we took in our content audit.

What You Might Want to Try

Taking on a content audit might seem daunting, but with just a few days of work a data scientist could produce a first draft. We’re hoping to make that even easier by finding partners and funding to build a web-based toolkit others could use.

In the meantime, if you do want to get started on this sort of thing, there are some key preparatory and strategic conversations and tasks that you should take on. First, before choosing any tools you need to define what you want to measure and ensure it is in alignment with your mission. Local newsrooms have a geographic focus, so that is kind of an obvious choice. Beyond that, what kind of representation do you care about? Linguistic? Quotes? Identities? Organizations?

Second, when you’re ready to do some technical work try to choose well-validated off-the-shelf machine learning tools before diving into generative AI systems like ChatGPT (more on this below). Established tools are often already evaluated and robust from prior studies and use. They also create reproducible results, unlike this new batch of generative LLM tools that can sometimes make things up.

Third, once you have data, be sure to take the time to interpret the findings with your team and broader community. The data you find isn’t going to be perfect but can reveal patterns that tell you something about your coverage and might suggest changes you could make. If certain populations, places, or topics are consistently underrepresented, make a plan to address this in future reporting.

Finally, make a commitment to regularly audit your content every year or so and publish it publicly to hold yourself to account. Integrating this kind of process into your annual workflow is the only way to create meaningful results and change over time.

Be a Critical Technology User

I could end this write-up here, but I think there’s an important reflection about technology that also needs to be addressed. Chat-based generative AI tools feel like a magical solution to using advanced computer-based language analysis approaches, but I encourage you to be a skeptical consumer in the face of the massive hype around them. Note that we only used the ChatGPT API where we could find no other solution. I have three main reasons why I think this stuff should be a tool of last resort: hidden costs, relational risk, and accuracy.

To start with, the energy costs of using these tools are massive. The power needed to create generative AI models and run them is unsustainable and exacerbates existing ecological disparities. The now-famous “Stochastic Papers” paper from Bender, et al documents this in depth. Power consumption needs are driving a massive race between companies to secure energy supplies for their server farms in a world already suffering from scarcity. Many alternative non-generative machine learning models that have been around for a while can run on your laptop computer, take far less power to train, and produce accurate and usable results for many audit-related tasks.

On top of that consider the history of Big Tech as an untrustworthy partner to newsrooms. Remember Google’s AMP? The Facebook-driven pivot to video? Suspiciously motivated funding opportunities? Copyright concerns about training data? Where do you draw the line for your publication in regard to working with Big Tech companies whose choices are so-often demonstrably misaligned with the news industry? Using well-packed open-source software alternatives can help you avoid many of these relational risks. Or maybe using open-weight versions, where you can tune the model’s parameters to control behavior more directly, is a good middle ground for you?

Finally, consider the fact that generative AI tools are designed to statistically make things up. The large language models underlying chat-based interfaces are just giant systems trained on text to count how often words appear next to other words; they’re probabilistic Xerox machines. It is very hard to both validate their outputs and ensure reproducible results. Non-generative machine learning tools don’t introduce the same data quality risks, and many of them have already been validated through prior use and research evaluation processes.

Final Thoughts

A content audit may sound like a challenging high-tech undertaking, but The Scope’s case shows it’s within reach for small, mission-driven newsrooms if you can find a little technology assistance. With just a few free and affordable tools and some clear goals, you can gain valuable insights into your newsroom coverage’s strengths and areas for improvement. I encourage you to try these evolving approaches out, making critical and intentional decisions that engage with the ethical quandaries I’ve shared, as appropriate for your organization. Running any sort of content audit is going to help you assess, and demonstrate to your readers, how your content is in alignment with your mission and goals.

Originally published to the Generative AI in the Newsroom blog on Medium.

Computation + Journalism 2024 Symposium wrap-up

Fri, 01 Nov 2024 05:09:00 +0000

We recently co-hosted the Computation + Journalism Symposium 2024 at Northeastern, bringing together academics, journalists, data scientists, editors, and more to explore innovate work at the intersection of fields. Keynotes from Julia Angwin, Aaron Sankin, and Alan Mislove.

Reviewing the papers and workshops, three central themes jumped out to me, each explored a little more below:

Generative AI-based tools for newsrooms and writers
New methods for tracking and studying flows of information
Tracking and fighting disinformation

The logo we ended up using across various assets was kind of an accidental design I made as a one-off for a quick last-minute need 🤣.

Unsurprisingly, generative AI was a major theme across multiple submissions and many more informal chats. Questions about accuracy, ethical responsibility, and transparency surfaced repeatedly. Researchers and industry professionals highlighted the challenges and nuances of incorporating large language models into newsrooms. I ended up seeing the main message as one related to developing balanced frameworks to preserve editorial judgment and audience trust while still experimenting with benefits from genAI’s capacity to remove non-critical journalism tasks. From recommender systems to dynamic article restructuring, some projects explore how computational journalism can adapt content to readers without sacrificing broader informational integrity. This included my own submission, with Meg Heckman and Liz Hadjis, which explored methods and critical questions related to a genAI-assisted content-audit of a hyper-local publication: “A Case Study in an A.I.-Assisted Content Audit”.

A contingent of research presentation focused on developing new methods for analyzing and tracking stories across media ecosystems. Several papers introduced tools that leverage natural language processing and network analysis to trace story origins, evolution, and reach across different platforms. These approaches help researchers identify key influencers in story propagation and detect shifts in narrative framing over time. By visualizing these dynamics these methods aim to give journalists and media researchers a clearer view of how stories gain traction, morph, and impact public discourse across diverse media landscapes. I’ll be studying these more closely vis-a-vis opportunities to collaboration, and new feature design, for Media Cloud’s searchable Online News Archive (which I co-direct).

The third major theme that surfaced for me related to ongoing work to understand and combat misinformation and disinformation across the field. This is especially relevant to the wide variety of elections happening in 2024 across the globe. Several papers unpacked strategies for identifying, tracking, and mitigating false information, offering innovative computational approaches for real-time detection. In parallel, there was an emphasis on educating audiences, fostering media literacy to empowering readers against misinformation. The importance of context in debunking misinformation, particularly through visualizations, also emerged as a core discussion point. This focus on active audience engagement reflected a larger trend at the symposium: computational journalism isn’t just about producing content but creating a more informed and resilient public.

Attendees at our data sculpture workshop showing off the pieces they made.

All told, this symposium underscored the need for continuous collaboration between technologists and journalists to ensure that innovation aligned with the values of the field and the needs of the public. The workshop I co-led with Andreés Snitcofsky pushed the edge a bit, inviting those groups to come build data sculptures around a table full of craft materials. Honestly that’s one of the reasons I’ve enjoyed the few C+J events I’ve attended; the mix of practitioners and academics makes the whole event feel more grounded. Plus it’s a great reconnect with, and make new, friends around shared interests in an informal and smaller setting.

Thanks to everyone who helped make it a great event. Particularly to my co-organizers John Wihbey, Chenyan Jia, Piotr Sapiezynski, Libby Phillips, and Susan Conover. Keep an eye on the Internet Democracy Initiative YouTube channel for talk vidoes, which are being posted soon. Hope to see you all at the next one in Miami!

Pre-print: Comparing Author Extraction Libraries

Wed, 30 Oct 2024 05:09:00 +0000

How effective are existing tools for extracting author information from global online news? Author extracton is a key task in media research that supports a number of different types of research questions. We’ve hesitated to support this broadly in Media Cloud, because our past evaluations of off-the-shelf software solutons haven’t performed consistently and related ethical concerns about weaponizaton against journalists. We decided to revisit this quesions last year and just published a pre-print to document our results. Read “Author Unknown: Evaluating Performance of Author Extraction Libraries on Global Online News Articles” by Sriharsha Hatwar, Virginia Partridge, myself, and Fernando Bermejo.

Given the diversity of languages and content styles in today’s news landscape, researchers need reliable ways to gather accurate metadata for large-scale media analysis. However, many of the author extraction libraries were designed for English or other commonly studied languages, often neglecting the nuanced challenges that multilingual content poses.

Counts of documents and number of author annotations total in the final evaluation dataset.

Our study introduces a new multilingual dataset to evaluate several popular libraries (Newspaper3K, Trafilatura, news-please, Go-readability, and ExtractNet). The results reveal an wide range of performance outcomes. Libraries like Go-readability and Trafilatura stood out as the most effective in certain scenarios, but the effectiveness of each tool varied considerably based on language and region. For instance, while one library might perform well on English or European-language news, it could struggle with languages that have less standardized structures or different metadata conventions. This highlights a critical shortfall: these tools, though popular, aren’t yet universal solutions for a diverse, global media landscape.

Rouge-1 scores for each library in each language. Higher is better. Dashes indicate instances of no character overlap.

From a practical standpoint, our findings suggest that researchers working with non-English or multilingual datasets should prioritize testing and validating their tools within their specific research context before embarking on large-scale data collection. That’s not an especially surprisinng result… “your mileage may vary.” The research also points to a promising direction for future work—developing multilingual models or even language-specific modules within existing libraries that can reliably handle a wider range of text sources. A clear missing piece here is a comparison to results from recently-released generative AI models; our study was carried out before those surged to the fore-front. Consider that fodder for a future blog post.

Thanks to the author team and the broader set of people who volunteered their time and language expertise to help us build the evaluaton dataset. For those interested in the technical details, the full paper delves deeper into the evaluation methods and our recommendations for best practices in multilingual media analysis. We hope our new dataset and this evaluation help push this area of software development forward.

Read “Author Unknown: Evaluating Performance of Author Extraction Libraries on Global Online News Articles” on arXiv.

Your Early Fall AI+Journalism Updates

Wed, 11 Sep 2024 05:09:00 +0000

Welcome back from the summer. So much has happened at the intersection of AI and journalism while you’ve been at the beach. Here’s my latest summary of what caught my attention.

Skill Building

The Online News Association (ONA) has launched a huge new effort to build AI skills in newsrooms with support from various funders. Follow their AI in Journalism Initiative for trainings, reports, case studies, convenings, and more. Keep an eye on their blog for updates.

A recently launched collaboration between the London School of Economics and Google News Initiative is offering funding to help put those skills to use. Read about the “JournalismAi Innovation Challenge”, global call for small and medium-sized publishers, to see if one of your ideas is a good fit.

Integrations (or lack thereof)

Newsrooms continue to experiment with new product features built around the current batch of technologies, like Gannett adding AI-generated overviews. Over at NiemanLab, Neel Dhanesha dug into the fake authors on generated stories from Hoodline, finding a lot of curious patterns of faux diversity. Google added AI overviews to their main search results; read what news audience directors think about that (again from NiemanLab).

Those fighting the scraping-based collection of their content have been using robots.txt to try and block the crawlers from big tech companies; Reuters Institute published a report about how that is approach evolving back in February. Ben Walsh has a regularly updated dashboard with more up-to-date details about adoption. Cloudflare, which is used by huge portions of the internet to serve content faster and protect against attacks, even added a simple checkbox to block “AI Scrapers and Crawlers” to make this easier for publishers.

But what about readers? Reuters Institute published a report on “public attitudes to the uses of generative AI in news”. Read their summary thread on xTwitter for key highlights.

Legal Woes

OpenAI has continued to try and create legal deals with publishers despite more of them suing for copyright violations. Columbia Journalism Review is maintaining a dashboard of deals and rumors. Even with signed deals (with Time, and The Atlantic), it isn’t quite clear how news organizations will benefit. An early example is Business Insider’s deal with OpenAI, which NiemanLab delved into. Maybe revenue sharing like Perplexity is piloting will happen?

Critiques

I tend to remind people that “AI” is a marketing term, not a technology. As such, the hype and news coverage around it feed into cycles of investment, stock growth, deal-making, and imagined futures. Prof. Rasmus Kleis Nielsen from the Reuters Institute dug into the role uncritical news coverage of companies plays.

In the academic realm, a paper my friend Harini Suresh and others presented at the Fairness and Accountability Conference Conference (FAccT) explored alternative pathways to building the low level “foundation” models that companies are relying on. Read their short second case study in “Participation in the Age of Foundation Models”, on journalism, to imagine how more participatory approaches might work.

Pre-print: Testing Generative AI for Source Audits in Student-Produced Local News

Thu, 15 Aug 2024 05:09:00 +0000

We recently shared new work at the AEJMC 2024 conference on prototyping an AI-assisted audit of gender representation via quotes in The Scope. We’ve published it as an under-construction pre-print to share the work and find related research to collaborate with.

Working with student journalists at The Scope, we tested ChatGPT to audit quoted speakers’ gender and social status in their articles. Quote extraction and attribution is not an off-the-shelf task that has already been solved by NLP researchers, so we turned to ChatGPT. This approach not only helped us identify the types of voices being amplified—often diverse and non-elite—but also allowed us to consider the potential of AI in making source audits more accessible for smaller newsrooms.

Our method combined AI-powered quote extraction with manual validation over a two-phases process (extraction and disambiguation). One prompt extracted quotes, associated speaker attribution, and any pronoun used to describe them. The second clustered names in each story to identify unique speakers.

Women were quoted more than men, perhaps due to the prevelance of female writers.

While the results were promising, it raised important questions for us about about using AI tools in content audits. Using generative AI, especially in smaller, community-focused settings, prompts us to think about the biases AI may carry, its environmental impact, and the risks of potentially “mainstreaming” perspectives encoded in AI’s training data that may not align with local values.

We’re left excited about generative AI tools’ potential for transforming media audits, yet also cautious and critical. As AI tools become more embedded in newsrooms, we must navigate issues like intellectual property, energy usage, and the AI’s role in representing local voices. Our study serves as a first step in both a practical experiment and a call for broader discussions on responsible uses of AI in community journalism.

Grant to support AI monitoring of news for human rights violations

Tue, 06 Aug 2024 05:09:00 +0000

So excited to announce that we’ve received a grant to support our work on the Counterdata Network from the Data Empowerment Fund. Our project builds on a powerful legacy—think of Ida B. Wells using news to expose lynchings in the late 1800s. Today, we’re continuing this tradition by developing AI models that empower advocates to uncover human rights violations more effectively. This involves custom-trained machine learning models to identify relevant events, designed with a focus on equity, localism, and data justice, all using a deeply human-centered approach. Our software tools are actively used by dozens of human rights groups. With this support, we’ll be able to improve their stability and expand our network of partners.

Pushing Tech Forward

Our system pulls from various online news archives, both open-access and subscription-based, using keywords and dates to fine-tune search. We’re also developing models leveraging techniques like transfer learning and word embeddings and offer browser extensions that integrate directly into advocates’ workflows.

Rethinking the Model

Traditional commercial AI doesn’t meet the needs of these groups, so we’re adopting a co-creation approach, rooted in feminist principles. We’ve spent four years building trust and interviewing community members—often taking months before the first line of code is written. This focus reduces emotional strain and widens the scope of what can be achieved.

Open Questions

Many questions remain for us: How do we sustain these civic technologies as public goods? What community governance models make sense? And how do we best handle data from social media monopolies? Our hope is to contribute not just tools but also new ways of thinking about AI that align with human rights and justice.

Media Cloud Receives Major NSF Grant!

Mon, 01 Jul 2024 05:09:00 +0000

We’re thrilled to announce that Media Cloud has secured new NSF HNDS-I grant to take our global news archive and director to the next level! This grant enables us to extend and enhance our searchable news database, which currently includes nearly 2 billion documents. With this funding, we’ll improve infrastructure and introduce new tools to support impactful social science research. Media Cloud is emerging as a piece of critical public data infrastructure to support a wide variety of types of research across disciplines.

Our renewed work aims to provide researchers with unprecedented access to global online news data and powerful new visualizations. Researchers across fields—politics, health communication, economics, disinformation—will be able to track and analyze media narratives and framing with ease. We’re expanding our indexing capabilities, creating more flexible data collections, and improving stability so that the archive is as accessible as it is powerful.

Our commitment to social impact means that non-profits, journalists, and advocacy organizations can also use Media Cloud to support their work in media monitoring, social change, and public health. We’re planning new tutorials and features designed for users new to computational tools.

Stay tuned as we bring you updates on new features, collaborations, and ways to join us in exploring the world’s online news landscapes.

Data Theatre Collaborative wins Heart of CommUNITY Award

Sat, 15 Jun 2024 05:09:00 +0000

Excited to share that the Data Theatre Collaborative won a “Heart of CommUNITY” award from Northeastern University for our work with Livable Streets Alliance and Hyde Park Task Force youth.

Moira Zellner and I receiving the award. Photo by Alyssa Stone/Northeastern University

I’ve always believed that data should be accessible to everyone, not just analysts. That’s why the Data Theatre Collaborative is working to transform how communities engage with data through participatory civic theater practices. We partnered with the Hyde Park Task Force youth and Livable Streets Alliance to reimagine the SouthWest Corridor Park redevelopment. Instead of defaulting to traditional PowerPoint presentations that often exclude many community members, we brought survey data to life through theatrical performance.

Working alongside theater artists, we devised a 10-minute performance that presented the data in an engaging, accessible format. Our early results show that this approach is truly transformative - not only in how people understand the data but in who participates in these civic discussions.

I’m particularly proud of how this project demonstrates that data analysis doesn’t have to be confined to spreadsheets and graphs. By making these presentations more participatory, we’re helping ensure that community decisions are truly inclusive and representative of all voices.

Creative Data Representation for Social Justice Movements @ csv,conf,v8

Thu, 30 May 2024 05:09:00 +0000

I attended csv,conf,v8 in Puebla, Mexico, where I presented a talk on creative approaches to data representation that are more appropriately designed for social justice movemements. This connects to idea in my forthcoming book on “Community Data”. The group was an incredibly kind and welcoming mix of data scientists, data visualization experts, journalists, activists, academics, and more. Watch a video of the talk.

All the attendees say “hi!” 👋🏽

I presented my work on creative data representation approaches for social justice movements. Now-traditional data visualization practices are often rooted in cultures of surveillance, formalism, capitalism, and authority - epistemological frameworks that can conflict with the needs and values of community-based social justice work.

I shared several inspiring alternative examples that demonstrate what’s possible when we embrace what Turkle and Papert call “epistemological pluralism” - accepting multiple ways of knowing things. Drawing on Heron and Reason’s extended epistemology framework, I discussed how we can move beyond just propositional knowledge to incorporate experiential, practical, and presentational ways of knowing in our data work.

My own approaches to community-centered data representation try to put those theories into practice, from data murals, to data sculptures, to data theater. These creative methods aim to empower communities with information while respecting their own ways of knowing and sharing knowledge. These will be major themes of my forthcoming book, “Community Data”.

New Strategies for Using Artificial Intelligence in Journalism

Thu, 16 May 2024 05:09:00 +0000

Here’s the fifth in my series of round-ups on the latest in artificial intelligence and journalism. This time around it feels like the rubber has really hit the road, with lots of work on strategies and examples being shared around the web.

What’s Your Newsroom AI Strategy?

Larger news organizations seem to be expanding effort on concerting artificial intelligence strategies. Just as one example, in February the Washington Post appointed their first Senior Editor for AI Strategy and Innovation. In March NiemanLab interviewed the New York Times AI editor on how the foundational principles of journalism are worth returning to, and how that plays out in concrete examples of AI assisted reporting.

This is causing lots of reflection and strategic thinking at the C-suite level as well. The Newsroom Robots podcast interviewed the Dutch foundation for public broadcasting, and NiemanLab touched on AI with the Financial Times’ CEO.

Can AI Really Help You Report?

We also found a lot more write ups from journalist and technologist, reflecting on how AI tools can help them, or not. Check out the widely shared reflections from the Markup on the failures of trying to use ChatGPT as a reporting assistant. Simon Willison (Django, Datasette) shared a variety of more technical examples of how to use AI tools for working with data in journalism. In April the AP released the “Generative AI in Journalism” report they commissioned, based on a survey of hundred of news industry staff on ethical and practical considerations.

Lots of options for learners are rolling out as well. The Knight Center spun out a new course on AI and elections. The Pulitzer Center’s AI Spotlight Series is hosting and broadcasting a variety of trainings. The Online News Association launched their AI in Journalism Initiative, which includes hands-on, trainings, community convenience, and hosted talks.

Legal Questions Continue

No quick verdicts on the legal front. Various lawsuits are starting to move their way through the courts, and many outlets continue to ponder the legality of the generative models that underlie the chat based interfaces were currently calling “AI“. CJR summarized legal opinions from a couple of different sources to think about whether companies like OpenAI could be sued for defamation. Over at Fast Company, a musician draws comparisons to the age of Napster (for illegal music sharing) and the power of big tech companies in the modern hype-driven AI cycle.

Sadly the AI generated content scandals are still unfurling: Futurism dub deeper into AdVon and found their content was far more widely used than originally thought.

Originally published at https://www.storybench.org on May 16, 2024.

Guest on Schwabish's PolicyViz Podcast: Data Beyond the Screen

Thu, 25 Apr 2024 05:09:00 +0000

I was so pleased to be invited by my friend and colleague Jon Schwabish for an interview on his PolicyViz Podcast. The topic was “Data Beyond the Screen”, specifically focused on data sculptures. Watch a video of interview.

As someone who started in computer science before finding my way to more community-centered work, I enjoyed sharing how my journey has shaped my approach to making data more accessible and engaging for diverse communities. In our conversation, I emphasized how our society tends to privilege sight over other senses when it comes to data representation. To address this I’ve done a lot of work on data sculptures and embodied data experiences. These approaches help create new entry points for people who might feel excluded by traditional data visualization methods.

I highlighted the 1,659 table, a table my wife and I created during the pandemic, welded together from 1,700 pieces of cutlery – representing the number of households applying daily for SNAP benefits in Massachusetts. This piece exemplifies how physical data representations can spark meaningful conversations about difficult topics like food insecurity.

I also shared our ongoing work with data theater, where we’re reimagining community engagement through theatrical performances based on data. Instead of traditional civic meetings with projectors and sticky notes, we’re exploring how 20-minute performances followed by group discussions can create more inclusive spaces for community dialogue.

The goal of using these methods isn’t just about creative approaches to data literacy; it’s about empowerment and ensuring more voices have a seat at the table when data-driven decisions are being made. What excites me most about this work is how it challenges the traditional power dynamics of data collection and analysis. By bringing data back to communities in forms they can relate to and engage with, we’re helping reset these power balances and create more equitable ways of working with information.

For those interested in exploring these approaches, remember: working with communities takes time, and speed is often a poor metric of success. The relationships we build and the questions we encourage people to ask are far more valuable than quick results.

How can AI help us study the news? Here are my experiments with LLM-based query generation.

Tue, 26 Mar 2024 05:09:00 +0000

To understand politics, economics, events and health communication in the news, researchers study large sets of articles from online news sources. Media Cloud, a project that I co-lead, holds a historical archive of over 1.5 billion stories from over 100,000 sources and has been used to study police violence, influence of pre-print academic papers, political propaganda and numerous other topics.

A challenging part of these kinds of projects is selecting the right set of news stories about the topic being studied and finding the signal in the noise of billions of articles. Recent developments in AI made me wonder if large language models, or LLMs, with chat-based interfaces could help make this easier — could AI change how we perform media research in the short term? (Spoiler alert: probably not massively, but maybe in limited ways.)

AI as a question generation tool

The constant drum of marketing hype about AI has led to a broad range of uses focused on answering questions, as the main integrations are with web search engines. In contrast, I’m particularly interested in how AI might help us craft better questions, an area that is under-explored and quite concrete.

I have a concept of a question I want to ask about news coverage and keywords that might pull out relevant articles. The large language model has some embedded understanding of how my topic of interest is written about in the news because it has been fed the entire internet. The hypothesis here is that I can pull out some latent information that the LLM has but I might not think of, then, have it express that to me in the form of a query I can use in Media Cloud. There is no concern with synthesized data because I’m taking the question generated to use another database of news stories.

Most large archives of news (a.k.a corpora) support some form of boolean search, allowing you to combine keywords using “AND” and “OR” statements to pull out articles that match your query. Another approach often taken is to teach machine classifiers how to find the stories you care about. Both are approaches that often require specialized knowledge. The goal here is to see if LLMs can generate a boolean query string for me that helps narrow in on news stories that use those terms. This could help me (a) avoid having to learn arcane boolean query syntax, and (b) include terms or euphemisms I might not know about.

Context: Finding news about feminicide

One specific related project I work on is the technology for the Data Against Feminicide collaborative led by Catherine D’Ignazio’s Data + Feminism Lab at MIT. Co-designed with activist groups monitoring feminicide across the globe, our technology scours news articles. It searches for stories about gender-related killings of cisgender and transgender women and girls based on keyword queries. One of the primary sources is the Media Cloud online news archive. Then, it passes those through bespoke trained models co-created with the groups. Finally, it sends high-scoring results to the groups via email and a web-based dashboard. Think of it as high-powered, highly relevant, Google news alerts with global coverage.

Writing boolean search queries for this system is very complex, leading to this query + model approach I just described. Here’s a sample Spanish language query developed with one group:

(asesinato OR homicidio OR femicidio OR feminicidio OR asesina OR muerta OR muerte OR mata OR dispara OR balea OR apuñala OR golpea OR estrangula OR “sin vida” OR cuerpo* OR cadaver* OR restos) AND (mujer* OR niña* OR “una joven” OR “una adolescente” OR chica* OR prostituta OR “trabajadora sexual” OR “mujer trans” OR transgénero OR “hombre vestido de mujer”)

With queries like this, the system pulls stories from Media Cloud’s online news archive, Wayback Machine’s news archive and Newscatcher. Could we generate something similar with tools like ChatGPT? The promise of easier onboarding and better queries was enough motivation for me to experiment.

My initial prompt to both wasn’t particularly complex, and I tried to use generic terms. I described our system and need, and asked it to generate queries that might help:

I have a large ElasticSearch database full of over the extracted text of more than 1 billion online news stories. I want to find stories about feminicide, events where a woman was killed due to her gender. Provide a few boolean keywords queries I could use to pull just stories about these types of events.

ChatGPT produced generic results

ChatGPT v3.5 is the latest free-to-use model from OpenAI. The first results weren’t in the right format (it gave me JSON), so I had to specify that I wanted boolean search strings, and then I got some potentially useful results:

(femicide OR “woman killed” OR “gender-based violence”) “gender-based violence” (femicide OR “woman killed” OR “violence against women” OR “gender violence”)

These are on topic, but aren’t particularly good based on our experience building these queries. The issue of feminicide is seldom reported in such direct language. So, I asked it build queries that might include keywords that are “indicators of events;” the results were slightly better:

(homicide AND woman) OR (killed AND gender) OR (“dead woman” AND crime) OR (“murdered female” OR “assaulted woman”) (perpetrator AND female) OR (“investigation of gender-based crime”) OR (“suspect in femicide”) OR (“law enforcement and violence against women”) (protest OR “community demands justice” OR “rally against violence”) (domestic violence OR “intimate partner violence” OR “relationship-related crime”)

Still not great, but potentially more useful.

Gemini showed more contextual understanding

I tried the same query in Google’s free Gemini tool, speculating that perhaps a search engine giant might generate more useful searches. Gemini suggested a “multi-pronged approach that focuses on identifying patterns and indicators of feminicide” (emphasis from Gemini). These included:

Victim-focused keywords: “woman killed” OR “girl killed”; “female victim” OR “she/her murdered”; “honor killing” OR “dowry death” (in relevant contexts)
Contextual keywords: “gender-based violence” OR “misogyny” (use cautiously), “stalking” OR “harassment” (preceding victim keywords)
Exclusion keywords: “accident” OR “suicide”, “self-defense” OR “justified homicide”

The first intriguing result in the accompanying text is that it has a clear concept of what feminicide is and the social context around its reporting. It reminded me that the task is hard because of “underreporting” and lack of use of the term, “ambiguity” related to reporting on other types of killings and “sensitivity” to replicating negative stereotypes of women. While each is contestable, it showed a surprising attempt to contextualize the task assigned.

It also advised paying attention to the health of research staff, collaborating with institutional partners with domain expertise and recognizing underreporting. That’s a shockingly solid list of contextually appropriate caveats to include in automated results. Our team focuses on all of those. Perhaps Gemini was trained on data including research papers from the Data Against Feminicide team?

Can we let AI write the queries?

After a few hours spent iterating on approaches, I’m not ready to part ways with my colleagues’ expertise in constructing boolean queries. That said, the tools did help lay out the field of coverage (Gemini’s “multi-pronged” approach), gave little tips that helped to learn (the syntax examples from both) and created reasonable first drafts of queries. I am convinced that building an onramp that allows Gemini to suggest draft queries at the start of a project relying on boolean queries would be useful. This is especially true in learning settings, where I use Media Cloud’s web-based search tools with students in communications, journalism, marketing and others.

As an engineer by training, I’m not a fan of the phrase “prompt engineering;” writing prompts doesn’t feel like designing and building something with machines. In this case, spending more time developing this prompt didn’t prove especially useful. I wonder if trying it with a topic I know less well would reveal better reflections. Additionally, colleagues have suggested that inputting a known good query and asking it to adapt or iterate on that to improve it might yield good results.

Consider this piece a work in progress, surfacing ideas for experiments for another day. Overall, the longer conversations with the chat-based AI felt like talking to a middle schooler obsessed with Pokemon, dinosaurs, etc. There was deep domain knowledge but little capability to synthesize it with what I wanted from the interaction.

I think learners of various types are certainly capable of writing something like the first simple prompt to describe the topic they are interested in. In fact, it would likely benefit from having to describe what is and what is not part of the content they are looking for as they develop projects on media attention.

This approach to having AI generate queries for research shows promise for novices learning to use tools like Media Cloud for querying to find stories about human rights violations or other topics. More broadly, the idea of using new chat-based LLM interfaces to help structure questions or brainstorm is one I’m very engaged with. In both my research and teaching, I see that as a contextually appropriate use that learners benefit from.

Related work in research and data analysis is already showing how LLMs can help automate data extraction and transformation. I found early evidence that ChatGPT can do quote extraction and attribution far more effectively compared to existing libraries.

Pew researchers are using ChatGPT for a variety of research tasks, validating automated results against human results and seeing good performance on repetitive analysis tasks. Their most recent example automated the extraction of names of guests featured on podcasts from transcripts.

What have you tried? Please drop me a note and share any learnings or examples.

Thanks to Catherine D’Ignazio, Daniel Kornhauser, Peter Whiting, Paige Gulley, Helena Suárez Val and Kevin Zheng for early feedback on this experiment and post.

Originally published at https://www.storybench.org on March 26, 2024.

Spring updates on artificial intelligence and journalism

Thu, 14 Mar 2024 05:09:00 +0000

Here’s the fourth in my series of round-ups on the latest in artificial intelligence and journalism. Budding uses of AI in newsrooms

The Online News Association launched a new AI in Journalism Initiative to support journalist’s explorations of how AI technologies could help their work. They’re planning to offer training, community building, lesson sharing and broader convening.

Curious about what people are doing now? Columbia Journalism Review shared a vision for how newsrooms can adjust and experiment with AI technologies, and released a new Tow Center report on new organizations’ motivations, techniques and expectations. More concretely, Pew Research wrote about how they used LLMs to pull out guest names from popular podcasts. Their findings? After some validation on a sample, they found the model effectively replaced tedious manual desk work via automation.

Dealing with generated content?

Search engines face the looming challenge of dealing with reams of AI-generated content. 404 reported on Google’s apparent apathy to whether the content is human-created or not. A new scandal emerged and faded quickly: Car & Driver reported on Formula E removing their AI-generated influencer after fan backlash. The key comment? Outrage that motorsports would be so unwelcoming to women that only a virtual one could make it. They apologized, kind of.

This growth in manufactured content spills over to impacts on individuals. University Of Washington professor Dr. Emily Bender found a completely fabricated quote in a story published by Bihar, India news site Biharprabha. Read more the full story from The Quint.

What the future holds

Storybench reported on a few Northeastern events that tried to read the tea leaves and speculate about the future and social impact of AI on media. Jill Abramson and Ethan Zuckerman spoke at length about what the future might hold at an “AI and the Media” event.

A few days later Chris Perry gave a book talk on “Perspective Agents”, offering another take on what the future of AI and media holds. More speculatively, the Open Society Foundation hosted a (now-closed) call for imagining the future of AI and journalism 5–15 years out; keep an eye out for the fully fleshed out scenarios they’ll share this summer.

Continued copyright concerns (and deals)

That lawsuit between The New York Times and OpenAI continues to unfold: ArsTechnica published some commentary about the core legal questions, OpenAI claimed NYT cheated and Professor Nick Diakopoulos shared some experimental results to try and explain parts of the filing. Meanwhile, more publishers are suing OpenAI and Microsoft.

It’s going to take years to determine if this kind of use is transformative or not, but companies are still raking in the training data. Rumors emerged that Reddit signed a licensing deal with an AI company to allow use of their users’s posts to train models. The Verge reported on the breakdown in respect for the robots.txt file that has (so-far) been treated as “rules of the road” for scraping content; now AI companies are simply ignoring it. The results are quite apparent: The Times found Midjourney regularly created images that clearly violate copyright law.

Originally published at https://www.storybench.org on March 14, 2024.

New Paper about data theatre in Arts

Sun, 18 Feb 2024 05:09:00 +0000

Excited to share a new paper published by the Data Theatre Collaborative in Arts: Viewpoints/Points of View: Building a Transdisciplinary Data Theatre Collaboration in Six Scenes. Instead of a traditional academic paper, we chose to write up our study as a six-scene script, aiming to capture the complex dynamics of our interdisciplinary team and the process of creating data theatre. This unique format allowed us to bring our research process to life, focusing on how data science, theatre, and community engagement intersect to make data more accessible and meaningful.

Donovan, Halle and Finn work on their data theatre composition. Photo by Dani Snyder-Young

Using improvisation and movement exercises, we experimented with transforming abstract data points into relatable human experiences. For instance, one scene involved actors representing local housing data through physical space and movement, making visible the real-life impact of statistics on rent, income, and displacement. This approach let us explore the ethical questions that come with representing community stories through data and theatre, ensuring that these portrayals respect and reflect lived realities.

Presenting our findings as a play wasn’t just for creativity’s sake—it was a way to challenge conventional research presentation and address ethical concerns around data representation. By using the language of theatre, we invite our audience to think critically about how data affects real lives and to consider the role of community voices in data interpretation. This process emphasized that making data legible and impactful for communities requires collaborative methods that put people—and their stories—at the center.

Give it a read and consider how the format alters how you connect with and consider the research.

Your start of the year updates on artificial intelligence in journalism are here

Tue, 23 Jan 2024 05:09:00 +0000

This is third in my series of round-ups on the latest in artificial intelligence and journalism. Here are some recent links you might want to read.

Public understanding of AI and news

There is growing public concern about the impacts of AI and news, especially around the elections this year (in the U.S. and other countries). The U.S. Senate Judiciary Committee hosted a hearing on “Oversight of A.I.: The Future of Journalism”. Inviting publisher CEOs (and CUNY Prof. Jeff Jarvis), they touched on fair use, Section 230, deep fakes and more. Read a summary of the hearing at Ars Tecnica.

Concerns are growing in the regular public too. Our own Northeastern AI Literacy Lab published a report on how American are thinking about evolving AI technologies. Polling 1,000 people, they found a mix of caution, skepticism, and hope. A surprising 77% of respondents read news about AI at least once a week. Read the full report on “How Americans see AI: Caution, skepticism, and hope”.

Media outlets continue to try AI out

The variety of experimentation with AI technologies in news continues to grow. Norwegian daily Aftenposten is using AI to read articles with a synthetic voice, trained on recordings of their podcast host. Early data from their year-long experiment show no different in listen rates between human-voiced audio and generated audio. Read more at PressGazette.

Some of these experiments don’t go so well. The latest AI news scandal comes from Sports Illustrated. Media outlet Futurism found evidence that they had AI-generated fake reporters on their list of authors, and completely generated stories with no labels on them. Once contacted, SI deleted all the articles and blamed a sub-contractor. Read the scandal details on Futurism’s site.

In more ethical developments, the BBC laid out three guiding principles for connecting generative AI and their mission: the public’s best interests, prioritizing human creativity, and being transparent. Read their full statement for more on their plans.

Various papers consider copyright concerns

Global publisher Axel Springer signed a multi-deal with OpenAI. The agreement allows OpenAI to use published content as training data, and also will feature their publication’s content in results created by ChatGPT. Read more on Reuters.

On the flip side, the New York Times lawsuit against OpenAI continues to develop, built on the claim that their model’s copyright infringements “threaten high-quality journalism.” OpenAI has now responded, claiming it is merit-less. Read more on the TechCrunch blog.

Prospect’s navel-gazing “Media Confidential” (owned by Axel Springer) podcast interviewed German news leader Mathias Döpfner and had a short segment about what he thinks the future holds for AI and Journalism. His comments touched on legal frameworks and automation of routine journalism tasks. Read the transcript or listen to the whole interview on the Prospect site.

Originally published at https://www.storybench.org on January 23, 2024.

Keynote at Information is Beautiful 2023 Awards

Thu, 02 Nov 2023 05:09:00 +0000

I was honored to be invited to give a keynote at the 2023 Information is Beautiful Awards - Day of Inspiration, hosted by the Data Visualization Society. In the talk, called “Questioning Our Data Cultures” I decided to introduce some ideas from the book I’m working on, specifically about how our traditional approaches to data visualization and engagement need to evolve as data use spreads beyond its historical domains (science, statistics, and business) into new settings (newspapers, museums, civic engagment, CSOs, libraries).

The dominant cultures of data - surveillance, formalism, authority, and capitalism - don’t align well with what I call “pro-social settings” where the goals are participation, engagement, and empowerment. This misalignment creates barriers when we try to bring people together around data in community, civic, and advocacy contexts.

We need to address this gap, because it is creating disengagement, disempowerment, and active harms. I presented three alternative approaches I’ve been co-designing with community groups that better serve these new contexts: data scultpures, data murals, and data theatre.

These methods, and the underlying theories, form what I call “popular data” - a framework that emphasizes accessibility, participation, and meeting people where they are. This work will be detailed in my upcoming book from Oxford University Press, “Community Data,” as we continue to expand our toolbox for bringing people together around data in pro-social settings.

Creative Data Viz as Epistemological Resistance (AoIR 2023 talk)

Sat, 21 Oct 2023 05:09:00 +0000

I attended the Association of Internet Researchers 2023 conference last week in Philadelphia, where I presented my research on rethinking how we represent data in social justice and community settings. Read the short paper.

Utilizing the idea of “epistemological resistance” (borrowed from anti-colonial studies), I tried to challenge the dominant approach to data visualization that we’ve inherited from scientific and statistical traditions. While charts and graphs have become ubiquitous in community decision-making and activism, I argued that these conventional tools come with baggage - they privilege certain ways of knowing and understanding that don’t always align with goals of empowerment and community engagement.

To illustrate alternatives, I shared three powerful examples of creative data representation in community settings. In Mexico City, activists created a striking memorial by writing the names of feminicide victims on barriers around the main square (“Te nombré en el silencio”). Chicago protesters constructed a street data sculpture showing the city’s disproportionate police spending compared to other departments. And in Los Angeles, unhoused community members and activists collaborated on both a theatrical performance and mini-golf course to represent data about homelessness policies. These cases demonstrate how communities are finding creative, embodied ways to represent data that go beyond standard charts and graphs.

The core message of my talk was that we need to expand our data visualization toolbox to embrace multiple ways of knowing and understanding information. The standard approaches we’ve inherited from scientific fields don’t always serve the needs of communities using data for social justice and democratic engagement. By learning from activists who are already pioneering alternative approaches like data sculptures, murals, and performance, we can develop more inclusive and empowering ways to represent data. I’m excited to continue exploring this intersection of data representation, epistemology, and social justice - there’s so much potential for innovation here that could better serve communities working toward positive change. It will be a major focus of the book I’m working on.

Here are the updates you need on artificial intelligence in journalism

Wed, 18 Oct 2023 05:09:00 +0000

I’m continuing my series of round-ups on artificial intelligence and journalism. Here are some recent links you might want to read.

Reuters/Oxford and Journalism AI reports on what’s happening in newsrooms

The Reuters Institute and University of Oxford interviewed more than 40 new organizations from across the globe to uncover editorial strategies, deployment approaches, technical infrastructure needs, organizational needs and what might come next. Read the full report from The Reuters Institute.

JournalismAI spoke with hundreds of editors, journalists and technologists to inform their second global survey of what news organizations are doing with AI. The group is a collaboration between the journalism program at the London School of Economics and the Google News Initiative. Read the whole report on the JournalismAI website or the coverage of key findings on The Verge.

The Financial Times visually explains AI

We’ve all started using ChatGPT and other large language models, but how do they work? My favorite explanatory metaphor is to call them “super auto complete.” The Financial Times digs into the transformer models under the hood and visually explains how they are trained. Interact with the full story from the FT visual storytelling team.

More news sites blocking OpenAI scraping

Journalist Ben Walsh is tracking the robots.txt files of more than 1,000 news domains. This is the file websites use to indicate to web crawlers what they are allowed to access. He reports on X, formerly known as Twitter, that almost half are now including instructions to OpenAI that they should not scrape content from their websites.

Piloting AI tools for local newsrooms

With support from the Knight Foundation, the Associated Press wrote a report last year about AI in local news. Now, they are supporting five pilot projects that aim to put AI technologies in service of the needs of smaller local newsrooms. These include police activity summaries, automatic weather alerts, video transcriptions and summaries, city council meeting keyword extraction and editorial coverage planners. Read more on the NiemanLab blog.

Publishing cutting-edge research from academics

A new study from Journalism & Media looked at reader trust in AI-human reporting. Key findings relate to human control over integrations. Read the full paper open-access on the journal website.

If you’re an academic and do similar research on how journalists and newsrooms navigate the AI hype and practical needs, checkout the call-for-papers for the Digital Journalism special issue. Submit an abstract by Jan. 10. See more from Professor Tomás Dodds on X.

That’s all for this round-up. For more, check out our earlier update. Found something relevant and want to share? Add a comment below or drop me a line at @rahulbot@vis.social or @rahulbot@twitter.com.

Originally published at https://www.storybench.org on October 18, 2023.

Here are the latest updates on artificial intelligence in journalism

Thu, 14 Sep 2023 05:09:00 +0000

It seems like artificial intelligence is everywhere in the news, particularly in digital journalism circles. I’m keeping an eye on ongoing developments and will share an occasional round-up of links that might be relevant to you. Here’s what was happening in the last few weeks.

Google is pitching an AI-assisted news story writer.

This points to new ideas of what word processing tools might look like in the future. Imagine creating an alternative to Microsoft Word for journalists from scratch today knowing large language models were around to help you. Read the full story from The New York Times.

Should CNET be using AI to generate articles?

The CNET controversy might be the first industry event that would count as an AI scandal. Editors started using AI-generated articles that were just plain wrong. Before trying things out, maybe make sure you have some “responsible AI” principles in place. Read the full piece at Futurism.

Newspapers v. OpenAI

Many newspapers are starting to block OpenAI scraping of their content via the robots.txt file. Read more on ArsTechnica. The Times is going further and might sue, taking the “you’re using our copyrighted materials without permission” angle. The move comes weeks after the Times declined to join a coalition of media companies negotiating with Big Tech. Read more on NPR.

…but American Journalism Project partners with OpenAI

The American Journalism Project nonprofit is taking OpenAI money to distribute grants to smaller organizations to integrate AI tech. Read more at Columbia Journalism Review.

When do we (and do we not) use AI?

Northwestern University computational journalism professor Nick Diakopoulos is collecting and amplifying stories of “responsible use” in newsrooms. Browse the real world case studies at the “Generative AI in the Newsroom” project blog.

How should journalists use ChatGPT?

A recent University of Texas Knight Center webinar on AI and journalism was so oversubscribed that organizers had to switch from Zoom to YouTube. The session shared reflections on ChatGPT and best practices. Watch the hour-long session on their YouTube channel. More recently, they are launching a month-long online course to help others think through how to use generative AI in the newsroom. Register now (the course runs 9/25 to 10/22).

What to see next in AI tools

The Online News Association published the recording of an online session about AI tools with demos from Microsoft and Nota. The talk focused on AI as a support for various points in the writing and editing cycle, and shared early experiments and tools the companies are working on. Watch the hour-long session on ONA’s website.

That’s all for this round-up. Found something relevant and want to share? Add a comment below or drop me a line at @rahulbot@vis.social or @rahulbot@twitter.com.

Originally published at https://www.storybench.org on September 14, 2023.

A New Tool To Help Understand Partisan News in the US

Tue, 30 May 2023 05:09:00 +0000

The latest Zelda game is being covered a lot more in Democrat-serving online news sites. That’s one of the first random tidbits I noticed in “On Our Own Terms”, a new tool built by Claire Pan and I as part of the Media Could project.

Top terms from the week starting May 15th, 2023. I’ve highlighted the Zelda-related terms, which only show up on left-leaning sources.

There are more impactful details to be discovered in there, such as fact that Tucker Carlson’s dismissal from Fox News was covered in Republican-serving online news for longer than in Democrat-serving sites. However, this disparate coverage of Zelda is and intriguing thread to pull on in the future.

Top terms from the week starting May 1st, 2023. I’ve highlighted Tucker Carlson’s name which only shows up on right-leaning sources. This is over a week after he was fired on April 24th.

How we Breakdown Media by Partisanship

We created “On Our Own Terms” to let researchers and news-junkies perform a high-level comparison of terms used in news headlines across a political breakdown of media. This breakdown is a bit complicated to explain, but stick with me as a I run you though it. We’ve build on a Twitter user panel created by my colleague David Lazer’s lab, where they associated half a million Twitter users with their voter registration records. Mining that network for links shared in tweets, they’ve produced ratios indicating the comparative sharing between registered Democrats and Republicans by domain name. We’ve narrowed these domains into US-published news sites, and put them into 3 buckets: domains shared more often by registered Democrats on Twitter, domains shared roughly equally by registered Republicans and registered Democrats on Twitter, and domains shared more often by registered Republicans on Twitter. Hopefully I didn’t lose you in the middle of that, because the end results is three buckers of news sources that we can talk about as left-leaning, right-leaning, and central. That’s a bit hand-wavy, because we’ve not actually rating the sources or their content themselves, but it sure is a lot easier to explain than the rest of the paragraph above.

The “On Our Own Terms” dashboard uses this political breakdown of US online news sources to show you top words in headlines on a week-by-week basis. Why? We have more than a decade of strong evidence about the trend towards polarization in news sources in the US (particularly on the right). In addition language analysis shows how politicians are increasingly using completely different language and terms in congress. Combine there and you can see how as a country we’re beginning to have very different parallel conversations, creating serious threats to shared consensus on reality and overall civic cohesion.

Why Show Top Terms?

Almost 15 years ago the Media Cloud project began with one of the primary goals to map the broader media landscape. A word cloud of top terms was one of the first dashboards the team created (that predated my work on the project). We’re in the process of rebuilding right now, in collaboration with the Internet Archive’s Wayback Machine, so I wanted to honor and return to that history by building an updated version of that dashboard as a first step.

The mediacloud.org homepage circa 2011, courtesy of a snapshot on the Internet Archive’s Wayback Machine.

Looking at top terms in headlines can give informed news followers a quick summary of the events and people that are driving news, and sometimes even clues about the narratives. Of course, out of context terms don’t really explain what different stories are, so we’ve set it up so that clicking on a terms opens up a search in our online news archive for stories that have that term in their headline.

An example of searching for the amount of stories mentioning “tucker” in the week starting May 1st, 2023 in the Media Cloud search tool

An Evolving Tool

Over the summer we will be expanding on this dashboard, giving you an alternate way to explore the news. As I mentioned, we’re still reconstructing this searchable online news archive, so there will be some hiccups and bugs. That said, give it a spin if you’re intrigued and we think it’ll will reveal some interesting threads to pull on, perhaps even more relevant for our lives and civic structures than who is resporting on the latest Zelda game. Just bookmark On Our Own Terms and it will update automatically every week. Want to help? You can check out the source code and historical data in our open-source code repository.

Teaching Physical Computing with Mini-Mini Golf 🤖⛳️🎉

Wed, 03 May 2023 05:08:31 +0000

My family knows one thing about vacationing with me: if we’re anywhere near a mini-golf course we’ll have to stop and play it. Waterfalls, windmills, pirates, animals — I love it all. This semester I was teaching Physical Computing course again, and I wondered… could I combine my love of the quirky American leisure activity with Arduinos, sensors, motors, and more? So I decided to have students design interactive tabletop mini-mini golf holes for a public showcase.

Me trying out the Phineas and Ferb hole. (Photo by Matthew Modoono/Northeastern University)

What is Physical Computing?

Physical Computing is a phrase we use to talk about the craft of building systems that sense and react to the world and us. It is a wonderful way to introduce learners to working with sensors, motors, LEDs, and more because it exists in the real-world and not just on-screen. There is a certain whimsy, delight, and magic to working with products like the micro:bit, Arduino, and Raspberry PI. This year is the second time I’ve taught Physical Computing course here in Art + Design at Northeastern University, so I decided to try some new things.

We used the SparkFun RedBoard from their Inventor’s Kit. (Photo by Charles Gauthier)

Typical introductions follow the mold of Tom Igoe’s decades of work at NYU’s ITP program, they offer invaluable resources, pedagogy, and examples to learn from. With a lack of access to a group soldering space due to construction, I decided to go all-digital in my course. This meant I was focused on microcontrollers and modules that could connect to them easily over things like I2C and other busses to extend core capabilities.

Sadly, another space-driven constraint was that we couldn’t design full-sized mini-golf holes. So taking a cue from my wife’s work with the Beautiful Stuff Project, I decided to work on square 2-foot wooden boards. Students created table-top holes playable with marbles and 3d printed palm-sized putters.

Conceptualizing Interactive Mini-Mini Golf (with AI)

Mini-golf holds a nostalgic charm in the US, associated with family trips and simple fun. The history is connected to this, from the emergence of putt-putt as a pastime associated with the expansion driving and the highway network. Some of my students had fond memories of mini-golf, while others hadn’t ever played. To help get into the mindset I took the whole class on a field trip to a recently opened novel new Puttshack indoor mini-golf place. It included a bit more gamification around the playing experience, and lots more LEDs, but the experience was fun and also helped set the mood and tone for the project.

Mini golf takes a variety of forms across the US.

Brainstorming afterwards, we narrowing in on a course theme of “cartoons” — they all had some animated TV show that they liked and thought would translate well into recognizable physical form. As a test of teaching how to brainstorm with new “AI” tools, I introduced using GPT and other large-language models. My first prompt? “What obstacles would you build in an interactive mini golf hole based on the TV show Tom and Jerry?” The suggestions were good, but not that creative; they were the kind of things we immediately thought of.

OpenAI’s Chat GPT-3 has on-target, but rather obvious suggestions.

Continuing, I introduced using Dall-E and Stable Diffusion for generating image-based suggestions on how holes might look with those themes in mind. These were a bit more off-the-mark.

OpenAI’s DALL·E 2 had some rather hallucinogenic suggestions for mini-golf holes with a SpongeBob theme.

Overall the novelty of brainstorming with AI tools was compelling, and engaged students, but the actual utility of the suggestions wasn’t that great. However, like the field trip it did seem to help get them into the right playful and nostalgic mindset, especially for folks that hadn’t ever played before.

Soon we were off and running with five holes, each with a cartoon theme:

Tom and Jerry
Phineas and Ferb
Cowboy Bebop
SpongeBob SquarePants
Kim Possible

A Pop-Up Putt-Putt Event

Geared down motors, shake sensors, mp3 players, servos, neopixel strips — students pull together all sorts of wacky themed obstacles to create their holes. A laser-cut Tom the cat smashed a hammer down to block your path to the hole. Spinning record turntables decorated with Cowboy Bebop iconography redirected your ball away from your target. Eugene Crabs used a claw to try and deflect your ball from heading up a hill. The Kim Possible jingle played to alert you when an obstacle would move. A cage-o-matic created by Dr. Doofenshmitrz raised and lowered over the hole.

After two quick weeks of building and playing, students were ready (enough) to show these interactive obstacles at a 2-hour pop up playing event. Working with the Center for Design , I set up each hole on a table to create a five-hole course. We made score cards, 3D printed hand-sized golf putters, and invited in the Northeastern community to play.

Prof. Pedro Cruz joined us and played the students’ SpongeBob themed hole. (Photo by Matthew Modoono/Northeastern University)

The Tom and Jerry hole, amongst others, included some laser-cut character obstacles. (Photo by Charles Gauthier)

Students chatting around the Kim Possible mini-mini golf hole. (Photo by Charles Gauthier)

The event was a delight. Students’ friends came in to play, and faculty chatted with me. The holes worked and failed as you might expect; building reliable and repeatable physical computing devices is hard, especially with humans involved. That was one of the main goals of the project, to experience using motors and sensors in a short setting where things had to interact with real people. Overall we had a blast, and I’m excited to teach this again and expand it into a longer unit. Maybe with enough studio space we can create full sized holes.

Data journalism? You can do it.

Tue, 28 Feb 2023 05:08:31 +0000

Data is still hot, but the new skills, math, and technologies can feel overwhelming. In my experience journalism students and professionals approach learning data journalism with both excitement and trepidation. However, over a decade of teaching data literacy to many types of learners I’ve found that journalists are some of the best positioned to dive into data storytelling. If you’re on the fence about trying data journalism, consider this piece a pep talk. I’m here to convince you that fear is the main barrier. If you overcome it you’ll find that you are well prepared to start working with data, and that there is a robust and supportive community waiting for you.

Generated artwork from DALL-E depicting a data journalist.

Increasingly data is central to telling the most important stories of our time, and accordingly the tools and community have matured significantly. Now is a great time to try your hand at integrating data into your stories, creating new ways for readers to understand the issues you cover while simultaneously taking advantage of the popularity of data visualization to bring in new readers.

Your Skills Translate

For an organization with global reach, the UN World Food Programme headquarters are in a rather unassuming office park outside of Rome. About 5 years ago I found myself in a lively discussion there about the meaning of the Spanish word “contar” (with a Colombian attendee of a data literacy training I was helping at). The language of “counting” is a common part of data production; it is quantitative data production. I was describing this with the Spanish verb “contar”, which I intended to mean “to count”. My new Colombian friend reminded me that it also means “to tell”. As this dual meaning of “contar” suggests, from the moment of production data is just us telling ourselves stories. Journalists are storytellers; who could be better equipped to do data work than them? In English the closest analogue is perhaps “account”, where “to account” for something could be the act of counting, while “an account” is often a story of something that happened. This powerful duality in the terms connects the act of recording to the act of storytelling – literally the same word is used for both.

Many core journalism skills directly help in data work.

Many journalism skills easily translate to data work. Consider the first step of working with data - creating it. What would you call collecting a set of interviews? I’d call that producing a dataset. Journalists regularly produce and merge multiple datasets as part of their everyday reporting. What about validating a dataset to verify it is accurate? Sounds a lot like fact-checking to me. Newsrooms, and the profession at large, have robust theory and practice for ensuring the accuracy of the data they collect. How about being critical of datasets and what biases they bring? Journalists are primed to be skeptical of sources, to look for multiple viewpoints; yet another skill that translates directly to working with data. Assembling many datasets into a consistent and compelling narrative? Again, we see journalists doing this regularly by pulling in descriptions, quotes, perspectives and more.

Your journalism skills and training translate directly to telling data stories.

You Can Math

Years ago I visited the classroom of someone that helped me on my journey into data science - Professor Allen Downey. His books on programming and statistics are must reads for anyone learning programming and data science, and were particularly valuable for me as I tried to fill in gaps in my own self-taught skills. Allen generously invited me into his Olin College classroom to introduce his students to a different approach to storytelling with data. As I contrasted storytelling with statistical analysis, he jumped in with a way of talking about math that has stuck with me since then. He pointed out that math, at its simplest, is just counting. At its most complex, it can include rigorous statistical methods with lots of Greek letters. But there is, he argued, a LOT of space in-between. The in-between includes many techniques that offer analytic power to non-statisticians. Normalization is just smart dividing. Binning is just basic rounding. Here was someone that wrote a widely read book on programming and statistics arguing that you didn’t have to be a statistician to produce useful and reliable data analysis. I’ve hung onto this idea ever since, introducing it over and over again in workshops.

Knowing when to normalize, when to bin, when to remove an outlier - these are the challenging concepts. Math is not the hard part; most of it is just counting and even my middle school age kids are pretty good at that (although my son did struggle with forgetting that the number 4 existed for a while).

Tools are Getting Better

I wrote a whole academic paper on the challenges of teaching data journalism in a world of “tool overload”, so I truly feel the pain of navigating the overwhelming set of options for working with data online. There isn’t one tool to rule them all, but there are some compelling examples demonstrating how computational innovation is making data tools easier to learn and more widespread in use.

It used to be that you either learned Excel or programming, but now there are robust major tools like Tableau that bridge the gap. While they can’t help you figure out the right standard chart to support your story, chart choosers help you move from narrative form to a chart that can tell that type of story. Check out the DataVizCatalogue for more summaries of chart types and their intended uses. Worried about production value? DataWrapper is a great example of a simple-to-use tool that produces great looking charts online. Want to make more advanced chart types? RAWGraphs lets you just paste in your final data and try out some more novel chart types you might feel intimidated by.

Some leading tech for creating interactive data pieces emerged from news organizations.

In this industry this isn’t just a story of newsrooms adopting tech startups’ latest products; impactful innovation has come from newsrooms themselves, or intentionally built for them. The Flourish platform for online data visualization was conceived and developed for newspapers. They’ve had so much success that the online design app giant Canva recently acquired them. The key author of d3.js, the go-to technology for creating custom interactive data visualizations, is Michael Bostock. After creating d3.js at the University of Washington he spent years creating award-winning interactive pieces at the New York Times. More recently, the Svelte framework has taken the Javascript developer world by storm, praised by developers for its ease of use. The primary developer, Rich Harris, was a journalism student who learned to code (short documentary). He created Svelte while working at the Guardian.

Still overwhelmed by online charting or analysis tools? Don’t be afraid to return to your paper and pencil like celebrated data visualization designer Giorgia Lupi so often does. I often argue that there’s power in more informal approaches to data visualization.

The Community is Still Growing

You’re not alone on this journey. The community is growing and moving from its technological roots to embrace more diverse sets of data storytelling learners. Reddit’s DataIsBeautiful forum is a great place to get inspired about what to make (or what not to). The Data Visualization Society is a relatively new organization hosting welcoming events for learners of all types. Books like Scwabish’s Better Data Visualization and Fiegenbaum & Alamalhodaei’s Data Storytelling Workbook offer concrete and practical advice and guidance. Here where I teach at Northeastern University our Storybench blog regularly shares tutorials and behind-the-scenes stories from the field to help you see data storytelling in action.

As a journalist, you’re better equipped than most to enter the field of data storytelling. You should try it, because it will make your reporting — and the community itself — stronger.

Thanks to my colleague Meg Heckman for advice on an early draft of this piece.

Originally published on the StoryBench blog.

Digital Storytelling to Support Connective Journalism

Wed, 08 Feb 2023 05:09:00 +0000

Journalism serves many roles in society - informative, investigative, normative, and more. As the tools and pratices of interactive digital storytelling continue to grow, how can they help the connective role journalism plays in society? Read on for some background and a recent experiment I did in creating a digital story focused on building community connections.

The Roles of Journalism

Communciaton scholars often talk about the differing role communcaiton plays in society. Media critic and journalist James Carey is credited with a popular distinction, which teases out the idea of communciaton as transmission and ritual. The latter thinks about how communication plays a role in social interaction and construction of community. Journalism certianly plays the informative role, but also the serves what Carey would call ritualistic functions.

Journalism Professor Jeff Jarvis argues that community in fact means connecting people intimately over time. His definition of journalism centers on “conveneing communities” to build share understanding and world views. Journalism functions as a form of civic participation - increasing people’s feeling of community and willingness to participate in that community. The decline of local news in the US has demonstrated this; people’s sense of community, involvement, and pride decreases as local news sources dissapear. We need more journlaism focused on community-building role.

From a lens of community connectivity, the digital transformations of the last 30 years have created gains and losses. Take classified sections as an exmaple. Most discussions of the migration of classified ads from digital news sites to Craigslist focus on the massive loss of revenue, but with this ritual role in mind it can also be thought about as a loss of community connection. Browsing the classifieds, and posting a listing, instilled a sense of belonging; one associated with the individual and the community but also with the newspaper that hosted it. The news organization often was central to the relationship between the self and the place. Losing classified ads was a blow to the connective role news organizations played. Taken more broadly though, digital storytelling created new ways to discover your place and community as a reader. Consider browsable maps or databases of local businesses as an example. In addition (the early days of) comments and citizen journalism created new digitally-mediated ways to build that sense of community via a news provider’s website.

Scholars Dr. Regina Marchi and Dr. Lynn Schofield Clark offered the idea of “connective journalism” to get at the new ways that role plays out in the diverse media landscape of online news and social media. Sharing links, liking them, reposting them - these are ways people engage with one another that build connective tissue in journalistic ways. That mediates a sense of self and relation to a larger community, especially for marginalized groups like youth. Digital news interactives can be crafted to specifically try and build community connectivity in ways inspired by that work.

Building An Example

So what about a broader application of this idea of building for community connectedness? How can we more intentionally use digital storytelling to support the connective role of journalism? I decided to explore this by piggybacking on the attention to the men’s FIFA World Cup late last year.

The tournament, held in Qatar, was widely criticized for labor issues, gender rights issues, and overall corruption. However, it is still the largest sporting event on our planet, commanding massive amounts of media and popular attention. Rather than further reinforcing this flow of attention and money into FIFA’s pocket, I decided to try something different. I built a digital story to connect readers with local immigrant fans and their cultures.

Screenshot from the Our Cup interactive.

Our Cup uses the reader’s rough location and census data to discover the largest three local immigrant communities from countries competing in the World Cup. I was able to precompute this data for every county in the US via the Census ACS “foreign born population” data. I manually curated a set of information for each country that I thought might be “culturally relevant” – things that might connect a reader to that community and their culture. Specifically the country links to:

Local restaurants service country’s cuisine, sourced from Yelp
Recipes from the country you can make at home, sourced from Yummly
Background about the country, via Wikipedia
A few playlists on Spotify, via Every Noise at Once
A team guide, which The Guardian sourced from local journalists
A heat map of where in the US immigrants from that country live

The goal here is to give readers who might not know about local immigrant populations a chance to learn about them via the media event that is the World Cup. The links push from the sport story to a connective goal, from awareness of the local immigrant communities to direct or indirect engagement with them.

More Experiments Needed

This piece served as a “thing to think with” for me, a chance to explore what a digital story that is built with connective journalistic goals in mind could look like. Is it a success? In one way it is, because it helped me think this question through. In another way I can’t really tell, because I didn’t do the qualitative work to talk to readers about their perceptions before and after. In yet another way it isn’t a success, because it is a bit of a toy. That is all right with me, because I learn best by playing with toys.

I’ll keep looking for other people playing in this space, and the toys they are building. Hopefully we can continue to innovate how and why we build digital stories. There’s more to explore, and we need to think harder, in order to flesh out how approaches to digital storytelilng can support the important connective role journalism plays.

Thanks to my colleague Fernando Bermejo for advice on an early draft of this piece.

Originally published on the StoryBench blog.

New Paper: Taking Data Feminism to School

Tue, 05 Jul 2022 05:09:00 +0000

Excited to share a new paper out in the British Journal of Educational Technology. I worked with collaborators to assess what data feminism looks like in K-12 data science education. We retrospectively review 42 youth data programs and projects, assessing each against the key principles of data feminism.

Taking data feminism to school: A synthesis and review of pre-collegiate data science education projects

Victor R. Lee, Daniel R. Pimentel, Rahul Bhargava, Catherine D’Ignazio

Paper Abstract

As the field of K-12 data science education continues to take form, humanistic approaches to teaching and learning about data are needed. Data feminism is an approach that draws on feminist scholarship and action to humanize data and contend with the relationships between data and power. In this review paper, we draw on principles from data feminism to review 42 different educational research and design approaches that engage youth with data, many of which are educational technology intensive and bear on future data-intensive educational technology research and design projects. We describe how the projects engage students with examining power, challenging power, elevating emotion and lived experience, rethinking binaries and hierarchies, embracing pluralism, considering context, and making labour visible. In doing so, we articulate ways that current data education initiatives involve youth in thinking about issues of justice and inclusion. These projects may offer examples of varying complexity for future work to contend with and, ideally, extend in order to further realize data feminism in K-12 data science education.

Download the paper from BJET website

Upcoming AMC FAccT Talk: Towards Intersectional Feminist and Participatory ML

Mon, 20 Jun 2022 05:09:00 +0000

We’ll be presenting our collaborative work on the Data Against Feminicide project at the 2022 ACM Conference on Fairness, Accountability and Transparency. We’re very excited to put forward this work as a case study in intersectional feminist and participatory approaches to machine learning.

Towards Intersectional Feminist and Participatory ML: A Case Study in Supporting Feminicide Counterdata Collection

Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Giulia Taurino, Wonyoung So, Catherine D’Ignazio

Paper Abstract

Data ethics and fairness have emerged as important areas of re- search in recent years. However, much work in this area focuses on retroactively auditing and “mitigating bias” in existing, poten- tially flawed systems, without interrogating the deeper structural inequalities underlying them. There are not yet examples of how to apply feminist and participatory methodologies from the start, to conceptualize and design machine learning-based tools that center and aim to challenge power inequalities. Our work targets this more prospective goal. Guided by the framework of data feminism, we co-design datasets and machine learning models to support the efforts of activists who collect and monitor data about femini- cide — gender-based killings of women and girls. We describe how intersectional feminist goals and participatory processes shaped each stage of our approach, from problem conceptualization to data collection to model evaluation. We highlight several method- ological contributions, including 1) an iterative data collection and annotation process that targets model weaknesses and interrogates framing concepts (such as who is included/excluded in “femini- cide”), 2) models that explicitly focus on intersectional identities rather than statistical majorities, and 3) a multi-step evaluation process — with quantitative, qualitative and participatory steps — focused on context-specific relevance. We also distill insights and tensions that arise from bridging intersectional feminist goals with ML. These include reflections on how ML may challenge power, embrace pluralism, rethink binaries and consider context, as well as the inherent limitations of any technology-based solution to address durable structural inequalities.

Download the paper from the ACM digital library or the FAccT website

Upcoming talk at PaCSS'22: Partisan Media Coverage and Intersectionality

Thu, 16 Jun 2022 05:09:00 +0000

I’ll be speaking at the 2022 Politics and Computational Social Science conference today with my colleague Meg Heckman. We’ll be presenting work, with Emily Boardman Ndulue, that took an intersectional lens to analyzing how online news media covered the election of current US Vice President Kamala Harris.

Partisan Media Coverage and Intersectionality: A Case Study of Vice President Kamala Harris

Rahul Bhargava, Meg Heckman, Emily Boardman Ndulue

Talk abstract

Partisan news coverage of politicians is well documented and studied, as are the many ways sexist and racist tropes have historically played out about those who identify as women, people of color, or both. Our current U.S. online media environment, meanwhile, has been shown to have become increasingly polarized. Kamala Harris’s election to the vice presidency presents a unique opportunity to study how these trends play out with regards to her various identities (woman, Black, Southeast Asian, mixed-culture household). To analyze coverage of Harris, we built on prior work to create collections of online news media grouped by a partisan score based on link sharing in a panel of Twitter users associated with voter registration records. We gathered and analyzed a corpus of news stories about VP Harris from these sources from August 2020 through April 2021 (n=17,165). Employing computational and qualitative methods, we found strong evidence of coverage of Harris playing out differently across an asymmetrically polarized media landscape. We found evidence of varying levels of sexist and racist tropes in all sources—but on the far right we discovered a particular cluster of (often factually incorrect) narratives that did not appear elsewhere on the partisan spectrum. Our work contributes to our growing understanding of how online media portray female politicians with multiple identities, suggesting that new intersectional biases may play out as this population grows.

Download the slides

Read the associated piece for Nieman Labs: It’s O.K. to write about women, fashion, and politics — but here’s how to do it better

Helping Computers Find Food in Text

Wed, 15 Jun 2022 05:09:00 +0000

Computers are good at processing large amounts of information, but bad at intuiting what that information actually is. For an ongoing research project looking at mentions of food in online media, we’re trying to help computers get better at recognizing entities in unstructured text. Given the text of an arbitrary webpage or social media post, how successfully can we get a computer to pull out all the words that are food?

How do we recognize foods in texts? source: Bon Appetit

The term “named entity recognition” describes a set of algorithms designed to adapt computational language models to understanding unique classes of information. As a subset of natural language processing (NLP), the goal of named entity recognition is to assign unique values to words and attribute some meaning to them. This contrasts with most other parts of the NLP pipeline, which assess generalized, syntax-related structures like part of speech and dependencies. The focus of this blog will be on how named entity recognition and adjacent strategies can form structured categories for masses of unstructured data sourced from food-related content. Within computational linguistics, a variety of approaches have formed to optimize the extraction of this sort of information.

Technical Approaches to Named Entity Recognition

Some approaches to named entity recognition do not engage much with machine learning environments. Researchers (generally speaking) have three different types of named entity recognition at their disposal:

The first, referred to as the “dictionary-based approach,” catalogs as many known named entities as possible within categories. This approach makes intuitive sense since human recall of “named entities” might engage the same sub-categorization process. However, from a computational perspective, dictionary-based recognition can incur a large memory overhead. Additionally, this approach inefficiently allocates this memory; infrequently used words or archaic terms take up space without offering a large use value. More importantly, the dictionary approach cannot incorporate new words into its lexicon, as it has no means of categorizing available words into existing datasets. Here, intuition breaks apart since the human brain can do this by processing new information about unfamiliar entities. Even if this hurdle were overcome, the dictionary might struggle to categorize words with a variety of applications in different “subcategories.” Thus, the dictionary-based approach does not offer suitable performance for our task at hand.

The second, known as the “rule-based approach,” employs linguistic patterns and context in order to determine classification of words. Consider the sentence “put the pot of pasta on the stove.” If we are looking for the food term, we can use linguistic features to get there. We can see the entity we are searching for, “pasta,” and within English syntax we can locate this as a descriptor of the direct object of the sentence, “pot.” Within a computational context, the source for rule-based named entity recognition comes in tagging the dependencies of individual words within a sentence; it utilizes the dependency structure, as opposed to semantic proximity, to determine the likelihood of a word appearing as a named entity. This approach still requires the use of machine learning, since neural networks must model sentences into a list of dependencies, and a training set that describes the named entity and its location within the dependency structure must be created for this approach to succeed. However, the reliance on linguistic rules can lessen the complexity of the algorithm, as well as improve accuracy of findings regardless of irregularly named entities, such as those with two or three words. This approach sees benefits within the silo we are working in. Research of food recognition has employed a rule-based approach to high rates of precision and recall. Rule-based approaches show some promise for our task, and we’ll return to them later in this discussion.

Finally, the “machine-learning-based approach” relies purely on the proximity of words to one another to predict named entities. Using a labeled training set of data, the machine attempts to classify entities, and then checks its work. Over time, the system should become capable of determining food entities, independent of anything other than the string of text fed to it. This is computationally heavy, but a powerful means of inculcating independent machine recognition. The benefits to this independence are immense; it means that over time, irregular patterns, and unfamiliar terminology can be incorporated with greater success and reflection of reality. As such, most research around named entity recognition has coalesced around this approach, taking insights where needed from the prior two.

One approach to entity classification via machine learning begins with the word2vec - high dimensional vectors representing word meanings and use. In our context, it is helpful to understand word2vec as a prerequisite to named entity recognition techniques (as opposed to its own, freestanding method). Broadly speaking, word2vec utilizes the machine learning approach described above to categorize words based on their proximity, and then mutates auxiliary data structures to reflect this. Some implementations, like skip-grams and bag-of-words, do this very directly through enumeration of words within a given proximity. However, I wish to focus on a word embedding implementation. Word embedding quantifies the utility of a given word through representing the word as an n-dimensional vector. At first, these numbers are random, and produce little predictive value; however, through continual training, a model tweaks the values of the word based on its contextual placement. In time, the set represents the proximity of the word to a cluster of adjacent vocabulary (Altosaar). Word embedding is the bedrock of applied named entity recognition. For a machine to understand the use value of a word, we must first make the linguistic assumption that words with similar applications (even if not synonymic) co-occur more frequently than otherwise. Word embedding condenses this process into numerical representations, similar in nature to hashing in cryptography. Embeddings are also, then, predictive in nature. By assigning clusters of relative proximity, we can utilize word embeddings to guess consecutive words in a string. Learn more in this helpful write from Carnegie Mellon.

Recognizing Food Entities

With these approaches defined, how can we incorporate lessons from their use to create an effective named entity recognition model for food? To answer this, we can look at some strategies employed by relevant projects in the existing academic literature.

The first of these, produced by scholars of the Complex Systems Lab based in New Delhi, combines the rule- and machine-learning- based models to create a graph which reflects the recognition and tagging of ingredients, the processes employed to mutate the ingredients, and the equipment used to facilitate the processes. First, the model iterates over a set of labeled ingredient data. This data incorporates a variety of attributes, such as quantity, temperature and state of ingredient (e.g. chopped, freeze-dried). With this information, parts of speech and dependencies are then flagged to create relationships to processes and utensils. Finally, an NER tagger is trained to deploy flags on word tokens used to describe processes and utensils, like “mix” and “whisk.” The result of this collection is a repository of inter-connected entities, bound together by relationships inferred from their parts of speech in shared sentences and dependencies on one another.

This approach is at the scale of a single recipe, as opposed to a project that might analyze individual entities over a swath of recipes. This might come as the greatest limitation of this research. An approach based on modeling the relationships within a single entity can be difficult to use larger datasets, as processes and utensils can vary greatly based on application, and do not provide too much valuable information about which foods are used and how they are described. Additionally, the emphasis on the food-making process obscures our focus on the food created, as opposed to its components. Because of this, the object of this research might not seem immediately pertinent to our interests.

However, there are many insights to garner from the methodologies employed. First, the use of some rules-based strategies within the implementation of the machine learning model can be a great asset to our understanding of food entities. Regardless of the desired outcome of our project, using dependency structures to inform how we process food entities can be of use. With our present work, more support is needed to facilitate our model’s recognition of multi-word entities. The use of dependencies can help tag these words as joint entities. Further, the use of multi-faceted ingredient data, such as state and temperature, can add nuance to the sort of final meals described.

The second approach utilizes a rule-based approach in order to flag entities as food classifiers. Developed by Macedonian and Slovenian computer scientists, “FoodIE” utilizes a basic NER model at first but iterates over the training data to develop predictive models based on parts of speech and dependency. Through parti of speech and dependency analytics, the semantic classifications assigned to named entities are more nuanced than other models. Advanced semantic tagging describes each item in the sentence, along with possible classes that the item relates to (for instance, “beef” is classified under the subfamily of “bovines”, and “soup” has additional classifications of “wave” and “cloudy). FoodIE is a highly successful model, with 97.6% and 94.3% precision and recall rates. The model can also distinguish between dishes with more than one word, even in the case of compound nouns like “fruit salad.” This suggests that the more a model can incorporate insights from the context of the text, the more granular its observations will be.

From these case studies, as well as the survey of literature on named entity recognition, two pertinent insights should be incorporated into our research. First, our model needs to better identify multi-word entities. Thus far, the model atomizes each word and does not consider context when flagging words. One tool to adopt from the second approach is dependency checks. If our model discovers a classifier belonging to food (which it does with high recall), we must build support for it to consider incorporating surrounding words into the final entity. The model is already capable of this for other tags developed in the broader NER model but does not have the power to do so for our tags. Second, we ought to consider lemmatizing our entities. I have noticed that our early iterations of the model struggle to comprehend words used in different contexts with hanging characters (for instance, our model currently tags egg, but not eggs). This is a simple fix – SpaCy describes its pipeline as lemmatizing succeeding named entity recognition, but this is not fixed, so the order of processing can be inverted. We might want to use this inversion more broadly by scanning for entities first, then running a set of algorithms to determine the belongingness of adjacent words in dependency or part of speech. For lemmatization, however, that must precede NER.

Conclusion

Generally, incorporating named entity recognition techniques will prove essential on our own quest to map cultural signifiers attached to food. The choices we make in how we recognize food-based entities as such varies greatly depending on our use case. A survey of relevant literature grounds our research by providing leads to solve problems along the development process as well as spark new ideas for manipulating existing datasets. No approach that we choose will yield perfect results, so it is important to know how to manage our margin of error to fit the task at hand. Ultimately, we hope to make decisions that harmonize the flow of information at each technical step of our model’s pipeline. Once it is performing well, we look forward to sharing this model.

Understanding the 2020 “Racial Reckoning” In the Media

Thu, 09 Jun 2022 05:09:00 +0000

Just over two years ago, on May 25th, 2020, George Floyd was murdered by Minneapolis police officer Derek Chauvin. The shocking video of the incident quickly circulated across the internet through social media. Mr. Floyd’s murder sparked countless protests across the country and globe in the months that followed, capturing the public attention and initiating a national resurgence of the civil rights movement, often described as a “racial reckoning.”

Individuals, organizations, industries, and institutions across the country were again forced to acknowledge the realities of systemic racism in modern America. Journalism and news media were no exception. Media organizations in particular have a responsibility to model anti-racist practices not only in their internal policies and actions, but in their public-facing role of crafting narratives and framing national discussion as well. Researchers are finding impacts of this already – our own news analysis after the 2014 death of Michael Brown showed increased media coverage and discussion of the deaths of people of color at the hands of police. Another analysis of digital news following Mr. Floyd’s murder showed similar trends of increased coverage which mostly portrayed protestors positively. Mr. Floyd’s death and the ensuing Black Lives Matter protests dominated public discussion and demanded cultural changes - how has the media landscape shifted since then?

Acknowledging a Problematic History

Newsrooms have historically been, and continue to be, predominantly white. Majority white newsrooms were criticized following their coverage of the Civil Rights Movement in the Kerner Commission’s report of 1968, and in 1978, the American Society of News Editors (ASNE) made a commitment to greater newsroom diversity and declared a goal that newsrooms would be reflective of their community demographics by 2000. In 1998, only 11.5% of daily newspaper reporters were journalists of color, still far below the stated 26% goal. The ASNE then postponed its goal date to 2025, but a 2017 analysis still found actual statistics to be far from the stated goals.

Photo of a Black Lives Matter Protest in 2020 by Jared Wickerham for the Pittsburgh City Paper. (Source)

Even as newsrooms slowly become more diverse, professional journalistic structures and practices are still major obstacles to fully addressing the profession’s preservation of whiteness and exclusivity. Just one example of this occurred at the Pittsburgh Post-Gazette in June of 2020, when editors at the paper prevented two of their top reporters, both of whom are Black, from reporting on the Black Lives Matter protests in Pittsburgh. This occurred in part because Alexis Johnson, one of the excluded reporters, shared a tweet of her own personal commentary on the protests that she felt was funny and thought-provoking. Despite no staff social media policy, the upper-level management rejected her multiple story pitches and eventually outright excluded her from reporting on the protests because they felt that further coverage could be seen as biased and cause “the credibility of the newsroom [to] be questioned.”

Objectivity and neutrality, which have long been core tenants of journalistic practice, have contributed to maintaining white perspectives in professional journalism as the only form of “objective” and “neutral” reporting. Ultimately, this means that reporting that is not racially informed is considered properly neutral and objective, furthermore promoting the neoliberal ideal of color-blindness. In contrast, racially informed reporting, reporting from journalists of color, and stories for audiences of color are viewed as biased storytelling or relegated to niche media.

The dedication to “objectivity” in reporting is just one example of centering white perspectives in journalism; multiple publications have admitted to producing overtly racist content and enacting discriminatory business practices in both the past and present.

Screenshot from Editor-in-Chief Goldberg’s 2018 letter about National Geographic’s history. (Source)

National Geographic began their own racial reckoning in 2018, when Susan Goldberg, the magazine’s first female Editor-in-Chief, acknowledged the publication’s problematic past of disregarding American people of color and pushing stereotypes of exotic and primitive native people around the world. In the years following, the magazine hired more women, publicly covered LGBTQ+ issues, and added two women of color to their executive team.

National Geographic recommitted to this work during the summer of 2020, and staffers say there have been improvements but more are needed, structurally and socially. Lower level staffers felt that changes in coverage, content, and workplace culture were lacking and that their pushback was met with resistance from upper-level leaders.

Across many media organizations, lower-level staffers have felt the added pressure to advocate for these changes and were sometimes met with dismissal, leading to intense stress and a complicated relationship with the brand and their work. Unfortunately, vowing to value and prioritize people of color and then continuing to ignore or tokenize their ideas and careers is not uncommon, as an investigation by the Nieman Lab found in newsrooms and Diversity, Equity, and Inclusion (DEI) positions at publications across the country. These reporters shared anecdotes which included feeling overlooked, overworked and emotionally drained, in addition to being stifled by the editing process and limited in their ability to adequately call out instances of racism and sexism.

Varying Paths within One Publisher

An interesting contrast exists between the responses from two prominent Condé Nast publications: Bon Appétit and Vanity Fair. In late May of 2020, Adam Rapoport, then-Editor-in-Chief of Bon Appétit, wrote a column addressing the murder of George Floyd and affirming the magazine’s editorial mission and values of justice; just a few weeks later he resigned when a picture surfaced of him in brownface. Following his resignation, staffers opened up about the toxic work environment at Bon Appétit, where BIPOC staffers were taken advantage of, underpaid, and mistreated. Assistant Food Editor Sohla El-Waylly shared that she had been paraded as a token of diversity in Bon Appétit’s popular Test Kitchen videos, but her appearances were not compensated, unlike her white coworkers. In addition, Bon Appétit’s content contributed to the overall food-industry trends of white-washing. This is exemplified by instances of white cooks and creators taking credit for and profiting off of diverse recipes that are painted as “trendy,” when they are not outright ignored or discredited.

Screenshot of the web edition of Vanity Fair’s September 2020 edition. (source)

Though institutional issues at Condé Nast remind us workplace and content discrimination was most certainly pervasive throughout the company, some individual publications fared better than others. Vanity Fair has made strides in inclusive coverage since Radhika Jones took over as Editor-in-Chief in 2017, making her one of the only top editors of color in Condé Nast’s history. Immediately after taking control, Jones championed increased content featuring people of color, especially as cover stars. Following the 2020 summer of Black Lives Matter protests sparked by instances of police violence against unarmed Black people including George Floyd and Breonna Taylor, Ms. Jones assisted in the curation of an issue that highlighted Black stories and voices. The September edition, which is annually one of the most important issues, was guest edited by best-selling and highly-revered author Ta-Nehisi Coates and featured a portrait of Breonna Taylor on the cover. The issue highlighted content from multiple Black writers and creators, sharing stories of everything from the Black Lives Matter and abolition movements to the realities of unpaid college football. This edition of the magazine, titled “The Great Fire,” exemplifies moral solidarity in journalism. Moral solidarity in the news portrays marginalized people as “subjects of justice” with personal and political agency, and requires comparatively more privileged groups of people to amplify the experiences, desires, and solutions put forth by the affected communities.

As part of Condé Nast’s company-wide reckoning, they pledged to incorporate more inclusive hiring practices to diversify their staff at all levels, including editorial. As part of their accountability goals, Condé Nast committed to releasing an annual Diversity & Inclusion Report, which includes statistics on new-hire, editorial, and overall staff diversity. In 2021, 32% of all U.S. staffers identified as people of color (compared to approximately 28% in 2020) and 41% of new hires were people of color (compared to approximately 38% in 2020). However, as of 2021, 78% of Senior Leadership positions were held by white people, up from 77% in 2020. These numbers unfortunately are in accordance with the anecdotal struggles from reporters in newsrooms across the country who feel much of the work towards inclusivity rests on the shoulders of staffers of color tasked with standing up to upper management.

Major Challenges Remain for Media Companies

While these staffing numbers are just a piece of the overall picture, they give an indication of the significant obstacles still in place that may prevent a truly inclusive workplace and perspective shift at Condé Nast and newsrooms around the country. As we move towards a new era of journalism, the role of apology letters and declarations of inclusivity are still unclear. Idealistic narratives of American progress promote the idea of a society that naturally trends towards justice and equity. However, this narrative perpetuates an optimistic view of slow but consistent progress that can minimize the severity and urgency of current racial inequities.

As Condé Nast’s annual Diversity & Inclusion Report gives us a statistical window to the hiring practices and future diversification within the organization, the parallel issue of content diversity is just as important to study. A recent piece by The Hollywood Reporter highlighted the new wave of POC editors at various publications across the country, discussing the challenges, responses, and ways that they have banded together to promote long term change in their newsrooms. Across the country, journalists at all staff levels have shown a dedication to racial equity in media, inspiring hope about the direction of the profession despite the significant progress left to be made. The racial reckoning of 2020 catalyzed a long-overdue reflection and restructuring process in professional journalism which will set industry standards for years to come. As we move forward, true progress towards racial equity in the media will require pay and benefits transparency, comprehensive shifts in internal policy, a new definition of journalistic objectivity, and diligence at all staff levels.

Upcoming Talk at C+J'22 Conference: News as Data for Activists

Wed, 08 Jun 2022 05:09:00 +0000

I’ll be speaking at the 2022 Computation + Journalism conference, hosted at Columbia University from June 9-11. I’ll be presenting a paper on the software architecture supporting the Data Against Feminicide project.

News as Data for Activists: a case study in feminicide counterdata production

Rahul Bhargava, Harini Suresh, Amelia Lee Doğan, Wonyoung So, Helena Suárez Val, Silvana Fumega, and Catherine D’Ignazio

Paper abstract

News articles are an important source of data for recording and aggregating a range of social phenomena. In this paper, we ask if and how technology can support civil society activists who challenge asymmetrical power relations by producing counterdata—datasets missing from mainstream counting institutions. We consider a case study centered on activists who monitor feminicide, or the lethal outcome of gender-related violence, often using news as a main source to identify and compile databases of incidents. We describe a system that we collaboratively built with activists, aimed at relieving some of the emotional and time-intensive labor this work entails. The system discovers relevant news stories on multiple systems, classifies them based on machine learning models, clusters them into groups of stories about the same incident, and delivers regular email alerts to users. Currently, 26 groups across different geographical regions are using the system, and groups who broadly monitor feminicide report that they are regularly discovering new cases. We also reflect on the short-comings of the pilot system for groups with more specific, intersectional monitoring focuses, and the implications of biased narratives or under-reporting on the system’s design. This case study contributes a grounded example of computational journalism built in collaboration with, and in service of, activists working on critical human rights issues.

Download the conference paper

Slides from our talk

ICA'22 Poster: Politicization and Polarization of Pandemic News Coverage

Thu, 26 May 2022 05:09:00 +0000

I’m excited to join a large team of Northeastern collaborators to present ongoing work at the International Communication Association 2022 Conference. I supported the data acquisiton and analysis pieces of this work by Larissa Doroshenko, Ryan Gallagher, Shreya Singh, and Brook Foucault Welles.

The work compares media coverage of the pandemic at the state level with national coverage. Key findings:

Politicization: overwhelming citation of political leaders compared to health officials and experts, trend strengthens with time
No Political Polarization at the local level: few states cover pandemic similarly, but there’s no clear red-blue division,
Strong Political Polarization present in national news: left-leaning media discuss pandemic with different themes compared to right-leaning media outlets

Associated Paper Abstract

As the United States federal government largely abdicated responsibility for managing the COVID-19 pandemic early in its emergence, mainstream and online conversations about the public health crisis were quickly politicized and polarized across partisan lines. While it is an international crisis, the ramifications of COVID-19 are often local, and local news media have played a critical role in disseminating information as the pandemic evolves. Here, we use a large corpus of local media news stories, structural topic modeling, and clustering analysis to determine to what extent local media followed the queues of national and online media in politicizing and polarizing COVID-19 stories. We find that while there is clear politicization of the pandemic—with politicians being mentioned across topics far more than public health experts—there is a lack of political polarization in local news media outlets. We discuss the implications this has for using local media as a unifying medium in public health communication.

Download the paper

Finding Tweets in Online News

Tue, 23 Nov 2021 05:09:00 +0000

The use of embedded content from Twitter, YouTube, TikTok, and other platforms has grown to become a norm in online news content. When these platforms emerged journalists employed existing industry norms to treat them like traditional sources - content that needed verification, vetting, and review that was often associated with breaking news (Hermida, 2012, Vis 2013). In the time since it’s early years, Twitter has become a major platform used by journalists and politicians alike. This has led to a pivot in research work to instead address Twitter as a platform of authority, where news is being made (Molyneux 2021).

News shapes how readers perceive the world. Social media, specifically Twitter, is used in news media and that plays into that formation of world-views. Understanding the flow of information is vital for interrogating evolving news norms and the emerging media ecosystem that ties news sources, social media platforms, and people together. This is critically important for research domains such as misinformation and hate speech, where social media has been found to carry an outsize impact on controversial perspectives achieving normalization via amplification in traditional news.

Research to understand the evolution and current practice of using tweets in online news relies on large scale automated analysis to discover embedded tweets, and mentions of tweets, in articles. A typical quantitative news analysis project will involve stages such as web-based article collection, removal of non-content HTML, addition of metadata such as date and author, sentence and word-based tokenization, and storage in some system for easy querying and retrieval. Finding embedded tweets, and mentions of tweets, can be considered a specialized content extraction task, one which various studies and datasets have already undertaken (Gruppi 2021).

The official instructions for embedding a tweet

Performing the technical task of locating embedded content accurately is thus critical for making accurate research conclusions about Twitter’s role in online news, and connections to these active research areas. As journalistic digital practices evolve, the technical approaches to embedding tweets have diverged from just following Twitter’s guidelines. After reviewing existing state of the art, we found significant amounts of content was missed. That experience motivated the creation of tweetfinder - a new library to extract embedded tweets and mentions of twitter from online news content. Learn more on the tweetfinder homepage. In this blog post we document the ways in which existing technologies fail to capture the full range of approaches to embedding tweets in online news, review approaches we take in tweetfinder, and showcase initial comparative results.

Functionality

tweetfinder is a Python package that parses HTML content for embedded tweets and mentions of twitter. Users can provide a URL or raw HTML text as input, and it provides convenience methods to return lists of those two types of references it finds. These two types of processing merit separate discussion.

At a high level, any URL passed in is first downloaded via the requests library (with a user-configurable timeout). If the user passes in raw HTML as the input instead of a URL, this step is skipped. The HTML content is then passed through the readability library to remove HTML content that isn’t related to the core article content itself.

Finding Embedded Tweets

Twitter’s official documentation offers a “supported” approach to embedding tweets in web content - namely inserting it via a <blockquote class=”twitter-tweet”> tag. The current library most-often used for finding tweets in webpages, goose simply checks for that approach (source code). We found that not all websites follow these guidelines, and hypothesized that prior work relying on this method of counting embedded tweets could be undercounting. An example can be seen in articles on the Guardian, where class attributes on blockquotes are random-looking strings such as “dcr-i43ppq”. In addition, evolving norms of web-app development have led to some news sites taking an approach of rendering their HTML content via client-side Javascript. This means checking the raw HTML of a webpage outside of a web browser hides any tweets embedded via Javascript commands. To find those, one must run the page source through running a browser. An example can be seen in articles from Fox News, where embedded tweets are only seen after the page is rendered in a browser. These findings informed our approach to iterating on past solutions.

First, we take a multi-pronged heuristic approach to building a pipeline to locate embedded tweets. Our pipeline begins with the HTML (parsed with the BeautifulSoup library), because readability removes tags we need as signals such as blockquotes. We process the HTML tree using the following heuristics to locate any embedded tweets:

Check all blockquote nodes for a class of “twitter-tweet” (the officially supported approach)
Check all blockquote nodes for a child a node linking to twitter.com (because sometimes organizations don’t include the class attribute)
Check all div nodes with a class attribute of “embed-twitter” (a pattern we saw on CNN and other major US news sites)
Check any div nodes with a class attribute of “twitter-tweet-rendered” and extract the tweet from any child iframe node (a pattern we saw on a few Javascript-rendered websites)

Second, we provide examples of how to pre-process a URL through a browser so any Javascript is able to run fully. These utilize the selenium library to render URL content in a headless browser, wait a few seconds for it to fully load, and then extract HTML from the browser before shutting it down. While computationally expensive, this approach does ensure more complete results.

Note that the above heuristic approach could be accurately criticized as brittle; a cat-and-mouse game with online publishers who constantly tweak their content management systems. We anticipate it will need continual iteration to keep up with patterns employed by news producers large and small. Wider adoption of this library and/or approach by the research community would assuage this concern due to many eyes monitoring the evolving norms and patterns.

Finding Mentions of Twitter

A second key functionality for researchers is to find mentions of tweets in news. Mentions of tweets, or paraphrases of Twitter content, are another approach news media takes to integrating social media content into their news stories. Existing work takes the approach of building libraries of phrases and keywords and checking content against them (Rony 2018, Molyneux 2020). tweetfinder integrates prior keyword lists and expands them to allow users to locate these types of mentions of twitter content in news online.

This functionality is built on the content extracted via readability, which we further process to remove all HTML tags. We then process this text-only content to perform a simple find function against every phrase on our keyword list. Note that these lists are user-configurable. We include phrase sets from prior work, and our own list. A clear limitation is that this approach currently only works for English language content; tweetfinder uses cld2 to detect the content language and raises an error if the user attempts to list mentions on non-English language articles.

Results

To characterize and evaluate performance we assessed against three different corpora:

2021 manual: A set of 41 articles we hand-picked and manually reviewed to identify the ids of all embedded tweets
2021 random: A set of 500 random articles from 2021
2020 relevant: A set of 1000 random “relevant” articles from 2020; ones that contained the word “tweet”, “tweeted”, or “twitter”

Each of those corpora were pulled from the Media Cloud database of news stories from national US media sources (Roberts 2021). These datasets are available in the tweetfinder/test/fixtures` folder of our software repository.

Embedded Tweets

In our literature review we found the standard library for extracting embedded tweets was goose; it was either cited directly in each paper referenced here, or we contacted the authors and they indicated that they used it. This sets up a clear comparison for evaluating embedded tweets. As discussed previously, goose analyzes HTML for the Twitter-recommended pattern of embedding via <blockquote> HTML elements classed twitter-tweet.

With the 2021 manual corpus we can do a more detailed evaluation of precision and recall of the two libraries at the tweet level. We manually reviewed each story and annotated them with the ids of the specific tweets they contained.

Library	Tweets Found	Precision	Recall
manual reivew	40	1	1
`tweetfinder`	26	0.925	0.694
`tweetfinder` (with Javascript)	37	0.946	0.972
`goose`	22	0.956	0.611

Table 1: Performance against small manually coded corpus

Unsurprisingly precision is high for all three automated approaches; there are not many elements that look like embedded tweets that are not actually embedded tweets. Recall varies far more. This small corpus shows processing the Javascript is critical for finding a higher percentage of the total embedded tweets (ie. tweetfinder with Javascript). This table omits a “goose (with Javascript)” option because executing the Javascript replaces the blockquote element with an iframe, leaving goose unable to detect embedded tweets at all.

With this assessment of library performance in hand, we can turn towards characterizing the total number of embedded tweets found in each corpus we created. Table 2 compares the performance of each approach on this “in the wild” data.

Library	2021 manual	2021 random	2020 relevant
`tweetfinder`	26	24	601
`tweetfinder` (with Javascript)	37	28	844
`goose`	22	22	469

Table 2: Total embedded tweets found by corpus

In each of these corpora we see tweetfinder outperforming goose, with or without Javascript. The simple heuristics included, while perhaps brittle, are successfully finding closer to the true number of embedded tweets as a reader would see them. It is particularly worth highlighting the 2020 relevant results, which were selected to more likely include such content. On this corpus of stories, tweetfinder finds more finds 28% more embedded tweets than goose, and tweetfinder (with Javascript) finds 80% more. This is a significant and notable difference that impacts any research based on embedded tweets in news articles online.

Mentions of Tweets

In addition to finding embedded tweets, to support research on the prominence and influence of Twitter in news, tweetfinder also supports finding “mentions” of Twitter or tweets. This support is limited to English in our initial release, but could be adapted to support other languages; users can easily create and use phrase lists of their own to extend existing capabilities.

This feature does not lend itself to evaluation in the same manner as finding embedded tweets, because the software is simply finding predetermined strings in text. In lieu of that, we include below some characterization of the quantity of mentions found in each of our corpora with each of the sets of phrases included.

Building on prior published work in the domain of computational journalism, we include three separate lists of phrases in this initial release:

Basic: A short list of basic phrases associated with Twitter generated by the us (examples: “tweeted”, “in a tweet”)
Rony 2018: A list shared by the authors in their paper (examples: “posted on Twitter”, “tweet sent”, “wrong on twitter”)
Molyneux 2020: A short list shared by authors in their paper (examples: “retweet”, “according to a tweet”)

In Table 3 we include a characterization of the behavior of each of the libraries against the three corpora described above.

Corpus	Total mentions	Basic	Rony 2018	Molyneux 2020
2021 Manual	61	53	60	1
2021 Random	37	35	37	1
2020 Relevant	936	768	935	44

Table 3: Mentions of Tweets Found in each corpus by phrase set

One can see that the very short “Basic” list finds a significant portion of the mentions that are included in the far longer Rony 2018 list. This suggests that the majority of journalists use only a small set of phrases when mentioning twitter content in articles.

Conclusion

This type of infrastructure work is critical to help accurately describe how users and events on Twitter are spilling over into more mainstream news coverage. If deployed at scale, this capability to find content more robustly could help identify types of news stories and track their predominance over time. For example consider the “Twitter listicle” type of story - a short piece made up of a dozen or so embedded tweets about the same topic. Are these stories more common on digital native media sources vs. traditional print news? Has their frequency of use changed over time? Or consider how often tweets are mentioned a number of times in a news story. Is that frequency higher in news publications from states whose politicians are more active on Twitter? These types of questions are important to understanding the still emerging interplay between online news and Twitter, a platform heavily favored by journalists and politicians alike. We hope tweetfinder contributes to our ability to understand this.

A Portable Electic Warming Blanket

Wed, 17 Nov 2021 05:09:00 +0000

My brother-in-law Alex has been in a wheelchair since an accident a few years ago. Being tinkerers, we’ve had fun brainstorming assistive technologies that might fill various needs that have come up for him. Early on this involved digging into products that exist already, but more recently we’ve been talking about making our own. Assistive technologies are hard - the needs are hyper-specific to individuals, the environments they need to work in are varied challenging, and the requirements for repeatability and robustness are grueling. All that given, he recently pushed me take on a half-finished project that could transform his experience outdoors in New England winters - an portable electric warming blanket.

The portable warming blanket under construction in my workshop

Some Context

Quadriplegic people often have circulation problems in their extremities (more details). This is a major concern for folks that live in New England and want to be outside in the winter! Improvements in fabrics and battery technologies has helped, with companies selling off-the-shelf heated clothing accessories; we have multiple heated vests in my family that get worn most of the winter. However, these don’t really address the specific needs of this community.

Two years ago Alex worked with a talented team of MIT undergraduates to design and build a custom portable heated blanket for him to use outdoors when he is in his motorized wheelchair. They did some great work researching options, identifying components, building a blanket, and iterating with him… but didn’t quite get it working right in the time they had. Alex brought this under-construction project to my workshop to see if we could finish it up and get it working!

The heating pads are wired inside a 2-layer blanket, closed up with velcro

Getting It Working

The MIT team had a good design for the project overall, but had a few components they hadn’t quite selected right:

The Arduino Uno they had picked as the brain was a bit physically bit for what was required.
The heating elements weren’t all wired the same - some were in pairs while some where wired in quads. This created current irregularities and uneven heating.
The control box didn’t have an easy way for Alex to switch heating levels.
The wiring wasn’t weather proofed for snow and light rain.
The circuit required two batteries - one for the blanket (at 12V) and one for Arduino (at USB 3.3V).

The circuit itself is fairly simple, but it required a lot of wiring

Luckily, I have some experience with heating components - a few years ago I built a pair of heated earmuffs for my wife as a birthday present. After digging around my workshop I laid out a new circuit with scrap parts I had here. I found an old RFduino that could be a smaller brain, a push button that Alex could use, and properly sized wires for the current (among other things). It’s a joy to use up old parts I’ve had lying around just waiting for a project!

The closed up blanket and controller box

Knowing Alex enjoys the cold months outdoors, I added in some weatherproof wires and stuck the controller in a waterproof housing too. Add in some hot-glue to hold it all together and we had a functioning blanket! You can check out the full parts list, circuit schematic, and code on GitHub.

Field Testing

Alex with the blanket, ready for an outing!

With the newly wired blanket working, Alex took it up to Maine for a trip to test it out in the wild. The overall results are promising - it held together, didn’t electrocute anyone, and kept him warm! We need to redo some of the wires and things to make it easier to use and position on his lap, but overall this little project was a success!

Advocating for Food Security with a Data Sculpture

Fri, 01 Oct 2021 05:09:00 +0000

The pandemic has affected so many aspects of our lives, amplifying disparities and challenges that already existed. For far too many households simply having access to enough food to eat is a daily challenge. During the early months of the COVID pandemic, an average of 1,659 new households applied for SNAP benefits every DAY in Massachusetts. My wife Emily and I created a data sculpture to bring attention and action to this problem - titled “1,659”. It was inspired by that number — each of the 1,659 households is represented by one knife, spoon, or fork. The pandemic is slowing here, but these families and others facing food insecurity still struggle to put food on their tables.

This summer and fall our sculpture has been touring various sites across New England - urban farms, city halls, university campuses, and arts galleries. At each stop we’ve engaged with fascinating people and had compelling conversations into the how and why of food insecurity.

Sometimes people come to the table, learn about SNAP, and are curious if they are qualified to enroll in it. At sites like the Revere Farmer’s Market staff is trained to help people get over the paperwork of the application process in multiple languages - they can take action that very day.

Other times we end up talking about the scale of the problem before and during the pandemic. Those numbers are bigger and harder to think about, but spur many to ask about how they can get involved in helping. At sites like Northeastern University, there are groups on-campus working with local organizations that welcome volunteers and donations.

In other situations people wonder about the scale of the problem in their area. In a space like the W Gallery in Wayland, a generally affluent suburb of Boston, visitors are often curious about how the problem touches them and their community. In those settings the need is often hidden, something faced by “others” that somehow don’t count as community members in some eyes.

Across New England, hunger continues to a problem for far too many in our communities. This piece attempts to pull people into that issue with the data, but using the language of the arts in public space. Its beauty provokes, hopefully spurring questions, understanding, and action.

Visit the 1,659 micro-site to learn more about the sculpture, the team behind it, and a schedule of where you can see it. Follow Emily’s work on her Instagram - @studio_b63.

2020 - Year of the Data Sculpture?

Thu, 27 May 2021 05:09:00 +0000

The hype around data continues unabated, with the processes of quantitative analysis seeping into more and more of our lives. From government policies, to business decision making, quantitative data have become central to a growing part of people’s lives. We are presented daily with charts, graphs, maps, infographics, tables, and more. Despite this increase in use of data, our toolbox for interacting with data is far too constrained! Charts and graphs aren’t always the best approach for telling a story or engaging a group of people around data. My work over the last 10 years of running data literacy workshops has been focused on building a better toolbox, one designed around context of use, appropriateness of representational encodings, and respectful of the people and processes data represents. Over the last few years in the broader field I’ve noticed a growing trend pushing in the same direction. People are increasing exploring sound, space, feel, shape, and more while they design data stories! Perhaps it is hyperbole, but I think in reflection last year is shaping up to be the year that physical data visualization in particular exploded as a new norm. Politicians, communities, activists, and artists across the US built data sculptures to tell the year’s most important stories. Read on for some examples that demonstrate why I think 2020 was the year of the data sculpture.

Big and Little Data About the Pandemic

The year of the global COVID-19 pandemic reshaped our world in many ways, including putting health data at the center of daily lives. Updates with charts and numbers were suddenly coming at us every day via tv, social media, and emails. A message about changing a line chart, “flatten the curve”, become a household phrase! These numbers, and the charts representing them, helped guide us as we tried to understand the scope and risk of the pandemic. Note that data visualizations were used by public health experts and conspiracy theorists alike; this isn’t a practice reserved for use by “official” sources. However, people in the public service used data at a pace seldom seen before in their industry. I want to highlight Gov. Cuomo’s use of the covid “mountain” model as one example of data sculptures making their way into the popular vernacular.

Gov. Cuomo in front of Mt. COVID. Watch his short talk-through.

First, let me offer some background. Through early spring of 2020 in the US it was abundantly clear that the federal response to the pandemic was woefully (and willfully) negligent. President Trump’s briefings were becoming more and more crazed, so America by and large checked into another source - Gov. Cuomo’s briefings abut the outbreak in New York State. Many responded well to his fact-based, data-driven updates. By his June briefing, which included the “covid mountain”, New York had turned the corner and was effectively “flattening the curve”.

Let me be clear: I’m not arguing that this is a good data sculpture. He unveiled an object that looks like a museum model of a landscape, probably carved out of foam, covered, and painted. The horizontal axis, left to right, encoded time since the first infection in New York. The vertical axis represented the amount of cases per day. There was a little “42” annotation stuck on top that marked the peak infection rate. Honestly I don’t think the 3rd dimension, coming forward, encoded any data at all… it just makes it look more like a mountain in service of his metaphor. This is a prop used to tell a story about New York locking down and getting the infection rate under control. Let that sink in - one of the top public officials looked too during the spring peak of the pandemic used a data sculpture to explain what success looked like. Gov. Cuomo’s central storytelling device? The narrative metaphor of “climbing a mountain.” I don’t even think that is a good metaphor. The sculpture was met with derision on social media, widely panned for looking like a science fair project.

A few of the COVID-19 memorials that sprung up on town greens across the US during 2020.

On a more human scale, the loss of lives in many communities across the US has been immense. During the summer of 2020 a popular memorial idea sprung up and spread - the idea of setting up empty chairs to represent the numbers of lives lost. From Nashville, to Milwaukee, to Philadelphia, to the White House, these makeshift memorials are physical data sculptures that literally represent the brothers, sisters, mothers, fathers, and more lost to the pandemic. This idea is simple, understandable, and heart-breaking all while respecting that the data represents actual people. It effectively embraces the loss and attempts to provide some spectacle of the collective grieving process.

Taking City Budgets to the Street

The Defund the Police movement in the US emerged as an actionable agenda from the summer of peaceful protests in opposition to structural racism in the US. In particular, the movement argued that local funds were overspent on supporting a violent armed police force rather than being distributed to social services that serve as protective factors against crime and violence. Arguments over the phrase itself sometimes eclipsed the concrete and actionable suggestion that the vast majority of 911 calls could be diverted to non-armed responders. Activists began deep dives into budgets all over the US, comparing funding rates for police against those of social services, arts, libraries, and more. In general the results provided strong evidence that reallocating police funds would improve public safety.

Here’s the problem - budgets are boring. People understand that money matters, and that budgets represent priorities, but oh my goodness in my experience nothing puts people to sleep faster than a set of bar charts showing budget allocations at a community meeting! This movement wasn’t happening in meetings. This movement was happening on the streets, where peaceful protests were met by a wave of violent police response.

A cardboard 3D bar chart showing Chicago’s budget allocations. Source: #ashapoesis on Twitter

Chicago activist Asha Ransby-Spron, along with other young organizers at BYP1000, tackled this head on by created a physical bar chart to show budget allocations out of stacked and painted cardboard boxes. In Asha’s words: “These boxes visualize Chicago’s city budget. The tall stack is the $1.8 billion police budget. Housing & public health each only have one box. “ This bar chart, brought to life with the spectacle of street art, provokes reflection and conversation. You simply can’t ignore it a giant chart made out of cardboard boxes towering over you. More importantly, I’ve written previously about how research has shown that this type of “informal” visualization is more likely to motive questions and conversation than a formal chart. These activists are inviting people into the debate with this type of street chart.

Understanding Climate Risk

Globally, data-driven planning processes are fully baked into strategies at large non-profits to address topics like climate risk and crisis response. This is complex data, covers a complex topic, has very inconsistent completeness, and has loads of uncertainty associated with it - basically the hardest combination of problems to face when visualizing data! My colleague and collaborator Pablo Suarez has been using a physical, arts inspired, approach to bringing people together around this data to drive planning and decision making in support of climate resiliency.

Pablo explaining his Risk & Time data sculpture. Source: World Bank blog

One example is his Risk & Time piece, presented at the 2019 Pacific Catastrophe Risk Assessment and Financing Initiative forum (I know it’s a mouthful, but that’s just how international agencies name things). Yes, technically that piece is from 2019 and this blog post is about 2020 as the year but I hope you’ll give me some leeway here. His data sculpture showcased a timeline of environmental measures, extreme weather events, impacts on people, and economic outcomes. Importantly, Pablo worked with local artists from Vanuatu who used indigenous materials and techniques to make symbolic representations of the events placed within the piece. This evocative piece invited you into a timeline of these climate change fueled disasters, provoking a different sort of conversation about climate resiliency and the international response to these growing risks. The objects and their hand-crafted feel put viewers in a different staet of mind when engaging with the data. This arts approach effectively showcasing considering the material and makeup of data representation.

So What’s Next?

Why the surge in data sculptures? I think we’re collectively realizing that as our society becomes more “datafied” we need to get better at two things. First, data presentations need to meet people where they are in regards to data literacy and visual literacy. Second, using a more creative toolbox for presenting and sharing data invites more people to join into data-driven processes.

These points are especially important when considering questions of access, power, and efficacy:

Access: Who can access data when it is locked in a online spreadsheet? Who can’t access it in that form?
Power: Who gets to decide what data is collected, and who it is collected from and about? Who owns the data?
Efficacy: Who believe they can make change with data? What forms of the data are accepted and validated as legitimate?

You can probably tell where I’m going here… we can increase access, democratize power, and build efficacy by embracing a more creative toolbox of data storytelling techniques like data the sculptures I shared above.

That said, data sculptures can do more if we can move beyond just remaking charts in 3D. Building out of materials that match the message, and supporting more than just the initial 5-second wow factor, are two approaches that I’d love to see more of (a point I’ve expanded on before). There is power in the simplicity of the examples here - people just understand them very quickly. I won’t argue that building a clear and understandable data sculpture is inferior to building an overly clever one. Still, I think there is more opportunity here. A cardboard chart surprises and delights, but when they come over and talk to you about it how can you build on that descriptive technique with the further data you want to show them? Touch and feel are underestimated and underused here. Physical cues and muscle memory work differently than visual perception. My colleague Dietmar Offenhuber has pointed at a study which demonstrates that physical interactions with objects are held in our long term memory in detailed and robust ways. We just engage with physical things differently.

I hope I’ve proven my case, that 2020 was the year data sculptures broke out of scattered use in art and academic projects and into the mainstream. I think their popularity will continue to grow. This should serve as a reminder that our toolbox for data storytelling is far bigger than traditional chart and graphs. Sculptures, murals, sonification, participatory experiences - these are all approaches that meet people where they are and engage them with and around some data. So the next time you’re telling a data story, remember to look beyond the bar chart.

Designing Data Theatre - Learnings From Experts

Fri, 14 May 2021 05:09:00 +0000

Throughout our collaborative creative research process, our team investigated multiple theatrical and data visualization methodologies that can be leveraged in pursuing our recurring central goal: How can we create theatre-based experiences that bring people together around data in social contexts, in order to build empowerment, challenge power structures, and build capacity to speak the new language of data? This blog post summarizes some of what we’ve learned from deeper exploration of the history of theatrical engagement, and interviews with some current practitioners.

Building on History

Source: Bertolt Brecht’s ‘Three Penny Opera’

We were particularly interested in participatory techniques that engage audience members in dynamic ways — from inviting participants into a dialogue with performers, to involving them in the decision-making process of performances. Historically, performance that breaks the metaphorical and infamous “fourth wall” has been used to incite social change, and rouse audience members into action. Although not well known outside of the theatrical realm, the traditional function of drama as a form of passive cathartic “entertainment” is dismissed or rejected by many well-known contemporary playwrights. Famous 20th century playwrights such as Atonin Artaud and Bertolt Brecht, known for “Theatre of Cruelty” and “Epic Theatre” respectively, used different means to shock, overwhelm, or implicate audiences to play a role in the theatre. These playwrights rejected passivity in favor of facing realistic implications of their theatre work. In our work, a lot can be learned from studying the outcomes of participatory theatre and data visualization methodologies. We initially looked in depth at forms such as Theatre of the Oppressed, Playback Theatre, and Verbatim Theatre, and outlined elements that would translate into the world of “Data Theatre”: blurring the traditional delineations between audiences and performers, and welcoming the narratives that emerge from interaction. Our hypothesis is that this can empower community groups to reflect, respond, and react to data that they see as “representative” of their experiences.

In my last blog post outlining the outcomes of our prototype workshops, we summarized some of the key takeaways from our activities and participant experiences. In tandem with these workshops, we interviewed two experts in the fields of playback theatre and information design, hoping to inform our process through a multitude of mediums. These conversations enabled us to continue to explore the parallels of our work with the theatrical methodologies we originally studied, and discover new potential for interdisciplinary collaboration in working towards building and sharing a data theatre “activity guide.” Our first interview was with Christopher Ellinger, Founder and Director of True Story Theater, a nonprofit playback theatre company that offers improvisational performances and workshops to community groups, businesses, and individuals in New England and California. Our second interview was with Frederica Fragapane, an award-winning Italian information designer, whose research on the mutualistic relationship between data visualization and performance inspired much of our initial observations. The opportunity to speak to two dynamic experts in both the practitioner and theoretical realms of our research allowed us to deepen the connections we saw in our investigative stage, and further our usage of interdisciplinary methodologies in designing and testing activities for our community partners. For this reason, our team thought it was imperative to highlight the parallels we drew in the interviews, and further the conversations we started around these methods.

Intentionally Design Your Invitations to the Audience

Source: About True Story Theater

Our first interview with Christopher, the director of True Story Theater, allowed us to return to the conceptualization behind the experience of “shared storytelling,” which influenced the structure of the reflective components of our prototype workshops. Playback theatre allows participants to share their personal experiences in a dialogue with performers, that are then “played back” to them through modalities such as moving sculptures, shorts skits or scenes, group choir, etc. Participants are encouraged to reflect on their experience watching the piece, and to reveal if it changed or informed their perspective on the experience. This deeper understanding of participant experience is designed to build self-awareness and promote empowerment, reconciliation, and decision-making. True Story Theater’s mission of promoting social healing through theatre aligns with our goal of building empowerment and communicative tools through interacting with data, breaking away from the inaccessible “data experiences’’ that often characterize community relations. Christopher shared how True Story grew out of the Improv Theatre Lab, adopting playback theatre due to its “social impact.” He described how his group began diving into oral traditions inherent in the human experience, and how to modernize the practice of “telling a story and being heard in a deep way.” The ensemble needed to be trained to draw out the “bigger social context of the stories so it is not just the individual story.” This remark immediately resonated with our impetus for exploring data theatre — the notion that traditional data representations only present a narrow snapshot of something like food insecurity, and the potential to “rehumanize” or “ground” data in the social context of the real people it represents. Christopher argued that playback differed from Theatre of the Oppressed and other methodologies in that it honors the authentic personal experience of participants, and allows other observers to relate through reflecting on their own social identities. The concept of opening space and bridging the gap to authentic experience provides an important clue in our work engaging with data how can we honor individuals through the lens of a collective story? Data is packaged as a statement on a collective group or population, and finding ways to honor the individual experiences is imperative to our work.

Christopher also shared the structure of a True Story workshop with our team, giving us valuable insight into how to foster space for the ideal playback environment. Their warm up usually features an activity to create a physical connection with the group — which was adapted for both the Zoom and in-person environments. In our workshops, we noticed that the warm-up activities played a large role in setting the tone for engagement with the provided datasets, and the outcomes of each activity. Our conversation with Cristopher cued us into the importance of adapting our warm-up activities for the specific community groups we work with, honoring the contextual space (academic, business, social/religious, etc.) He also shared the value of a warm-up activity that introduces a useful theatrical tool to the group, such as a question prompt or mental exercise. This insight will be a part of our continued conversations in how to translate our work outside of the theatrical environment, and into environments with community partners (which may have limited exposure to participatory theatre toolkits). The interplay between “personal and collective” storytelling may not feel natural at first, but establishing an environment that can assuage fear or hesitancy may encourage participants to accept this invitation.

Furthermore, Christopher shared some challenges commonly faced by playback theatre artists, which may arise as we translate our work outside of the university. Christopher acknowledged that “We expect every topic to meet some emotional resistance,” and that past experiences must be acknowledged and validated in order to help release them. Some of his advice:

Creating emotionally safe spaces can facilitate work with traumatic or difficult topics.
Utilizing small groups to reflect on a less intimidating scale is one way to enable story-sharing, as many individuals fear the spotlight when tapping into vulnerable places
Frequently, the opportunity to resonate and respond to representations of other experiences within a group also helps some participants overcome resistance or hesitation — within limits.
Hearing other painful and traumatic experiences could trigger collective social trauma, and it is important to tread lightly.

Christopher noted, “We are not looking to hit your over the head with traumatic experiences. We are looking to find the right balance of hearing what the reality is of other people’s pain and struggle with the incredible resources within the individual and within the collective groups.” Finally, Christopher emphasized the importance of recurrent check-ins and evaluations with people on multidimensional levels, ranging from small group discussions, big group discussions, and even stand-alone reflection events. In utilizing some of these tools to open the door of data embodiment, we must be mindful in creating space for careful reflection of raw, vulnerable, and often unfamiliar emotional experiences that may arise in the process. Using these techniques to aid in our goal of reconnecting traditional data representations with authentic human experience will help us build empowerment and challenge power structures; however, we must take into account the necessary scaffolding and infrastructure.

Data Viz and Theatre Can Reinforce One-Another

Source: Data visualization and theatre: a story of mutualism by Federica Fragapane

Our second interview, with information designer Federica Fragapane, focused on interdisciplinary context of our work. Learning about the interplay between Federica’s work in the information design realm and in the production of her original plays gave us valuable insights in how a multi-faceted creative toolkit can enhance the experience of audience members/participants. While it may seem that this cross-disciplinary approach is complex and lofty, developing data visualization tools alongside her performances clarified the impact of her work through engaging multiple senses. Federica described her frustrations with the marked difference in perceived value of her originally separate work in information design and theatre, and how this influenced her journey into combining them. While her work in the theatre was one equal side of her career passions, she acknowledged that the arts were not valued or supported in the same way in her community as her work in information design for crime analysis. Federica’s explorations in combining elements of graphic and visual design with theatre were born out of this frustration; she discovered that her work in the data viz and theatre fields influenced each other:

“Telling stories is wider than just numbers.”

Designing visual shapes based off of character relationships, through height and distance, helped her understand her characters on a deeper level. From studying the length of words and sentences as well as developing sensors to detect and project movements behind the actors visually, Federica discovered that the resulting visualizations amplified what was happening on stage. She noted, “I like theatre to be straightforward for the audience,” and “bringing organic shapes to data… can really help in giving them a little more life and showing them as something that is not as abstract as a number.”

Hearing these outcomes of her work resonated with our goal of opening paths of communication of data through theatrical embodiment. Despite data becoming more central to decision making and public understanding, current channels for “speaking with data” are too narrow to engage a broader public. This inaccessibility is perpetuated by an ignorance towards a broad range of engagement with information across different populations. Exploring interdisciplinary modalities aids in creating a clearer, more straightforward communication tool for communities, who may not have had the ability to engage with this data before. Furthermore, Federica highlighted how the aesthetic factor of beauty in her work contributed to her goal of having a heightened emotional impact on audiences through combining data visualization and theatrical modalities. She noted that beauty is inherent in the things she does, and can be a good way to create interest in these topics. The senses are naturally drawn to aesthetic and beautiful things, as they create curiosity and draw attention. This immediately reminded our team of how many participants in our workshops associated the word data with negative connotations — cold, hard, distant, “math” thinking. Exploring these theatrical methodologies suddenly seemed worthwhile, as the arts provide an association of aesthetic beauty that can encourage newfound engagement and interest in datasets. In addition to drawing audiences closer to her work, Federica noted that data literacy is typically overwhelming and not easy to understand. She remarked that especially in our current times, “people will be further surrounded by data in the coming years…it can be dangerous.” Integrating data softly in other parts of their daily lives allows participants to see data visualization as not something that is “too far” from daily life. For Federica, exploring organic shapes created by actors highlights the organic nature of theatre — it cannot be repeated the same way each time. Seeing these shapes come alive helps her audiences develop this perspective in their day to day interactions. In our work, facilitating the embodiment of data will allow participants to engage with information in new ways, and enable individuals to influence and interact with the social and political policy that results. Hearing these shared outcomes and goals in Federica’s work encouraged us to continue to explore theatre for its aesthetic and artistic qualities, which have merit on their own.

There is Resonance Here

The strong connections and parallels drawn in these conversations with experts resonated with both our research and our prototype workshop outcomes. Participants reflected that our activities echoed what they know of the world of theatre:

“It’s like theatre — when you buy into it, you buy into it. There’s no way you can force someone to buy into it”
“Let the audience have freedom to take from it what they are going to take from it”
“You’re going to draw your own conclusions.”

Our activities also exposed how perspectives changed throughout engaging with collective portrayals of the human experience: “What made it so hard was the embodied assumptions I carried with me as a result of my privilege and status in society.” From these conversations with experts, the bonds between the participatory theatre methodologies we studied have strengthened, and we will continue to reflect and test our activities within the context of data embodiment for social empowerment.

Acting is often defined as living truthfully under imaginary circumstances. If one were to commit to this belief without boundaries, then we may view the entire world around us as a script — one in which we are all living our respective truths. Challenging the status quo may only be possible by reading between the lines of the data’s script.

Is it windier, or is it just me?

Tue, 04 May 2021 05:09:00 +0000

Boston is slowly emerging from its pandemic winter of isolation. As evidence mounts on the relative low risk of being outdoors and acquiring COVID, and with spring weather unfurling its warming embrace, people are rediscovering the outdoors. We’re heading outside in droves. I’ve personally been rediscovering, and rejoicing in, the rhythms and sensations of spring. A particularly noticeable one has been the driving spring wind - it has been ferocious and fierce. On my street limbs have been wrenched from trees, plastic patio furniture has been thrown down the sidewalk, and on trash day garbage has gone flying!

I began to wonder if this year so far has been particularly windier than last year. My astute wife pointed out in response that last year this time we were essentially locked down in our house not seeing anyone and barely going outside…. so my perception of the wind last year might be a little off. Undeterred by her common sense, and rejoicing in the bask of a calendar day with few meetings, I set out to investigate the data! Have the last few months in Boston really been windier than last year? Or am I just imagining things? This blog post chronicles my dive into the data, and my conclusion. No spoliers up here - you gotta read to the end!

Getting To Know the Data

Usually the most difficult part of any visualization project is finding and cleaning the data. In this case I knew I needed wind data, but I know nothing about it. To get started I poked around online for sources of wind data. The US has multiple robust infrastructures for measuring and keeping archival records of recent wind speed and direction information. The public gold standard is NOAA’s National Weather Service data. In the realm of private data, Weather Underground has a larger network of private weather stations, owned and maintained by individuals. Looking at their options, it seemed like weather is tracked by cardinal direction, average speed, and top gust speed.

Weather Underground has even got a buying guide with deals for people that want to buy this technology, put it on their property, and giveaway the data for use on the website.

Comparing these two sources, I found that NOAA failed me when I tried to get historical records. Sadly that left me with the task of scraping data from the Weather Underground website. I decided to use data from Logan airport, which seems to be a canonically referred to site to describe “Boston’s weather”. After some work in a Jupyter Notebook with Python I was able to pull in data for the last few months and parse it into a spreadsheet.

The data from Weather Underground has lots of columns, but I really only care about the “wind_speed” and “wind_gust” ones. They’re captured in miles-per-hour.

Investigating a Day’s Wind

I embarked on this dubious little project because last Friday was particularly windy - April 30th 2021. Here’s a visualization it.

A visual depiction the wind on April 30th, 2021. Time is depicted clockwise from the bottom of the chart. Blue bars are the wind reading each hour, with maximum wind gusts shown in tan behind. The longer the bar juts out from the middle, the stronger the wind during that hour. As you can see - we had pretty strong wind all day long!

Comparing the Last 2 Months

Of course I’m curious about the overall trend of wind around here, so I made a picture like that for every day in March and April of 2020 and 2021.

This is a pretty picture, but it is kind of hard to read in aggregate. There are some funny coincidences, like how April 21st was super windy this year and last year. Fun to look at and interrogate, but it doesn’t really help me answer my question… is it windier this year?

Conclusions

Answering data driven questions is hard; you very often end up making assumptions to simplify the reality and context. I’ve already assume the airport is representative of my experience even though I live about 4 miles away. In the data processing I assumed that the data log doesn’t have any gaps in it, and where there were multiple readings in an hour I just used the first one. Now to really answer my question I have to come up with some aggregate way to think about how windy a period of time was. Simple is usually better, so if I just take an average across all my hourly datapoints by year I get:

2021 - 12.19mph average wind speed
2020 - 12.94mph average wind speed

By that metric there is almost no year-over-year difference. Turns out it isn’t windier - it’s just me. I guess I’ll head back outside for some more subjective data collection 😉🌬👏🏾

Source Code

Jupyter Notebook I made that collects the data: on GitHub
Observable Notebook where I visualized the cleaned up data: on Observable

What does Data Theatre Look Like? — Lessons from Our First Workshops

Fri, 16 Apr 2021 05:09:00 +0000

Since the conclusion of the research phase and our last blog post, our team has been iterating on a series of activities revolving around our continued central goal: How can we create theatre-based experiences that bring people together around data in social contexts, in order to build empowerment, challenge power structures, and build capacity to speak the new language of data? In interviews with two experts in the theatre-based methodologies we studied, as well as two prototype workshops, we are moving closer towards bringing our activity design outside of our university community. This blog post is an overview of the activities we are designing, what we have learned from trying them out, and how this has shaped our thinking as we move forward. We’ve found three key takeaways from this work that we plan to dig into more:

Our theatre-based invitations to explore data help participants overcome barriers to “math thinking”
Embodying the data changes how participants reflect on the real people represented by the numbers and quotes.
Activating pathways of muscle-driven memory may change what sticks with participants in the longer term.

What Does a Data Theatre Activity Look Like?

Activity design is no easy task. Creating a playful setting that invites people into a theatrical performance centered around data is our driving challenge. We’ve run two prototype workshops within the Northeastern University theatre community, one virtual and one in person. Both tested out the two activities we have designed.

Each began with a fun activity to establish a playful atmosphere―such as the classic “wave” or a “living machine,” where participants collectively contribute signature sounds and movements in succession. These warm-ups established a safe space for play, which was imperative in disengaging any fear or anxiety triggered by the concept of “data.” Once the warm-up activity was over, we introduced our first data set―a handout with two graphs related to ice cream consumption. This lighthearted topic was our method of scaffolding our first use of traditional data representations (a line graph and a bar graph), in the context of a familiar and comfortable topic. Talking about ice cream encouraged participants to merge their creative inclinations with their “math” brain for the first time in the workshop, drawing upon the playful tone of the warm-up. Each group had about 10 minutes to review and reflect on one of the two graphs― a line graph representing ice cream consumption over the century, and a bar chart representing flavor preferences. Participants were instructed to choreograph the entire group in a movement piece, either representing a trend they saw, or embodying a reaction they had to a data point. This activity was followed by a reflection period, where participants could give feedback on how comfortable/uncomfortable they felt with the activity, or any observations they made in watching others. We also asked participants what they learned or took from this activity, and what benefits were yielded by this new perspective of data embodiment.

The second activity encompassed a more serious set of data that potentially personally affected some participants. We distributed two handouts related to food insecurity―by culture, by location in MA, etc. We gave groups more time to reflect on these handouts, and find something to respond to. Instead of having each team choreograph the group, participants created their own short performances related to a data point of any size from the handouts. We left the activity open-ended, allowing participants to choose anything from either handout and incorporate any number of elements―dialogue, movement, props, etc. The reflection piece for this activity focused on the transition to more serious data points, and how participants felt embodying something that is so deeply personal to many in the community.

Outcomes of our Prototype workshops

We’ve learned a ton from these two workshops, and have already begun to iterate on our activity designs. Even though both workshops were within the theatre community at Northeastern University, one was in-person and one was virtual. In the Zoom workshop, participants jumped into the warm-up right away, taking full advantage of their personal space. This high-energy atmosphere carried into the first activity―where the responses to the ice cream data prompted a wide variety of creative interpretations and responses. By using the “host view” feature on Zoom, participants were able to choreograph the team virtually in various sculptures, and play around with the geography of Zoom boxes and positioning. However, when we arrived at the food insecurity data, the Zoom participants seemed to pull away, not willing to make the jump into more solemn subject matter. They were cautious in questioning the assumptions made by the data surrounding cultural and geographical trends of food insecurity, and lost the experimental spark that dominated during the ice cream activity. Unlike the first activity, participants were assigned a specific graph or chart to look at―and “skated across the surface” in dealing with these difficult topics. The elements of props and movement were more sparse―many stayed put in front of their cameras when it was their chance to perform.

This contrasted with the second, in-person, workshop, which took place within an acting class. The warm-up in this scenario felt less impromptu and more ritualistic, and the intense focus of the participants alluded to the academic backdrop in which we worked. Accordingly, the first ice cream activity yielded polar opposite results- students were given free rein to choose what data point(s) they wanted to choreograph, and every group narrowed in on the same graph. Many of the presentations felt repetitive in nature, and the playfulness from the first workshop was transformed into something presentational. The second activity also yielded opposite results―when handed a more difficult, somber subject matter―and given free rein to choose―participants developed mini skits, and used the space in a dynamic way. We found it fascinating that two workshops utilizing the same subject matter, within the same community of people, yielded such different results in these two settings.

Overcoming Barriers to “Math” Thinking

Our theatre-based invitations to explore data helped participants bridge between traditional delineations of thought — math vs creative, left vs, right brain, etc. In the second workshop, we asked in the warm-up what comes to mind when hearing the word “data” for the first time. Responses included “information”, “numbers”, “technology”, “cold”, “hard”, “analytical”, and more. These traditional associations are rooted in the way we are educated — we are too often taught that you are either a “math” person or an “art” person. Traditional two-dimensional representations of data reflect these fundamental distinctions. In math, we are taught there is a right and wrong. Yet in the human experience, nothing is ever truly right or wrong or black and white. In its traditional formats, data is accepted as cold, hard, proof, and its certainty and authority is rarely questioned. Yet many times, data in the realm of topics such as food insecurity often only shows a narrow perspective, which could potentially perpetuate assumptions about certain groups and communities. We found that people were much more willing to interrogate and question data when asked to physically embody it, in contrast with looking strictly at two-dimensional representations.

The experience of living the data from this angle was new to them, but participants showed enthusiasm in their first encounters with blurring these traditionally steadfast barriers. One participant noted, “You are translating your math mind into a creative mind, into a story mind, into a people mind.” This process of translation may not be comfortable or easy, but it is very worthwhile. Another participant echoed that “(We were) looking at a piece of data and the numbers and quantitative stuff, but what’s the story here. I thought that was really fun.” And while one participant noted that “I was really self-conscious about being critical of the data set,” the group agreed with the sentiment that “I think that is part of finding the stories, you have to translate.” The benefits of moving away from traditional boundaries of “math” and “creative” thinking allowed the data sets to come alive in new ways, and leave participants with a broader understanding of the numbers and of themselves.

Rehumanization ― Representing Real People

In working with data such as food insecurity, which affects many college students on a personal level, it is easy to generalize experiences when looking at two-dimensional graphs and charts. Yet the numbers and statistics surrounding food insecurity aren’t just numbers― they represent real people, with real experiences and struggles. We are finding that theatrical embodiments of these data points rehumanize the data, by allowing participants to consider their own experiences and knowledge of food insecurity, and exposing implicit biases present in the narrow snapshot of a graph or chart. Consider the way a photo may show a snapshot of an event, such as a day at the beach, but does not expose the whole experience― the sweltering heat, the crying baby, the seagulls diving for your food. Participants reflected that the second activity “Grounded it in the actual people.” and “got me thinking about the people behind the data, behind the numbers.” They were “telling the story of the people behind the data.” By asking participants to think about this data in new ways, we are essentially teaching around the data — forcing them to expand their perspective and see the bigger picture. Rehumanizing the stories behind the data and empathizing with the human experience is much more impactful than associating lived experiences with only numbers. Participants acknowledged that “Statistics and data sometimes generalize experiences and are not telling the whole story just by looking at the numbers.” and “Using reflection and performance shows the story underneath it that the data attempts to bury.” In working with community groups that are directly reflected in these data sets, we would also invite them to take control over their stories.

“Datasets represent a scientific, one-dimensional perspective of an issue. This allows communities to redefine what is ‘a hard tangible fact’ with emotions.”

Muscle Memory vs Mental Memory

Another topic that emerged in conversation was the value of different capacities of memory within our nature, which have typically been compartmentalized in their functionality. While mental memory is associated with academics and the classroom, reading text or preparing for exams, muscle memory is reserved for sports or the arts, where techniques develop with significant training, repetition, and patience. When inviting participants to overcome barriers to traditional “math” and “art” brain distinctions, we also encouraged the mix of two different forms of retention― each with significantly different outcomes.

Participants noted that muscle memory is more visceral, involving more of your body than just your head. Data in these activities is experienced not only in mental assumptions and associations, but in movement, distance, and sound. The experience of muscle memory allows a deeper and more holistic experience of data in many parts of your body, as would the impact of kicking a soccer ball in an emotional game-winning goal. However, the downside of muscle memory is that it is fleeting―it can be ingrained with repetition, but the experience is never the same thing twice in a row. In essence, it is ephemeral―and your perspective on the experiences changes over time.

Mental memory is more permanent in nature―the things you learn are cataloged in textbooks, graphs, and charts. Relearning information won’t change the factual elements― it is often represented on paper, forever memorialized and unchanging. Yet this longer-lasting form of retention involves only the head, and doesn’t reflect a deeper physical commitment to the experience of knowledge. One participant made an analogy of looking through the window of the house, vs, being inside. Your perspective on information is much different when you are not physically interacting inside this house. We challenged participants to consider: is there a world where we use both? Where one informs the other, and interacts in a constant dialogue? Often we look at charts or graphs, or visual representations of things changing over time in relation to one another, and physicality may make it resonate better. Yet the subconscious part of the mind (frontal lobes) was not as engaged during physical movement as when analyzing numbers on a page. It is worthwhile to continue to explore crossing these boundaries, or considering them fluid. The untapped potential of combining mental and muscle memory is evident, and is beginning to emerge from this work.

What’s Next

After debriefing on each of these workshops, we are in conversations about how these activities would translate outside the theatre community, and how we can bridge the gap between data and our theatrical toolkit in a more gradual manner. We discussed incorporating more “scaffolding” between the two activities, whether it be a longer period of reflection between each, or incorporating a group discussion between the time the participants review the data and when they are tasked with choreographing or creating responsive pieces. We also are looking for new ways to set the playful warm-up tone intentionally, such as a physical or mental challenge that requires collaboration. Incorporating more structure in terms of assigning data points or charts to groups of participants, and relaying potential theatrical elements that may be useful, are other options we considered in moving this work to a wider audience. Our next big step is to work with community groups once the burden of COVID implications is lessened. Until then, I will be writing another blog post to summarize our findings from our expert interviews, and considering methods to implement this work in communities that are directly affected by the data sets we are utilizing. We would love to collaborate with anyone who sees the value of this work, and has ideas or insights into how to bridge the gap in our community work. If you are interested in collaborating or have any questions or ideas, please drop us a line!

Panel talk on Data Against Feminism Project 4/9/2021

Thu, 25 Mar 2021 05:09:00 +0000

I’ll speaking on April 9th with on the Angeles Martinez Cuba (of MIT’s Data + Feminism Lab) on the “Good Data / Data for [Public] Good” Panel at the NULab’s Annual Spring Conference. We’ll be discussing some of the under-construction technology co-design work we are doing with global advocates compiling databases of gender-related killings.

Register now.

How and Why We Sketch When Visualizing Data

Thu, 11 Mar 2021 05:09:00 +0000

If you’re like us, at some point in your early education you decided you couldn’t draw. Your doodles, like ours, didn’t look like you wanted them to. For many, this disappointment can persist into adult life. As researchers into how people learn data visualization, we’re here to tell you that it’s OK — stick figures are fine! You can learn to sketch your data stories; in fact, you’ll see that research tells us that sketching is critical for working in teams and for breaking through “visualizers’ block.”

Learning to go from data to story is hard. You often have to wear many hats — statistician, analyst, domain expert, graphic designer, and more. One practice used consistently by data visualization experts is “sketching.” Most folks who practice data visualization have notebooks full of ideas for symbols, encodings, and narrative flows. Previous discussion in the DVS Slack channel has addressed this. But what makes sketching such an effective method? What can people getting started with data visualization learn from this practice?

The initial sketch ✏️, and the outcome 💻
Would love to see others as well! #sketchvsoutcome
Put yours in reply 👊🏼🤗#dataviz #datavisualisation #illustration #GraphicDesign #illustrator #sketch pic.twitter.com/4z2IKlG28s
— Sonja Kuijpers | STUDIO TERP (@SonjaKuijpers) January 18, 2021

Sketching helps us integrate different kinds of knowledge

Why do professional dataviz experts sketch? This is an important question to ask if you want to help learners develop this practice. Luckily, academics have been digging into this for a while — we’ve tried to translate some of what they’ve found into understandable motivations.

For one, the creation of external representations aids the learning process in many different ways. There have been studies on the role of externalization in learning, and others on the application of drawing to reason and learn within science education. These studies find that:

drawing an idea makes explicit the metaphors and shorthands you’re using to think about the idea;
a shared representation leads to more productive conversation and ideation in group contexts;
physical representations allow you to see characteristics and relationships of concepts more easily.

We sketch for lots of reasons, but these studies highlight some of the key motivations that might inform your own approach to sketching for data visualization. Think about why you want to sketch: what do you hope to learn from the process that you didn’t already know?

From Tyler et al. 2020: sketching to learn applies to more than just data science.

The idea of integrating visual, verbal, and mental modes is further explored in Joanna Kedra’s thoughts on visual literacy. Kedra argues for the importance of visual literacy skills in an increasingly “multimodal” environment, and lays out visual literacy skills as being on par with reading literacy and numeracy. We evaluate and interpret images, “translating” between visual and verbal modes of communication, so sketching helps us try out images that communicate our ideas. Makes sense! Of course data literacy and visual literacy are not synonymous — data does not necessarily take a visual form — but, when we’re learning to sketch data visualizations, they are definitely interrelated.

From Kedra 2018: visual literacy isn’t just one skill, but a host of interrelated ones.

These academic research projects establish why sketching is so important. Our brains are constantly translating the visual and the verbal, so externalizing this process helps us communicate and process more effectively.

Sketching to overcome “visualizers’ block”

People new to visualization often fall back on using data representations they are familiar with — bar charts, pie charts, etc .— without considering alternatives that could more effectively, or more appropriately, show the data. When you’re sketching you’re exploring a broader design space of your data visualization, having a visual conversion with yourself (and others) about how to represent the data effectively for your context and goals. Grammel’s work on how novices create visualizations shows us how this default to known charts particularly happens when learners have difficulty figuring out how to answer questions they have about statistical relationships with which they are unfamiliar. Similarly, Walny’s work on how learners sketch shows us that learners default to numerical representations over more abstract or symbolic ones.

From Walny et al. 2015: a selection of dot plots and matrices produced by participants, making up one of the most populated categories of visualization in the study.

Our takeaway? Drawing can be daunting, but sketches are useful even when the output doesn’t look “professional.” We learn more when we experiment and try new things, and you can surprise yourself with what you see when you try to put your thoughts and ideas down on paper. If you’re stuck on something, even if you feel unimpressed by your drawing skills, try sketching your way out!

From Grammel et al. 2010: we tend to default to familiar forms when we’re confused, but there’s much more to data visualization than just bars and lines.

When trying to design a data visualization, you can get stuck in lots of ways. Bressa’s work with data visualization novices, workers at a food bank, reveals some more insights from which to learn. This group struggled to move from text to visual descriptions. They also had difficulty managing the spatial aspects of their visualizations. Sketching helps with both of these challenges! Targeted approaches to visual brainstorming can address these barriers more directly, for instance if you get stuck on how to visually represent a concept or word you can try an exercise like creating a visual “word web.”

Building and bolstering data literacy

Why do we care? Our Data Culture Project is a free, lightweight, self-service curriculum of playful activities designed to help people build “data cultures” in their organizations. One activity in this toolkit, Sketch a Story, focuses on helping all kinds of people learn how to sketch — challenging them to quickly draw visual “data stories” based on word frequency data in political speeches and song lyrics. This is an exercise in creating a narrative from data: whatever factoids or relationships stick out to them in the frequency data are transformed into a visual representation, with participants working together in small groups with big paper, markers, and crayons (or when circumstances require, using online tools like Miro).

This screenshot from our WordCounter tool shows the frequency of different words in Bob Dylan’s lyrics.

This sketch from one of our workshops shows a visualization of those lyrics.

We’ve run this activity dozens of times, leaving us with a corpus of hundreds of sketches created by diverse groups of data literacy learners — higher-ed students, staff at non-profits big and small, government employees, journalists in newsrooms, and more. Feedback from this activity suggest it has a positive impact: more than half of participants in a testing workshop said it made them feel more comfortable with analyzing text, and a majority also found it both useful and easy.

More examples of sketches from our workshops — showing many different ways to visualize.

A preliminary review of these sketches has shown some interesting patterns. For example, color (as opposed to other variables like size, shape, font, or position) seems to be the most common method used to differentiate different elements and categories in a sketch. And the majority of stories being told are comparisons: showing the difference between two parts of the dataset, or identifying an outlier within an overall trend. As we move forward with our exploration, we hope to discover more patterns, and add to what we know about the process of learning to sketch and work with data to tell stories.

Remember: stick figures are OK! The point of sketching isn’t to produce a beautiful work of art, but to get your thoughts and ideas into a format that leads you to new insights and ways of thinking. Importantly, sketching allows you to flex your creative muscles without any constraint: with your paper and pencil, you can do whatever you want. While more constrained tools like software and programs for creating data visualization undoubtedly have their place in the process, they don’t allow for the kind of unfettered exploration you get from just sitting down and sketching. Hopefully, this post helps you understand a little more about what research has to say on how and why we sketch to help us visualize. We look forward to learning even more in the course of our own research, and sharing these findings about the nature of sketching to help you with your own learning process.

Originally published on Nightingale

Talk - Get off the screen!!! 4/1/2021

Wed, 10 Mar 2021 05:09:00 +0000

I’ll speaking on April 1st as part of the ART/DATA/HEALTH Seminar series, which focuses on communicating public health data creatively during the pandemic (hosted by the University of Brighton). Register now.

Why is data so often trapped on our 2-dimensional screens? Why do we default to simple charts to show it? Our toolkit for bringing people together around data in civic health settings is far too limited. Rahul will discuss alternative approaches that build on the language of the arts, meeting people where they are, in formats they understand. Participatory theatre, community murals, public art installations - these techniques build a more appropriate toolbox for communicating with health data. Rahul will share examples from his work, and the underlying pedagogy behind it. You’ll walk away with a rich set of inspirations and concrete approaches you can integrate into you arts, health, and data practices.

From Listening Device to Helpful Partner

Tue, 23 Feb 2021 05:09:00 +0000

Have a Google Home? Try asking it “can I trust you?”. The response isn’t particularly comforting - a corporate statement about how it’s designed to keep your information secure and a link shared to your phone so you can review the data it has. I followed that link, and found my two most recent saved commands were recorded by accident and contained parts of conversations I had with others!

When the Google assistant thinks it hears its name, it comes to life and listens, regardless of whether or not you actually summoned this device. Now that we can peer into this data collection process and see its flaws, it is up to us to decide if we can trust it. This experience prompted me to think about how we relate to the digital devices around us. How can that help us rethink the particular forms of listening devices like Google Home? Continuing an earlier thread of rethinking the design of these devices, this blog post explores what we know about how people interact with the present, and future, of digital assistants.

Source: The Logical Indian

Evolution has given humans some pretty useful tools to deduce emotion and intent behind human actions, including being able to tell if someone is listening to us or not. The cues that one gives-off when actively listening to another person are completely absent in the design of these modern listening devices, and a little light doesn’t properly fill that void. Sherry Turkle writes in “In Good Company” that a robot imitating human gestures and movements pushes our “Darwinian buttons” and allows for the robot to “exhibit the kinds of behavior people associate with sentience, intentions, and emotions”. Receiving these social cues that we subconsciously crave actually improves the functionality of these listening devices. When the human brain sees a device exhibit some signs of social competency, it switches to a mode of conversation that is more two-sided and doesn’t feel like you’re asking a question to a blob on your counter. This improvement could lead to more efficient ways of communicating with your device and can even cause some people to use it more often.

There are other non-visual ways that the conversations we have with these devices can be improved. Researchers in the field of Affective Computing are learning and implementing new ways people can communicate with robots and services to make them more efficient and accessible. With affective computing, devices can interpret human emotion and more effectively respond to human requests and imitate natural, comfortable conversation. In addition to making these interactions more efficient, they also allow people with social disorders who would otherwise be unable to interact with these devices to have an effective dialogue with these devices. On the flip side, the addition of more sensors to computational devices opens up an even wider world of data collection and surveillance. Last year Amazon announced their “Halo” wrist band, which among other things can eavesdrop on the tone of your voice to guess if you are feeling happy or sad.

Conversational Technology

Without proper feedback during conversation, there is a disconnect in our brains and we have a harder time seeing these listening devices in the role they are trying to play- social robots. As AI companies try to inch closer to being efficient enough to actually show intelligence, being social is seen as a solution that allows for devices to mimic intelligence without actually being emotionally intelligent. A device that is able to properly pantomime interaction could be more effective at its job since the human brain’s most effective method of communication is audible conversation with another human.

In “Anthropomorphism and the Social Robot”, Brian R Duffy argues that the “ultimate goal of the human–computer interface should be for the interface to ‘disappear’”. This assertion builds on a larger set of beliefs the main creators of these interfaces share - that computation should fade away and simply be part of our surroundings. Groups whose members are at risk from expanding surveillance architectures vocally argue this point.

Source: “Anthropomorphism and the Social Robot”

While we now interface with listening devices without touching anything, speaking to one isn’t exactly the seamless interaction it could be. You can’t always ask these devices questions the same way you would ask a friend, these devices don’t understand context and can have a hard time with follow-up questions, and you may even find yourself talking in a different, more robotic voice while using one. When I talk to my Google Home, I often feel like I’m carefully stringing together the right words to get a relevant answer. It feels stunted and awkward. Making the user feel as if they are having a conversation with the device instead is how we turn these listening devices into social robots. Our hypothesis is that creating more an embodied device, which cues the intent to have a conversation with a human, is a different path forward than simply having the devices “disappear.”

Embodiment

If you’ve ever sent a sarcastic text or email to someone and it doesn’t translate properly, you know how it’s difficult to express certain emotions and inflections without the other party being able to see you. Humans also have the ability to anticipate actions and words while in conversation. This allows us to begin formulating responses before the other party finishes and we can also notice when someone else is about to start talking or take a pause. Being able to sense and anticipate the next thing in a conversation allows everything to seamlessly flow together and a conversation becomes a smooth back-and-forth, not a jagged request then response. These skills are essential, and many of those without them suffer throughout their daily lives.

Listening devices don’t have these additional perceptual affordances. They are unable to notice the nuance of human speech or observe our body language. In the future this may be possible and we’d guess that would help smooth these interactions. Physically embodying the equivalent of a nod, or an “uh huh”, are approaches to explore more.

Continued Exploration

This research and background into perception and human-technology relationships informs our work rethinking the design of listening devices. Our goal is to create a device that not only reveals that it is listening to you through anthropometric means, but also stimulates a smoother conversation through feedback. We hope this work eventually delves into questions of perceptual capabilities, but for now it is fun to consider the possible future design of a device that acknowledges and embraces its role as an omnipresent social robot more fully.

Musical Mosaics: a computational design prototype

Mon, 01 Feb 2021 05:09:00 +0000

Computational generative design is a fascinating space available to those with access to coding skills and prototyping machines such as laser cutters and 3d printers. But what happens at the intersections of simple computational design and traditional craft disciplines? We were interested to explore this, starting with a series of mosaic pieces based on data extracted from songs.

Using simple software I created a tool that would translate the audio volume of song over time into a radial design to inspire the creation of a mosaic mirror frame. The height of each bar indicates the volume of the song at that point in time (mapped radially from top around the circle). A simple mapping, but enough to evoke reflection about the composition and feelings the song evokes as a listener.

A screenshot of the web-based design tool being used for Joni Mitchell’s “Blue”

Mosaic made based on the data from Joni Mitchell’s “Blue”

The computer-generated designs inspired Emily’s hand-crafted design. Building on top of frames created on a ShopBot via CAD design tools, she laid out the mosaic with pieces of glass. This traditional form dates back thousands of years but is still a vibrant global practice.

Three different song-based mosaics under construction - “Red Red Wine”, “Somewhere Over the Rainbow”, and “It’s Not Easy Being Green”

This is work in progress. The aesthetic of these pieces lends itself to appreciation in its basic form as a mirror, but with the awareness of the data visualization embedded within it prompts reflection about the song and introspection about the viewers’ experience of that song and their relationship to it. This first simple data-based mapping to inspire mosaic design is intriguing enough to spark further discussions of how we can reflect music in mosaics, perhaps based on more complicated algorithmic analysis.

The code is available on GitHub

Talk - The Physical Life of Data

Sun, 24 Jan 2021 05:09:00 +0000

I’m excited to be speaking with my collaborator Laura Perovich about our Embodying Information project at the Northeastern University Center for Design’s panel on “The Physical Life of Data”. This panel talk is on Thursday Jan 28,2021 at 12pm (Boston time).

~~Register to join the event!~~

Update: video recording of the event

Swiped: How dating apps harm marginalized communities

Wed, 13 Jan 2021 05:10:00 +0000

We started out investigating media coverage of LGBTQ+-specific data breach issues, yet race and racism are unavoidable. Potential harms from online dating apps are constant, but our attention to these issues should extend far beyond the pandemic.

Image by amrothman from Pixabay

In April 2020, an Instagram influencer encouraged tens of thousands of followers to download queer dating apps to strategically out friends and family in Morocco who may be LGBTQ+. Followers used Facebook groups to then circulate photos of gay men to millions, putting them in danger of threats, evictions, arrest or assault, reported NBC News.

Homosexuality is illegal in at least 70 countries — including Morocco — which is why apps like Grindr have had to implement discrete icons in countries where being LGBTQ+ is dangerous. This danger is not merely international. In the United States and countries where homosexuality is legal, many in the LGBTQ+ community still face risks of harassment, physical harm, discrimination, and employment inequity. Dating apps play a critical role for many in the LGBTQ+ community — especially those who are looking for connection or for members of the LGBTQ+ community who have yet to come out to their network.

The COVID-19 pandemic exacerbates structural inequities and adverse harms to marginalized communities. At the same time, more people are heading online to seek companionship, further driving the online dating trend. This is an important moment to consider the potential dangers of data leakage and harmful dating app design. First-time FaceTime dates and new competitor platforms like “Quarandate” meant companies like Match were pressured to quickly incorporate new features to remain relevant. Within months of pandemic-driven physical distancing guidelines, dating app companies like Match Group and Bumble saw increases in messages exchanged and built-in video call tools to support virtual dating.

Such features can facilitate an individual’s ability to form personal connections in a difficult time. However, it is worth keeping in mind that online dating carries harms, not just for the LGBTQ+ community, but also for communities of color. “Online discrimination has reached epidemic proportions affecting not only Grindr but other social networks,” explained Grindr in a press release. At the height of the Black Lives Matter movement last summer, media articles highlighted how dating apps can exacerbate racism through “ethnicity filters.” In particular, a study from the University of Illinois and the University of Michigan “confirmed that racism on queer dating apps [like Grindr] can have significant negative health impacts on men of color, including depression and feelings of lower self-worth.” While these may not be data breach-related harms like those encountered by the LGBTQ+ community, they still highlight how these apps can intensify racist currents.

As online dating companies implement more data-collection features and users increasingly rely on dating apps, users encounter more types of data aggregation, creating opportunities for unwelcome third-party sharing or rich targets for data breaches and discrimination. Certain app features like “ethnicity filters” can propel racism in our communities, but these harms aren’t unique to pandemic times.

A review of media coverage and research on dating apps and harms to marginalized communities

To further explore this topic, we conducted a media analysis to help us understand what issues related to dating apps are driving current conversations. Using the open source Media Cloud platform, we identified nearly 21,000 stories about online dating in U.S.-based media sources published between Jan. 2019 and July 2020. We then analyzed these stories to understand patterns in coverage, including the broad trends that drive coverage of dating apps, the apps that get the most media attention, and the ways in which race, ethnicity and LGBTQ+ discourse about online dating are characterized. Ultimately, the project serves to highlight how a steady stream of media coverage on online dating may help focus public attention on the dating app harms faced by marginalized communities.

Overall, Tinder and Bumble drove popular media coverage of dating apps in the United States. Grindr emerged as the most popular LGBTQ+-focused app in news coverage. For all apps, coverage ranged from positive to negative headlines depending on the story. Over a year and a half, the news stories we analyzed were largely driven by three key themes: Valentine’s Day, potential regulation in the online dating space, and COVID-19 (as people began to turn to online dating as a way to date and maintain physical distance).

In general, data breaches and privacy concerns are now a way of life for people the world over. The concern is not specific to online dating apps. That said, dating apps ask for personal information to facilitate personal connections. We began our project by exploring media coverage of LGBTQ+-specific data breach issues, because privacy-related online dating concerns particularly important to members of the LGBTQ+ community. Yet, the more we found, the more we realized that privacy and security issues on dating apps are inextricably linked to issues of race and racism. Harms that any given community may face through its use of dating apps are unique to the social identity of that community. In our examination of dating app coverage about race and the LGBTQ+ community, we found that media coverage of race centers on the design of dating app features — such as Grindr’s race filter — and that coverage of LGBTQ+ communities focuses on data breaches.

COVID-19 has forced people to be increasingly reliant on online apps for social interaction. For those people seeking relationships at this time, their choices for meeting new people are often constrained to virtual spaces. “What has changed with the pandemic is that it’s more difficult to opt out,” Ailo Krogh Ravna, policy advisor and lead researcher on a Jan. 2020 study of online dating for the Norweigan Consumer Council (NCC) said to us. “People are more likely to accept data tracking in a crisis, putting information like your sexual identity at risk of being exploited.”

Third party data sharing is rampant in data-collecting apps. This isn’t an issue specific to online dating apps, but is increasingly important to highlight with apps that collect information such as sensitive conversations and dating preferences. Roughly six-in-ten online daters in the U.S. are concerned about data collection, the Pew Research Center reported. The Jan. 2020 NCC report highlighted egregiously complex privacy policies that made it impossible for users to give informed consent to excessive tracking and data sharing with potentially hundreds of third party organizations. The NCC report also identified 10 different data-collecting apps (including Tinder, OkCupid, Happn, and Grindr), which together shared data with at least 135 unique third-parties. Some of these third-parties were involved in advertising and/or behavioral profiling. They also reserve the right to share information with other companies. This means consumers cannot give informed consent to the excessive tracking, sharing, and profiling that pervades the advertising industry. Apple has very recently taken some steps to increase transparency of these privacy practices with their new “App Store Privacy Labels,” much to the displeasure of larger data-collection practitioners.

Dating apps can be used to fuel LGBTQ+ hate crimes. Malicious use of dating apps like Grindr may lead to specific LGBTQ+ hate crimes, putting people in physical danger. A July report released by Recorded Future highlighted a range of cyber threats and security risks associated with popular queer dating apps like Grindr, Her, and Scruff. The findings suggest that most apps are not doing enough to protect users.

Companies are starting to worry about racism in their dating apps. Following the outcry last summer during the Black Lives Matter movement, Grindr said they would delete their ethnicity filter, but Grindr users have reported that racism is still a problem on the app. In our research, we observed news coverage of online dating using the term sexual racism, “a specific form of racial prejudice enacted in the context of sex or romance.” This practice can “reinforce racial divisions and biases,” explained Cornell researchers. Furthermore, dating apps like Promatch, Aisle, and TrulyMadly “to a degree rely on LinkedIn profiles in their algorithms,” Glamour reports, suggesting yet another avenue for race-based concerns to play out. Research has also shown that job opportunities and race are inextricably linked.

Image by Andrej Lišakov from Unsplash

We found a steady stream of stories about app-related harms that does not ebb and flow; rather, the consistency of such harm-related coverage over time suggests that these harms may be a constant issue that only rises to national and international prominence during high-visibility cases like the example in Morocco. Looking at media coverage of Grindr alone in 2020, we found stories about harassment and stalking, unique security risks, inadequate data protections, and racist comments during the COVID-19 pandemic. Grindr, just like any other dating app, is a technology that can be used to develop personal connections in a time of social isolation as much as it may be used to exploit personal information.

“LGBTQ+ people are not any less vigilant than the rest of the population using social media or browsing the internet; it’s that they are just more vulnerable to malicious attacks for the simple fact that they exist,” Gary Goldman, Senior Program Director of Out in Tech, told us. Kelvin LaGarde of Columbus, Ohio who experienced racial discrimination on Grindr and other dating apps shared to NBC News that such change can only be effected through larger cultural changes. “The serious work that needs to be done has to be taken on by the various different communities across the nation,” LaGarde said.

Of course, there have been efforts to chip away at this systemic challenge. “We often learn about these things when it’s too late,” Goldman told us. “Resources like Safe Queers, an initiative of Out in Tech and African Queer Youth Initiative can provide open-source tools to help LGBTQ+ members communicate safely and anonymously on dating apps and to encrypt their information — especially as more people rely on these apps for social interaction.” In parallel, the law should not promote the development of dating apps that are unsafe by design, a core tenet explained in the New York Times by Ari Ezra Waldman, professor of law at New York Law School. For example, Scruff, a dating app for LGBTQ+ members, created features to obscure location, allow users to be on stealth mode, and requires iOS touch ID when launching the app. It even has a global support team to respond to potential harms within 24 hours.

This report is intended to further conversations about dating apps, their potential harms and benefits, and our understanding of their role in society. News coverage has amplified the more egregious harms faced by users of these apps. This research reminds us of those communities who face the most risk from data-collection technologies and that we need to do much more to protect them. While we continue to further rely on apps to keep us connected, emphasizing the protection of the digital lives and personal information of marginalized communities should far exceed the length of the pandemic.

Download the report.

Rethinking Listening Devices

Fri, 18 Dec 2020 05:09:00 +0000

A recurring truth of the future of technology is that it sneaks up on us unawares. Technology is adopted before we’ve all had time to consider its consequences, the potential harms that might come. This is certainly true of the rapid spread and normalization of digital voice assistants in our homes. This blog post covers a little background on that particular technology, why we choose to call them “listening devices”, and introduces a small project we are launching to interrogate our relationships with these devices as they move into our homes.

A “Normal” Technology

These listening devices are everywhere. If you don’t have one in your home, the odds are good that you’ve been to another home that does. Home “smart speakers” have become surprisingly ubiquitous. In January of 2020, it is predicted that there is almost 90 million of these listening devices in people’s homes in America alone. The majority of these units are bought for homes that did not previously have a listening device, which has brought the user base up to 34.4% of American adults. This rise in popularity is frequently compared to the rise of smart thermostats, and this comparison can actually help us understand exactly what these listening devices are doing in our homes.

Google bought the smart thermostat company Nest in early 2014 for a whopping 3.2 billion dollars. For a while, it wasn’t clear why a search engine company would purchase a home appliance company, but as the years have gone-by it has made more sense. Google isn’t just a search engine, it’s one of the biggest collectors of data in the world. According to Google’s own privacy policy, they collect and store data about your home that is collected by your Nest devices. This data includes temperature, humidity, power usage, and even video from the Nest cameras. So if this is all being done by devices that are meant to regulate temperature, one is left wondering what devices intended to listen to you are doing.

It’s all in a name. (Icons via the Noun Project: smart By Nithinan Tatah, T, Speaker By wira wianda, listening By bambaleq, device By DinosoftLab)

As they have spread, we’ve come to call these “smart speakers”. This adoption softens the blow by leveraging two well-known technological descriptors. “Smart” comes from the world of “smart phones” — devices that have overtaken the world and are global norms of everyday life. “Speakers” are devices that we’ve co-existed with for almost a century, bringing us joy through music, information from radio news, and more. How can a “smart speaker” be a threat? But there is alternative we could use that accurately describes their function. We choose to call these devices “listening devices” because it activates the role they play in our lives. They “listen”, actively observing what they hear. They are “devices”, designed based on a plan to achieve some aim, doing far more than simply speaking. We very intentionally call these things “listening devices” to provoke a more uncomfortable response.

A Very Real Privacy Risk

In order to record you, a listening device is supposed to hear a starting command like “ok, Google” or “hey Alexa”. However, in [2019 VRT NWS])https://www.vrt.be/vrtnws/en/2019/07/10/google-employees-are-eavesdropping-even-in-flemish-living-rooms/) (a Belgian news network) was given access to over a thousand of these recordings from a firm subcontracted by Google, and they found that many of these recordings were not preceded by “ok, Google”. A hundred and fifty-three of them didn’t even contain anything that sounded close to that phrase. It is also very common to ask your listening device medical questions, and VRT NWS found that the Google subcontractors have heard those questions as well. The Guardian published a very similar exposé about Apple’s Siri service last year. And let’s not forget about Amazon’s Alexa as well. The technical goal from these companies’ perspective is to use manual review to improve the device’s speech and phrase recognition, however the subsequent media storm clearly indicated that there was a disconnect between the public’s expectations and internal Silicon Valley ethical norms.

We need more investigation to understand exactly what these devices are doing, and research to question the potential implications. For example, the Northeastern Mon(iot)r Research Group dug into the question of how often devices are listening. They found TV shows do trigger numerous activations that record audio in the cloud. Read their “When Speakers are All Ears” study key findings for details. We strongly believe that this is exactly the type of thorough work we need from the academic community to help drive critical reflection, technology development, and policy responses. While this work begins to surface of the different ways these devices could be breaching our privacy, people give little thought when they invite one of these listeners into their homes. There are numerous incidents of these recordings attempting to be used against people in legal settings; it is still unclear just how often to us that is happening or how successful it is. However, at a high level it is clear that our collective desire to use these devices helpful features far outweighs our concern over potential harms.

A Design Problem

These devices are objects we place in our. What about their form? Does it offer clues about its function? Most of these home listening devices are blobs or obelisks that are meant to blend-in with a modern home; they fade into the background so all that is left is an omnipresent voice that responds when called-for. We know that these devices are listening to us, yet their design gives no strong indication that they can hear us. They universally adhere to the “Scandinavian sleek” aesthetic that characterizes our time.

A variety of “smart speakers” — cylinders and orbs (source: mon(iot)r research group

Our theory is that this approach to fading away is completely intentional. Time and time again we’ve seen Silicon Valley draw inspiration from norms in sci-fi literature. The “talking home” is a recurring narrative. Hal, the Starship Enterprise — these are just two of the most popular examples of speaking homes, where the intelligent AI system simply fades away into your surroundings. But this narrative design trick impedes any critical assessment of the technology because it is “out of sight, out of mind”.

This is a problem design can address. How can we design a listening device that prompts the user to think critically about its use? One approach would be to pull from a different sci-fi narrative trope — the helpful assistant robot! R2D2, Rosie from the Jetsons, the robot from Lost in Space. These are mobile characters that physically embody the computational assistant, even though they are not cloud-connected as our technology today has become.

That delightful robot was trying to warm us the whole time! (source)

An Alternate Path Forward

We are planning to build something that bucks the trend, attempting to be transparent about its role as a large set of ears in your home! Our goal is to redesign the listening device to support critical reflection. In order to accomplish This, we are going to have to design and build a listening device that truly lives-up to its name and looks like nothing else on the market. We have a few ideas about how to do this, and a few sketches for potential designs that do this.

This project builds on work from a variety of researchers, technologists, and artists. Karmann and Knudsen’s Project Alias is a “teachable “parasite” that is designed to give users more control over their smart assistants.” Sitting on top of a device, it plays the “man in the middle”, actively suppressing or altering the device’s ability to listen. It is an aggressively contestational design. Rubez Chong explored design interventions inspired by camp aesthetics to offer alternative objects that “hypervisiblize surveillance,” within a larger critique of surveillance technology and societal norms. The Algorithmic Justice League’s Drag Vs. AI project offers a withering take on the rapid adoption of facial recognition technologies by leveraging the popular culture power of drag makeup to confuse the algorithms. These projects are just a handful of those that inspire our thinking and offer us a design language to spark critical reflection on the technologies around us.

So what direction do we plan to go? One obvious core idea is that the device should give the user feedback while it is listening through visual or audio clues. Another way to draw attention to the device listening to us leverages those characters from sci-fi mentioned earlier. We can personify it to be friendly, attentive, or even scary. This is all in sketching and device prototyping now, so if you are interested in working with us on this project or know of related work please feel free to reach-out to us.

Listening device concepts and form inspiration (by Eric Lord)

Sketching Together Online

Tue, 08 Dec 2020 05:09:00 +0000

The switch to remote learning during the Covid pandemic has been challenging for educators of all stripes, and those trying to teach data storytelling have been no exception. Many of our data literacy activities in their current form are dependent on robust small-group discussion and real-time collaboration. This doesn’t translate that easily to online venues, as we discuss in our recent Pandemic Pedagogy workshop paper. Our WordCounter-based “Sketch a Story” activity presents a special challenge in that the main learning happens in small group collaborative sketching sessions. The experience of drawing together is one that’s particularly difficult to replicate online. After some initial failures adapting this activity, we decided to take a more concentrated look at some of the freely available digital options for sketching together online. This blog post shares some initial findings from a testing workshop we held earlier this semester on the subject, reviewing some of the online options and how well they worked for us.

Findings

Earlier this fall we did a quick survey of the online tools we saw our peers trying out for online sketching support in teaching settings. We picked a few to try out. Here’s a quick run-down of what we found, how easily they let us collaborate, and how the aesthetics of our sketches turned out.

AWW — A Web Whiteboard

The AWW interface

AWW ended up being one of the better-feeling drawing experiences we tested out. The UI lends itself well to this kind of activity, too — a wide-open space that feels like a big piece of paper you can scribble on, little clutter, and options that aren’t too granular for the task at hand. You get a pen, a marker, and a basic array of colors with an option to add more. The size slider on the drawing tools is a little uncomfortable, but the way the board smooths out your strokes after you draw them is a great addition for a web tool where people are likely to be drawing with a trackpad. There was a little bit of lag, but collaborative drawing with several people still felt doable.

The story we sketched in AWW — Jimi Hendrix’s lyrics have several trigrams about just chilling out.

Draw.Chat

The Draw.Chat interface

All in all, we weren’t big fans of Draw.Chat. AWW generally does the same thing but better, art-wise, if the device you’re using supports both. Finding and adjusting tools requires a little too much work, and the actual process of drawing doesn’t feel quite as smooth.. It’s a good option for if you aren’t already connected through another platform like Zoom for the activity — there’s a built-in system for text, video, or voice chat — but those elements are more distracting than anything if you don’t need them.

The story we created with Draw.chat — Katy Perry says “oh” way more than anything else — and “oh” is the most common word to be combined with “oh”!

Google Slides

The Google Slides interface

I’m a big fan in general of Google Slides as a tool for collaborative small-group work in a virtual, large-group setting, but the drawing tools across Google Suite have always been pretty bad. “Scribbling” in Slides requires you to manually select the tool for every line you want to draw, text is hard to work with inside of a drawing because it disappears while you’re moving it. Our suggestion would be to use Google Slides as a supplement to a platform intended for drawing in a large-group setting, where groups can post their own drawings and then view and comment on the drawings of other groups.

The story we created in Google Slides — Iron Maiden have much more violent imagery than the other artists, while the Beatles seem to sing the most about love.

Miro

The Miro interface

Our favorite of the set. Although it’s a little more crowded than AWW (due to the fact that it’s not actually primarily a drawing tool), the layout is easy to read, there’s lots of space to work in, changing the size and color of your pen is quick and easy, and best of all, Miro smooths your lines while you draw, making everything look and feel a LOT nicer than drawing with a trackpad usually does. The one caveat for Miro is that it requires an account to use, but the functionality of the free version is more than enough for this kind of task (you’re just limited to 3 boards).

The story we made with Miro — Shrug! Based on Madonna’s lyrics, she’s a pretty carefree person.

Recommendations

While we had our opinions, we know that there can be no single right answer in this situation — depending on the context, who your learners are, and what technology you’re using, the best option will be different. Our students don’t universally have the same computers, access to iPads or other tablets that make digital drawing easier, or even necessarily great bandwidth. Your constraints may be different. After having success with Miro for some other activities, we’re going to try it for supporting WordCounter in virtual workshops and teaching this spring. It has just the right tradeoffs, and the idea that each group could have their own section of the board is great — it lets instructors monitor drawing and makes share-back really easy. Let us know what you’ve tried and how it has gone for you! We’d love to hear about anything we’ve missed.

What Can AR Do for Climate Change Communication?

Mon, 07 Dec 2020 05:09:00 +0000

AR has emerged as a popular approach to interacting with technology - creators are leveraging it for video games, educational experiences, live filters and more. We are motivated by this development to explore how AR is being used for pro-social communication. What can AR offer to things like communicating climate change data? What AR do to help motivate people to take action on personal, collective, or governmental challenges? This blog post shares what we have found so far, and what it suggests for a path forward.

Google Image search results matching “augmented reality climate change”

An Established Technology

Hundreds of millions now have the ability to run AR apps in their pocket; it is an established technology with a vast user base. A dominant form of use has turned out to be in the world of filters, thanks to social media and our collective obsession with selfies! SnapChat and Instagram are perhaps the best example of this. Their platforms have popularized AR filters.

A few examples of filters on Instagram and Snapchat.

AR filters such as these are massively popular, but are mostly used for entertainment purposes. In parallel to this growth we’ve seen rapid development in the tool space for creating these AR experiences - with Apple’s ARKit, Google’s ARCore, and higher level tools like Snapchat’s Lens Studio, Facebook’s Spark AR Studio, and others. The barrier to creating AR experiences is rapidly falling.

Current Approaches to Using AR to Fight Climate Change

Having fun with silly selfies is great, but we think there is an under-explored design space here. Climate change is one of the most pressing challenges of our time. In the US politicians are divided, but the public is not - survey data shows us that the majority of americans believe the government isn’t doing enough about climate change, and that it should be a top political priority. Data plays a central role in assessing and communicating the risk and scale of the impacts, but the barriers to effectively communicating climate change with data have been well documented and discussed for years (see Moser 2010 for a good summary).

People throw every new technology that emerges at climate change, and AR is no exception. We’ve seen a few patterns in the small set of AR projects that have focused on climate change over the last few years:

Reflecting on our present conditions
Imagining the future of our climate
Designing resilient cities

Here’s a quick run down of some of the projects we came across in our survey of work.

Reflecting on our present conditions

A standard approach of data visualization is to “make visible the invisible”. We came across a set of projects that attempt to do this, showcasing the climate change happening around us already.

Screenshots from the AiR research app

We also found a number of academic researchers are investigating how to best use AR to show pollution levels. “AiR – An Augmented Reality Application for Visualizing Air Pollution” from Matthews, et al overlays your view with virtual smog based on local pollution levels. “Aire: visualize air quality” from Torres, et al last year similarly adds moving virtual pollution particulates on top of a live camera view via AR.

Imagining the future of our Climate

A standard way to imagine the future is to show it - AR makes that possible by modifying our view of the world around us. Numerous projects we found are using AR to show us the world of the future if no action is taken. The most common approach we saw to this was sea level rise. In fact, we saw so many examples that we’re pushed to call sea-level rise AR apps a cliche at this point.

The effects of climate change are altering the world around us.

See how the ocean could be impacted by 2100 with @SnapLensStudio’s Waterline Lens on @Snapchat. #EarthDay #ClimateAction pic.twitter.com/FnWTsylOpR
— UN Environment Programme (@UNEP) April 22, 2019

Here is a quick sampling of the apps we found that showed sea level rise:

Snapchat itself worked with the UN Environment Programme to create a lens in 2019 showing how sea level rise would impact your view of the world by 2100.
In 2019 the insurance company Hiscox created an app to predict flood levels on an AR model of parts of NJ (available for Android in the Google Play Store).
Artist Justin Brice Guariglia created After Ice in 2017 to show New York City residents how flooded it would be if the ice caps melt.
Geospatial educators included sea level rise in a larger AR app they’ve used since 2016 introduce students to concepts such as topological maps.
Journalist Eli Kintisch worked with researchers at MIT and RISD created the Looking Glass app in 2013 to show New England residents their risk from sea level rise.

Designing Resilient Cities

In the US at least, due to the lack of national action on climate change, cities have become a central organizing unit for our response. This has led to large, well-funded efforts like the Resilient Cities Network. Accordingly, we saw a number of AR apps that focused on climate-proofing the city.

Screenshot of Earthwatch’s app

Earthwatch Europe took an approach similar to SimCity in building an AR app that allows uses to try build a “climate proof” city by altering the makeup of land use, water supply, drainage, and more. The Klikovand task force worked with others to build an AR app that allows users to place green terrain in their built environment to provoke conversation about long-term climate resilience and day-to-day public works decisions. Artist Juan Carlos Gallo and others created an AR-connected mural in Miami to warn about climate change danger to localities.

An Opportunity for Evaluation and Impact

After exploring this space of climate change AR experiences, we are excited because we believe it has a lot of potential. We offer this blog post as a call to arms to study and explore the opportunity more! For instance, we saw little study of what people take away from these experiences - no assessment of whether their feeling of risk has increased, or if they are more motivated to support some policy, or even to share the message of warning with their friends. We are left feeling that there is more we can try out here.

We need to experiment and study more. Perhaps AR can actually be an effective tool to help us communicate and motivate action to the dire threats of climate change that are all around us. We just don’t know yet. Our next steps on this project are to audit the rapid prototyping tools we mentioned earlier to see what their design affordances are.

A screenshot of using Spark AR Studio- Ballori’s new look!

After that we’re hoping to build a simple experiment to study how people respond to AR experiences that are built around climate change data. Our key questions involve comparing AR experiences to other techniques like flat data visualization, infographics, or journalism-like narratives. We want to investigate question such as:

Does AR communicate the threat more effectively?
Are participants more motivated to act on a threat after an AR experience?

Drop us a line if you have a project we can learn from, or feel free to reach out if you’d like to collaborate with us.

Data Skits - Learning From Participatory Theatre

Thu, 03 Dec 2020 05:09:00 +0000

Last year, we came together and started exploring the experience of data through movement with our first “Data Moves” workshop. Our driving questions related to how these interactive and somatic methodologies could enhance the way we engage with information. We started with a simple question - what happens if we dance our data? Among many observations from that workshop, we saw a distinction between those who didn’t feel comfortable moving in a dance-like way and those who did. As a result, some participants began to create skits from the data stories they found. This sparked the idea of exploring the world of theatre alongside dance, in order to uncover the rich resources and pathways theatre has to offer. How can we create theatre-based experiences that bring people together around data in social contexts, in order to build empowerment, challenge power structures, and build capacity to speak the new language of data? In particular, what does this mean in our current pandemic, where the arts are reworking and reimagining themselves in response to social distancing guidelines?

This blog post summarizes a review of interdisciplinary literature on data performance through theatre, as well as case studies of potential theatrical approaches in social justice contexts. We derive a list of key observations that will inform the next phase of our investigation in data skits. These key takeaways serve as a preliminary starting place for the theatrical avenues that may be worthwhile in a future iterative workshop series.

Blurring The Line Between Audience and Stage

The first key takeaway that we noticed in our literature review is the participatory experience of blurring the lines between actors and spectators, or the stage and the audience. Traditionally, dismantling these boundaries has functioned to eliminate the cathartic element of theatrical convention, and promote audience participation both within and outside the theatre. The audience may have strong visceral and emotional reactions to a piece, but they are more likely to take action when they are invited to act in the experience of the theatre. We found a few different related forms of theatre, each of which offer worthwhile next steps in breaking down the hierarchies in ways that encourage equal participation between actors and viewers.

Image from Living Arts Playback Theatre (source)

Playback theatre, used to help people reconcile with their experiences through repetition and storybuilding, uses tools such as “moving sculptures” and “group choir” to reflect and dramatize stories shared by participants. This form of theatre dismantles traditional boundaries in that it forces audiences members to physically build the depth and breadth of their experience, and participate in telling the experience through a given set of performance norms. Participants share experiences that are “played back” to them through different short scene/skit formats, some of which require interaction and reflection. In one research study done with medical students, students were encouraged to share formative moments of their medical education, in order to increase their empathetic reasoning and compassion in the medical education realm. Other projects highlighted healing and reconciliation for post-war communities, mental-health recovery, or healing for adolescents in Palestine refugee camps. Playback theatre aims to create a deeper, multi-dimensional understanding of experiences, which helps in empowerment, decision-making, processing, and reconciliation.

Theatre of the Oppressed, a form of theatre designed for social and political activism, uses participatory theatre techniques to explore oppression in all forms- internal and external, individual and societal, and more. These techniques include “image theatre” activities such as human sculptures, but also problem solving through role play and even staging participatory scenarios involving an unaware public. Traditionally, Brechtian/”epic theatre” also aims to encourage participation and dismantle boundaries, through interrupting the theatrical experience and charging participants with decision making, analysis, and sharing judgement. Verbatim theater, theater crafted using and manipulating the direct words and phrases of participants or source material, has also shown its worth in empowering communities towards political action.

Image from Theatre of the Oppressed NYC (source)

Although these theatrical methodologies are a drop in the well of participatory dramatic resources, from them we learn the value of dismantling traditional audience/actor boundaries that can be leveraged in data analysis. These forms of theatre, participatory by nature, align with our goal to create participatory spaces that bring people together around data. We have solid approaches to doing this in non-theatrical ways, but the commonground is worthwhile to explore.

Narratives Emerge from Interaction

Another key takeaway from the literature is the value of structuring performance in real time, relying on mutual exploration, rather than beginning with an implicit agenda of disseminating information. This element of shared discovery and storytelling, structurally influenced by the perspectives and experiences of participants, implicates a sense of control and responsibility amongst the collaborators. Rather than starting with a dataset to impart on an audience, beginning with a set of questions can potentially stimulate a richer performative dialogue. These questions can be crafted by a set of data, and then shaped by the participants as they navigate through their interpretive/reactive experience. In the words of social science dramatists and researchers Ross Gray and Christina Sindig, “A drama that emerges from such a process is very much a negotiated settlement, a collective achievement.” This takeaway resonated with our goal of building empowerment through participatory performance - enabling decision making, multi-sensory processing, and an elevated sentiment of self-efficacy.

Our hope is that performative explorations of data will also bolster effective communicative skills. Through performance-based approaches, participants will learn how to speak and embody data, and leverage this knowledge towards goals within their communities.

Shared Goals

The literature also highlighted the structural element of data visualization and performance, in both the process and the outcome. The process and product of data performance occur simultaneously, as a voluntary creative experience allows participants to simulate questions, decisions, and outcomes. Information designer and author Fredrica Fragapane notes that there are 3 commonalities that highlight the mutualistic relationship between theatre and data visualization- they convey information, they visualize, and they represent the result of careful choices. The process of visualizing data for the goal of empowerment requires careful analysis of choices, and a participatory decision making process. This is inherent in participatory forms of theatre, and evidence of the simultaneous aspect of the process and product. This insight connects to our goal, as we want to explore existing datasets to find the stories in them while simultaneously turning these stories into empowerment that leads to action. We aim to increase the participants’ efficacy- their belief that they can influence the system presented in the data to make it better for relevant people and communities. This is reminiscent of the Brechtian theatrical goal of disrupting the status quo, through challenging traditional theatre boundaries and the power structures inherent in the data.

Image by Frederica Fragapane (source)

Another inspiration for this work can be found in theatre’s form - a real-time, enacted, performative telling of a story. Data itself does not exist on a two dimensional plane- it results from experiences, observations, and questions. However data is often presented in the abstract, disconnected from the real world phenomenon and people it portrays. Theatre offers us opportunities to reconnect data with its three-dimensionally, in its replication and representation of the human experience. This is very much in line with some of D’Ignazio and Klein’s arguments about Data Feminism - they specifically provoke us to “bring back the bodies”.

The approach of participatory theatre is valuable in reclaiming, reinterpreting, and grappling with data in new ways, which can act as a tool to fight power imbalances within marginalized groups. Theatre isn’t defined by any one of its components- movement, body language, space between bodies, etc. Rather it is a summation/amalgamation that transcends one singular sensory experience. From words, lines, and sentences, to shape, breath, and time, theatre makes use of the intersection of all of these tools. This connects to our pedagogy of constructionism- it harkens back to the approach of body-syntonicity in our early explorations. The invitation to engage in a multi-sensory experience strengthens the outcome of these methodologies. It can be a useful structure to recharacterize and redefine data, to create a more just power dynamic around information.

Next Steps

We are delighted about all of the parallels to our existing work, and new frameworks we can learn from. Our next step is to design activities that bring people together around data inspired by these lessons from participatory theatre. These will revolve around the pre-existing goals that drive our data work - encouraging empowerment, bolstering decision-making, grappling with past experiences, making sens of information, helping those without power learn to “speak” data, and creating a fun space. We’ll be doing this with the added challenge of experimenting in the virtual space, but look forward to learning with community members. Drop us a line if you know of related work, or want to collaborate!

Workshop for Librarians - How to "Speak Data"

Sun, 15 Nov 2020 05:09:00 +0000

We will be leading a virtual workshop for the Network of the National Library of Medicine. This online workshop is targetted at public librarians across the US. It is scheduled for Tuesday Dec 15th at 2pm EST. Register Now!

How to “Speak Data”: Librarians as Public Data Ambassadors

Data has become central to many aspects of civic life - governments run open data portals, organizations release public datasets, newspapers publish data-driven stories. How can librarians navigate this all more effectively? How can you help library patrons learn to use what they find in these data resources? Librarians can play an impactful role as “data ambassadors”, assisting their communities in finding and using data as part of their life as citizens. Join Prof. Bhargava for an interactive virtual workshop that will introduce participatory approaches to building your ability to “speak data”. You’ll leave the session with more confidence in your own ability to work with data, a language for talking about the role data can play in civic engagement, and experience with playful activities you can run yourself to build data literacy with your colleagues or patrons.

Librarians as "Civic Data Ambassadors" - revisiting a project

Mon, 02 Nov 2020 05:09:00 +0000

Librarians are key connections between the public and information of all types, but how can we help them introduce civic data better? The Civic Data Ambassadors program was a six-week online course that taught librarians how to be ambassadors for open data in their communities. Launched by the Engagement Lab at Emerson University, it was led by Catherine D’Ignazio and Eric Gordon and featured Rahul Bhargava as a video trainer (you can still watch all the videos produced). This program was built around many of the first hands-on activities that are now part of our Data Culture Project toolkit. It was funded by the City of Boston. This short blog post revists that project and reviews some of the post-course feedback to explore what we learned from these Boston-area librarians about how they can help library patrons access and utilize data better.

“As a data ambassador I will now be able to help others in my community access important information that could help them make more informed decisions, fight for change and be active members of their communities.” - Civic Data Ambassador Participant

Building Knowledge of Where Data Are

“Where” is the question about data that our Civic Data Ambassadors were most likely to understand as being part of their role before starting the course. The “where” of data is its most concrete aspect: where are datasets located, and where is data located within those datasets? Several of our participants noted that they weren’t themselves familiar with the kinds of data offered via “Analyze Boston” (Boston’s open data portal) before the course, and the course gave them a confidence in the ability to direct library patrons to data that might be useful to their purposes. One of our participant librarians, in their post-course surveys, described the role of a Civic Data Ambassador as being to ”help students, researchers, and the public search for, locate, and reuse civic data,” another as to “connect people with a starting point as they seek transparent information from the city in which they reside.” These definitions demonstrate an understanding of the importance of being able to direct people to where the data is.

Learning How to Use Data

What many of our Civic Data Ambassadors understood from the course is that having data is only half the battle–as one of them succinctly put it, “open data, without data training is still essentially closed data.” Although having access to robust datasets is important, equally important is knowing techniques for how to analyze and use data, a process that isn’t always intuitive. Several of the participants noted that specific tools such as WTFcsv and WordCounter that they learned to use during the course were helpful resources for this aspect of their role. Definitions that reflected this aspect emphasized the role Civic Data Ambassadors play in helping people understand the contents of civic datasets, as well as “making them more confident in using data.” As one participant put it: “I knew this data existed but I didn’t know how to approach using it, and what tools I could use to tell a story with data.” When participants help people understand data, they’re helping them see data as a cohesive narrative–not just an assortment of numbers.

Understanding Why Data Can Help

This last aspect of participants’ understanding goes beyond where data can be found and how it can be understood into the story that data tells and the impacts that it has. Several of our Ambassadors understood the “whys” of the data they share as being vital to different aspects of civic life–from creating “free and open democracy” to “shap[ing] the future of the City of Boston.” This category also includes awareness of the background of the datasets, and acknowledgement that data doesn’t come from nowhere. One participant said the following: “I learned so much about how to teach and guide others in the world of data, but also some of the decision-making that goes into each civic […] dataset.” Understanding the origin of data can be as important to its understanding as the tools we use to analyze it.

Next Steps

Across all these aspects of the role of a Civic Data Ambassador, based on both self-rating and qualitative comments made at the conclusion of the program, we saw participants evolve both in their understanding of how they can serve people seeking data in their libraries and in their confidence in their own abilities to help patrons with their data needs. We strongly believe librarians can help, and are helping, patrons navigate public data sources to support their community needs and goals. The high-level findings from this program inform potential next steps to take as we reach out to more library programs and we hope to continue collaboratively designing programs to expand librarians’ abilities and toolkits to act as “Civic Data Ambassadors”.

The Data-Driven Citizen

Mon, 26 Oct 2020 05:09:00 +0000

As a double major in computer science and cognitive science back as an undergraduate, I was fascinated by how even complex concepts might be effectively conveyed to a broad audience, given some understanding of how people think. One of my favorite class readings was an essay called “The Science of Scientific Writing,” which uses features of how people process information to show how simple changes in writing style can dramatically increase reader comprehension. I believe the same framework may be used in other communication channels to add clarity to previously impenetrable muddle.

Making complex topics accessible to broad audiences is no more critical than in the realm of enabling functional democracy. The democratic system, by its nature, requires that ordinary people understand and decide on a mind-bogglingly broad array of highly complex issues like international economics, healthcare infrastructure, legal procedures, and more. Or at least it requires a level of understanding that enables citizens to select the best candidate to decide for them, which in itself may be far from easy. At a moment in history marked by viral misinformation on topics of health, social, and civic import, it is clear our systems for effective public communication have broken down. Undoubtedly social and political factors are involved here, not simply barriers in communication technologies or poorly designed fact sheets. Nevertheless, an effective civic data communication infrastructure remains a necessary, even if not sufficient, component of any transparent and well-functioning democratic system. This infrastructure forms the foundation of an informed citizenry, productive public discourse, and, ultimately, intelligent policy decisions.

In this post I will review research literature and other sources that shed light on some key questions core to this design challenge:

How do ordinary lay people process data and statistics?
What are the current constraints and approaches to sharing civic data?
How do citizens evaluate these current design practices?
And finally, how might these design practices be improved to facilitate better citizen comprehension of relevant civic information, more data-driven advocacy, and better policy making?

These insights will be used to inform a larger research initiative, undertaken with Professor Rahul Bhargava of the School of Journalism and Media Innovation and the Department of Art + Design at Northeastern University as well as in collaboration with an external community-based organization. We plan to collaboratively prototype a design product aimed at improving on existing tool sets for communicating civic data to citizens of the United States at the local city level.

What we know about how lay people process statistics

To design a relevant information design product, it is vital to understand the perspective of the user. In this case, we are interested in understanding the way that US citizens engage with civic data dashboards and other related communication channels. A high-level trend across much of the research literature is that, unlike researchers, lay people tend to be more highly influenced by cultural, emotional, and personal factors in their engagement and evaluation of data dashboards. A study by Yuo Luo and Jiaying Zhao of the University of British Columbia, for example, noted that lay people can look at exactly the same data and have completely different conclusions based on their prior personal and/or culturally biased beliefs (“How to engage different audiences with the same graph?”). In fact, those with different prior beliefs actually focused visual attention on different parts of the same graphical chart.

Similarly, in a webinar reviewing multiple recent research findings related to public interpretation of civic data related to COVID-19 risk, researchers found that emotional and cultural influences highly impacted citizens’ perception of risk level given the same information, leading to widely different levels of compliance with public health guidelines (EPI-WIN webinar: “Influencing Risk Perceptions about COVID-19”). Citizens were also influenced by social factors such as the existing level of trust within their community. Without baseline levels of trust between citizens and their governments - whether local, state, or national, impactful engagement with community metrics was very challenging.

Interestingly, the particular format of how data is conveyed can significantly impact its persuasiveness for a lay person. Moreover, how exactly this plays out depends on whether they have strong prior beliefs about the topic in question. In a study evaluating the persuasiveness of different types of data visualizations on various controversial topics, viewers already biased in favor of key conclusions of presented data found that graph representations - rather than data tables - were more persuasive. Those biased against the conclusions presented, however, found that numerical tables were more persuasive than graphs (“The Persuasive Power of Data Visualization”).

Another study highlighted relevance to personal life experience as a factor. In this study, researchers at Bucknell University interviewed citizens of rural Pennsylvania about their preferences for and interpretation of various civic data visualizations. Their finding was that these residents preferred and engaged most with those whose topics were most relevant to the viewer’s personal experience and situation. They also preferred simplicity and attractiveness in visualizations (“Data is Personal: Attitudes and Perceptions of Data Visualization in Rural Pennsylvania”).

This preference for simplicity and usability was true also in a study on civic dashboards in Dublin in which residents said they gravitated toward and preferred visualizations that were simple, structured, usable, attractive, accurate, and which had personal relevance (“Creating design guidelines for building city dashboards from a user’s perspectives”).

Overall, the research literature indicates that personal and emotional factors, existing trust levels, and other contextual factors must be taken into account in the design of any public-facing data interface, as well as a sensitivity toward usability, relevance, and visual appeal.

Current constraints and practices in conveying civic data to the public

Existing data dashboard design practices by governments and nonprofits are constrained by legal guidelines, lack of resources, and challenges in coordinating among many organizational stakeholders (“Bringing Clarity to Transparency: A Study of Dashboard Implementation in the Public Sector”). Unfortunately, these constraints can lead to questionable data quality, incorrect data interpretation, bad analysis, manipulation of data in various ways that support a predefined view, and lack of the engagement needed to create impact. For example, researchers at Delft University of Technology highlighted these challenges and noted that dashboards, if not implemented correctly or supported by baseline levels of trust among the community, created the risk of even less transparency and trust (“Data science empowering the public: Data Driven dashboards for transparent and accountable decision making in smart cities”). They also noted that civic dashboards needed to be complemented by mechanisms for supporting appropriate data interpretation and effective citizen engagement.

Numerous research case studies also indicate the prevalence of lack of standardization, variability in adherence to accessibility standards, assumed knowledge of specialized systems (e.g., GIS), and challenging usability (“Use of Dashboards in Government: Fostering Transparency and Democracy Series”). Overall, **governments and nonprofits face organizational, resource, and coordination challenges, which often result in lower usability, data quality, and fewer support systems for facilitating effective engagement. **

How we might improve

Condensing and conveying such potentially disparate economic, health, social or other information in a form intuitive and appealing to engaged members of the US public with varying levels of analytical background is not an easy task. Nevertheless, insights from case study research reveal that user-centered design practices, prioritization of practical relevance, and compliance to best practices in usability can produce valuable results. For example, a case study documenting the user-centered creation of a data tracking tool for policymaker accountability details how a focus on data related to policy maker promises empowered citizens to better evaluate their elected representatives (“Promise Tracker and Monitorial Citizenship” (blog post)).

In another case study, University of Salerno and University of Campania Luigi Vanvitelli created SPOD, a Social Platform for Open Data, which enabled users to access, manipulate, and share open data while engaging in data-driven discussion about issues related to their local communities (“Engaging Citizens with a Social Platform for Open Data” (conference ppr)). This custom tailored solution additionally engaged citizens in open data sharing platforms by combining these with functions for forums and interactive discussion. This helped actively engage more citizens in exploiting open data, while facilitating more effective, data-centric discussion in a way not well supported on traditional social media platforms. Here the authors additionally leveraged the phenomenon of “Web 2.0,” or an internet built on distributed, user-generated content and interaction, that has redefined the concept of “active” media consumers “Active Audiences” (paper). Data dashboards may do well to empower citizens to engage among their peers to make the dashboard experience more dynamic, relevant, and engaging. This solution demonstrated a case study in the newest chapter in the evolution of citizen monitoring.

Key Takeaways

In summary, citizen lay people constitute a unique class of users of data communication products. These users are highly influenced by personal, emotional, social, and contextual factors and generally prioritize simplicity, usability, accuracy, and personal relevance. Unfortunately, historical and organizational constraints among government and non-profit providers of current data dashboards frequently cause questionable data quality, incorrect data interpretation, bad analysis, manipulation of data in various ways to support a predefined view, and lack of the engagement needed to create impact. Nevertheless, recent case studies demonstrate that improved and even engagingly interactive solutions are possible given practices in user-centered design, prioritization of practical relevance, and compliance to best practices in usability standards.

These insights will help inform my approach in my broader research initiative to prototype a design product that improves on these existing toolsets in several ways:

First, this design product will build on best practices in participatory design and user-centered research. Creating a dashboard that presents exactly the right data in the way most valuable and relevant to our users - engaged citizens - is critical. This process will result in a data dashboard that supports our users and external organizational collaborator’s specific, custom goals. Additionally, a scalable approach would be ideal to maximize impact and accessibility among many users.
Second, I will opt to target my solution at the local city level. This is because local communities, unlike (quite frequently) national communities, can often display the baseline levels of trust between citizens and government often required to engage in any productive way with data. Also, taking initiative in protecting citizens at the local level has perhaps never been more important than at this very moment. This niche additionally aligns with my own research interests.
Third, it will be designed for optimal simplicity and usability. Intuitiveness of a dashboard, specifically to American laypeople with varying levels of analytical background, is critical. Existing research demonstrates this as a critical factor in citizen value.
Finally, the design product should look attractive and engaging. Generating interest and engagement is a key goal of this initiative and a substantive factor in user experience.
To promote trust of relevant insights, I also hope to focus on high quality data sources that are additionally perceived by the public as most credible, de facto, and incontrovertible. If possible, it would be great to at least explore an embedded interactive component to the product, such as a forum capability, to make it more engaging and as an ongoing temperature check of the relevance of the data to the needs of users.

In closing, designing effective, engaging design solutions to promote an informed citizenry is an important task. Although doing so presents unique challenges - both from idiosyncrasies in how lay people process information and via organizational and regulatory constraints - approaches that incorporate user centered-design, prioritization of practical relevance and compliance with best practices in usability standards can result in valuable products. I plan to implement these strategies in my impending research initiative to improve on existing civic dashboard infrastructure.

Interested students, researchers, organizations, or collaborators would like to become involved with our project can reach out to bullister.d@northeastern.edu.

Paper: Pandemic Pedagogy @ IEEE Vis 2020

Tue, 20 Oct 2020 05:09:00 +0000

The pandemic has created significant teaching challenges for data visualization educators. Classroom-based hands-on activities don’t readily translate into engaging online experiences. This paper lays out three guiding principles I used to make this transition - maintaining learning goals, embracing the affordances of available technology, and activating the learner’s physical setting as part of the activity. I present two activities that I adapted to work online as case studies in how those principles play out - building data sculptures and sketching visual word webs. The three principles, case studies, and reflections offer insights and lessons to other educators designing and leading virtual activities with data visualization learners.

Presented at the VisActivities Workshop at IEEE Vis 2020.

Download the PDF.

Pandemic news coverage is falling, but cases keep rising

Mon, 21 Sep 2020 05:09:00 +0000

The Covid-19 pandemic has yielded unprecedented media coverage. Over the last six months, no single topic has taken over online news quite like the coronavirus has. In fact, at its peak in late March, almost 75% of all stories in top U.S. online media sources mentioned coronavirus in some way.

But that attention seems to be waning, according to our recent analysis of 12.5 million national- and state-level news articles collected through Media Cloud. Since its peak in March, media coverage mentioning coronavirus has tapered off while cases continue to rise – a troubling trend as the virus continues to infect and kill. The U.S. is currently logging more than 33,000 cases per day, and the death tally is over 190,000.

Mentions of coronavirus in 87 top U.S. online news sources between January and September 2020 (rolling 3-day mean).

So by how much, and where, has media coverage of Covid-19 fallen off? And does coverage track incidence rates in states that have had late surges of infections? We set out to measure how attention to this public health crisis has played out in the last six months across every U.S. state.

State-level media aren’t responding to more recent local outbreaks

While a handful of sources like The New York Times or The Washington Post are the nation’s papers of record, the vast majority of the nation’s papers cover smaller geographic areas. Take the St. Louis Post-Dispatch in Missouri or The Oregonian of Portland.

The U.S. has a fairly robust set of media sources like these that cover news at the state and regional level and Media Cloud has a manually curated and reviewed collection of online news sources for each state. In some states these collections are dominated by stories from business sources, like the St. Louis Business Journal and the Kansas City Business Journal in our Missouri collection. In other states, local TV stations produce most of the stories in our database, like KDXU in our Utah collection. No state collection looks exactly like any other, reflecting the diversity of local news outlets across the U.S and the varying levels of digital transformation that has brought them online or not.

Using this database of stories from these local online news sources, we looked to see how much attention coronavirus received in each state. The search terms we used included coronavirus, Covid-19, Covid and pandemic. Across all states, coronavirus was mentioned in 32% of stories published between February and September. This is lower than the national coverage shown above, but still remarkably high. Moreover, the pattern of coverage was remarkably consistent across states. There was a strong surge of coverage in March, then a slow decline until late May. At that point there was a large across-the-board dip as media attention turned to the killing of George Floyd. Then in June pandemic coverage rose and fell periodically, but overall continued a slow decline.

Each line represents the percentage of coronavirus-related stories published by sources in a state (3-day rolling mean).

We know the pandemic has spread across states differently, so seeing such uniformity in state-level media coverage over time is quite surprising. It left us curious – if you look at state-level coronavirus incidence and compare it to state-level media coverage, do you see a connection? Are local media sources responding to outbreaks, or trends, with increased or decreased coverage? To do this, we pulled in data on the number of new daily cases per 100,000 people from the Johns Hopkins tracker.

Arizona In Arizona, we see a major spike of new cases in July (purple). However, there is no commensurate growth in media coverage (teal) during that same time. There was a massive gap in local coronavirus coverage over the summer – a missed opportunity for Arizona papers to report local stories impacting readers’ lives directly.

Massachusetts In Massachusetts media, we see a fairly consistent tracking between the percentage of stories about coronavirus (teal) and the daily new cases (purple). Here in Mass, we had an early “superspreader” event but rates have fallen consistently since. Local media coverage has reflected that.

Looking across the U.S., you can see that the connection between media coverage and the incidence of new cases of coronavirus varies wildly by state. In states like Arizona, Louisiana, Texas and some others, online media coverage about coronavirus didn’t increase when new cases increased. In other words, online local news was not responding to local coronavirus outbreaks with increased coverage. What are the effects of that coverage gap? Why does the percentage of attention to coronavirus track national levels so closely no matter the state?

A state-by-state map comparing the percent of online news stories mentioning coronavirus (in teal) to the number of daily new cases per 100,000 people (in purple). Both lines present a rolling 3-day mean. Click to zoom in. Code and data is available on GitHub.

Media responsibility during a pandemic

In a recent Atlantic article and ongoing Twitter thread, UNC sociologist Zeynep Tufekci calls out editors for their image choices on articles about risky behavior. She points out article after article about risky behaviors featuring shots of beach crowds. This is irresponsible, she says, because science tells us that one of the safest places to be right now is outside. “Enough with the beach-scolding,” she argues.

My daughter is safe and sound, chilling at the beach.

This small example gets at a larger argument about the role and responsibility of media in a pandemic, and towards public health at large. Look at the gap of coverage in Oklahoma, where coronavirus continues its steady rise but media reporting has left it out of the headlines. Is the media doing a disservice to public health? Could this gap even be contributing to sickness and death?

Editorial decisions at papers across the country are driven by subjective judgments about newsworthiness and well-discussed business pressures related to pulling in more readers in a hyper-competitive media landscape. I don’t envy the editors making these calls about how to balance the parallel crises of 2020, but I’m left feeling upset that the summer didn’t see more day-after-day coverage of the pandemic as it spiked. Coronavirus continues to surge around us, and still upends our daily lives, but the news just isn’t covering it as much.

One guess is that editors are reflecting the “coronavirus fatigue” we are all feeling; there is only so much pandemic coverage we can handle. However, reading about the latest White House pandemic scandal is very different from reading about a coronavirus outbreak down the road. The former can leave readers feeling disempowered and defeated while latter could spur tangible local activism on a short timeline. Geographically localized outbreaks are an opportunity for smaller newspapers to lead coverage with their local expertise and networks. Northeastern’s dismissal of 11 students became a national story, and reporters and editors at the campus’s independent newspaper, The Huntington News, were sought out by multiple news outlets. Recent data also shows local news sources are more trusted than their national counterparts. Local outlets could leverage that trust to pick when the national pandemic response merits attention versus when close-by pandemic rates are critical enough to focus on, selectively working to avoid the coronavirus news fatigue. At least, I hope they can.

Acknowledgments

Thanks to Aleszu Bajak for editing and data sanity checking help. Also thanks to Emily Ndulue, Ethan Zuckerman, Dennis Jen, and Fernando Bermejo for comments.

Originally published on the StoryBench blog.

Welcome

Tue, 01 Sep 2020 05:09:00 +0000

Welcome to the online home of the Data Culture Group at Northeastern University. We are an interdisciplinary research group led by Professor Rahul Bhargava, housed in the College of Arts, Media and Design.

The Data Culture Group builds collaborative projects to interrogate our datafied society with a focus on rethinking power and participation in data processes. We are housed in the College of Arts, Media and Design at Northeastern Univeristy and led by Professor Rahul Bhargava.

Our projects are collaborations with community groups, media companies, social scientists, computer scientists, and students.

“It is only when the they go wrong that machines remind you how powerful they are” - Clive James

Data Culture Group @ Northeastern University

Learn CARefully (keynote on teaching AI for data journalism at SciCAR)

Some Context

Be Specific In Your Learning

Learn by Trying Things Out

Center Journalistic Principles

Stay Critical

Reflecting on Knowing

Learn CAR-efully

Whose Quotes Get Cleaned Up?

The Politics of Quotes in News

Can Data Help Us Understand This?

How’d I Do This?

So What?

Data Drums wins a Data Sonification Award

Beyond Accuracy: A Case Study of Sociotechnical ML Evaluation

Some Background

Improving the Model

Experiment 1: Should we incorporate better data?

Experiment 2: How about more complicated models?

Experiment 3: How confident should the models be?

Towards Sociotechnical and Community-Centered ML Evaluation

Data Journalism in Dark Times: notes from the C+J keynotes

When Data Journalism Isn’t Enough

Data as Weapon and Shield

User Agency vs. Algorithmic Power

Navigating Narrative Warfare

What Stays With Me

Notes from MISI Summit on Building Trust in Climate and Science Journalism

The Business of News: Audience Trust and the Future of Science and Climate Policy Journalism

Local News, Attention, and Transparency

What’s Actually Working

The Hard Questions

Notes from csv,conf,v9: Community-Centered Data Stories

Filling Information Voids with Chatbots: Using LLMs on WhatsApp for News Access

Blazing Fast Automagical Metadata

Building CSV-powered tools for social sciences

Keynote: Developing tech with community: the example of Open Data Editor

Democratizing data: data literacy for community action

Keynote: Clara Jiménez & David Fernández Sancho

Data without borders: Straight out of spreadsheets and into the streets

Keynote: Fifteen Years into the Open Data Movement

Reflections: Data as Community Practice

ACH Keynote on "Civic Jugaad - Reclaiming Digital Innovation for Social Good"

1. Connect people

2. Amplify resistance

3. Build with, not for

4. Study systems of power

Civic jugaad as a response

Protest Mapper: a new tool for journalists covering protests

Media Attention to Protests

Finding & Evaluating a Public Protest Dataset

A Mapping Solution

AI for Good: Five Key Principles for the Pro-Social Sector

1. Center Underserved Issues

2. Build With, Not For

3. Study Up

4. Look for Quick Wins

5. Consider Hidden Risks

Nebraska Data Art Jam: Beyond Visualization

Dr. Laura Guertin’s Data Quilts

Matt Waite on Data Journalism with AI

Robert Twomey on Data and Art

My Own Take on “Data Beyond the Visual”

Journalism papers at IEEE Vis 2024: selected highlights

Talk Summary: Dr. Vishwanath on Science Communication & Misinformation

The Problem: An Overloaded Information Ecosystem

Inequalities in Health Communication

Structural Solutions: Rebuilding Trust Through Community

Reflections on 'UN Human Security in the AI Era' Event

Existential Risks in the AI Age

Restoring Trust in an Age of Worry

Towards Equitable AI Futures

Sculpting Data with Clay – A New Take on Data Physicalization

ListeningCups: Tactile Data Stories

Rain Gauge: Visualizing Climate Data with Sustainability in Mind

Clay as a Medium for Data

AI-Powered Content Audits for Local News

What We Did, and What We Found

What You Might Want to Try