Great questions, Andres! Yes, we're certainly planning to create additional stories for the project, and we currently have a new story (about algorithmic bias in the criminal justice system) in review that we're hoping to test with youth soon. Working directly with teens and learning about the issues that most resonate with them helps provide ideas for new stories -- but if anyone has ideas for other stories that they'd like to see, please let us know! We hope to get to the point where students are contributing their own short stories to the project, but in this first year we focused on supporting students in telling their own stories through comics (using the comic-making software, Pixton EDU). This enabled students to create their own stories about issues of interest or concern, but in an approachable way that seemed to be really fun and engaging for most students (and, added bonus, worked well over Zoom, too!).

In terms of designing their own AI systems, we used Google's Teachable Machine to introduce students to machine learning and in particular, to highlight the impacts of biased training data. Teachable Machine is easy for students to use and understand, regardless of their prior experience with computer programming, which made it a good fit for our purposes. Some other online tools, like App Inventor and Machine Learning for Kids, offer similar functionality but provide opportunities to integrate some basic programming as well.

One challenge we face in our project, particularly in our online summer camps (due to time limitations), is finding the right balance between time dedicated to exploring the technology and tinkering with the technical/AI tools and time spent diving into the relevant ethical issues. It can be a tricky balance, allocating sufficient time for each component, so that students are able to develop their understanding about how AI works, while also having time to reflect about its broader societal impacts.

People at the Processing Foundation (https://processingfoundation.org/), including Daniel Shiffman who runs the popular Coding Train channel in Youtube are very interested in AI Ethics. That'a a community that could be interested in your project, and in user-contributed stories.

Thanks for the suggestion, Andres - we'll look into that!

Thanks, Michael – glad to hear you enjoyed the Alexa opening.  : )  (With so few photos and videos from this year’s remote programming, we had to get a little creative with the video!)

In developing this first set of short stories (which were authored by Ellie Haberl, a member of our team), we initially identified key AI ethics issues that we wanted to highlight, including algorithmic bias, data privacy, etc. Ellie then worked her magic to create original short stories, including realistic fiction and speculative/dystopian fiction, that served as an anchor for class discussion, activities and reflection. (Unfortunately, we had to trim from the video some of Ellie’s discussion about the story development process, due to time limitations; hopefully, she’ll be able to chime in here with some additional detail!)

And yes, I think the experience was quite eye-opening for the teens who participated, many of whom came into the class without a clear idea of how AI was relevant to their lives. Students frequently referenced the stories when discussing different issues and in some cases, commented that a story had inspired them to make some type of change in their use of technology (e.g., checking their privacy settings, turning off notifications, etc.).

HI Michael,

I'm on the team as well, and wanted to add a bit about our story development process.  We intentionally made the stories short  (tried to keep at 1000 words, although in a few cases the length might be 1200 words), featured teens as the protagonists who experience an AI related ethics issue in a strong plot, balanced gender in characters, made them open ended so as not to suggest  solutions, since that is part of the conversation that follows).  We also checked the readability of the texts so that they were appropriate for middle and high school students. The stories are open education resources, and can be accessed at our website.  We would love to hear from you if you decide to use the stories or have suggestions for us. 

Thanks, Chelsea! To some degree, the different projects are a result of our team’s collective expertise (which includes computer science, STEM/maker education, and multimodal literacy) and the pandemic. Our original plan was to integrate the AI ethics modules into robotics summer camps, but when COVID-19 restricted our ability to run in-person programs, we had to find other options that would allow students to explore and reflect on these critical issues, even when participating remotely. It was important to our team that we preserve the creative making and design aspects of the project, as we wanted to ensure that the program effectively engaged middle and high school youth (most of whom were already feeling burned out by online school) and provided students with opportunities to reimagine AI and design new solutions that addressed the critical issues we were discussing.

Comics provided a way for students to contribute their own stories in a way that was creative and fun and not intimidating. Pixton EDU was a great tool for this, as it provides all the art assets that students need so that the focus is on the storytelling process, rather than on drawing the comics themselves. It also allowed students to create avatars, which worked well for e-introductions and getting to know each other in a virtual space. We used Adobe Spark (and in the ELA class, WeVideo) to create posters and videos, again because these tools were easy for students to access and use online while still offering a lot of room for creative expression. We selected Juji as a chatbot platform because it aligned with our class goals while not requiring extensive previous technical experience.

At the end of each camp or class, students selected one of the tools that we had worked with to create a final project and they were fairly evenly divided across the three tools, which was interesting. Pixton and Spark were easy to facilitate online, but we did run into some technical glitches with the chatbot software (Juji) initially. Fortunately, the developer was fairly responsive and we had a better experience working with that program in the fall classes. We’re planning to test some additional tools this summer and based on that experience, we hope to add some new resources to our website in the coming months.

Great question, Phillip! One of the key issues we focus on in the class is algorithmic bias. There are numerous examples of biased decision-making by algorithms across different fields, including education, health care, employment/HR, the criminal justice system, etc. AI is playing an increasingly important role in decisions like who is eligible for medical treatment, a mortgage, or parole, and in many cases, these biased algorithms disproportionately impact people of color. In the class, we dive into this in a few different ways. For example, a story that introduces algorithmic bias (in a future dystopian world) is complemented by a video and news stories related to racial and gender bias in facial recognition systems. (If you haven’t already seen it, I recommend viewing the film Coded Bias, now on PBS, which focuses on Joy Buolamwini’s research on bias in facial recognition technologies. In the class, we show a shorter video about her work, called Gender Shades.) Students then work with Google’s Teachable Machine to see how biased training data impacts the accuracy of their models. This fall, we’re hoping to test a new story (currently being reviewed) that addresses algorithmic bias in the criminal justice system.

You can find the stories and some related resources on our website (https://www.colorado.edu/project/imagine-ai/), but it’s still a work in progress. We’ll be adding additional resources to the site over the next few months.

Thanks for the question, Jeremy. We are building on work by researchers who have used pre and post drawings to understand shifts student understanding of science concepts  and another group of researchers using pre post drawings to understand shifts in how students think about social problems. Because of the interdisciplinary nature of AI ethics, both of these lines of inquiry are relevant to our work. 

In addition to the drawings our analysis will look across multiple data sources including pre and post surveys, student created comics, videos and chatbots, chat transcripts from the Zoom sessions, field notes, and interviews. We're in the middle of data analysis now (and we have a lot of data) and have two papers in the works. We're not ready to share our findings just yet, but I'd be happy to contact you when those papers are ready. 

HI Jeremy,

Thanks for your interest and questions!  In addition to Ben's information about our varied data sources, we're also exploring using/adapting existing instruments assessing conceptual change in the core AI concepts that are being developed. Do you have any recommendations for us?  Thanks!

I wish I had a recommendation for you -- but you are in a cutting edge area, so I don't. I guess I'd suggest you look up Shuchi Grover at Stanford and ask her -- she's a leader in assessment of computation thinking concepts.

Great question, Jessica. I think that balance you speak of is very much context specific. I believe that AI ethics is a topic that belongs in a Social Studies class or Language Arts class as much as it does in a CS class. But it can't be taught the same way across disciplines. In a CS class, more time spent experiencing and developing an understanding of the underlying technology makes sense. In a Social Studies class there would likely be an emphasis on the positive and negative influences AI is having on our society (and will increasingly have in the future). In an ELA class I would expect more focus on how AI is influencing the way we communicate and consume and create media. There can't be just one way to teach this topic.

But a complete treatment of the topic in any discipline requires us to move between disciplines. We recognize teachers as experts in their own spaces, but they may need help delving into topics outside their individual comfort zones. This is why we are creating AI ethics modules focused around stories that can be adapted to different contexts, rather a single fixed curriculum. We hope these resources can support, for example, the humanities teacher who has never had to explain algorithms or machine learning to students, or the CS teacher who has never facilitated conversation about racial bias. 

Bridget - This sounds wonderful. You can reach me via email at: dawson@coe.ufl.edu. Have a great weekend -Kara