Great questions! Part of our initial experiments are to evaluate the usability of the game from differing perspectives. The first is for ensuring that our introductory tutorials are sufficient to orient unfamiliar players. Currently, we are evaluating a guide that introduces many aspects of the game to the player, while also setting up the story to drive exploration. Second, we are also trying to understand how the gameplay translates to understanding concepts related to data collection, cleaning, and understanding with these early tutorials. 

Our evaluation uses a phased approach over the next few years. Right now we are conducting cognitive walkthroughs with early phases of the game, but these will turn into observed gameplay sessions in controlled environments, then ramp up to the usage of the game in a classroom, and eventually across multiple schools. Each phase provides iterative points to address deficiencies in the gameplay and how it influences understanding of various CSTA standards.

We also have feedback from an educator advisory panel that is helping us to ensure that the transition to the classroom is as seamless as possible. These sessions stressed how important it is to have the game connect with an LMS so that teachers can launch worlds, assignments, and evaluate progress directly from the LMS dashboard. This also helps mitigate teacher unfamiliarity with the Minecraft setup (or at least, that is the hypothesis).

Thanks for your interest!  Would love to talk more about these parts fo the research.

Thanks for this response to Dalila's question. I am also interested in the transition to classrooms and think the LMS integration could definitely help teaches access the activity more easily. When it comes to students, I wonder how you plan to onboard students who might not have exposure to Minecraft. How will scaffolding work for the gameplay sessions? I used to teach Middle School students and know that many would be very interested in this type of exploration in Minecraft - but I'm wondering about the ones who would not be as excited about this activity and supporting students with IEPs as well. 

Also great questions. For now, we have just a simple introductory tutorial to Minecraft. However, we plan on expanding this so that there is scaffolded introductions in the game (including the game mechanics like moving around, accessing a menu, explanation of menus, etc.). This lengthens the amount of time that some players need for the introduction, but is absolutely necessary to accommodate the differing experience levels. We also anticipate having a "skip this tutorial" button for more experienced players--but the assumption will be that they are not familiar. Not everyone will be excited by Minecraft, but we hope to generate some level of interest for everyone. 

I haven't thought enough about students with IEPs. That's a great point that we do not yet have a solution for. 

Dear Jodi, Thank you for watching our video and asking these great questions! I wanted to chime in about your question related to the standards. We are decomposing CT practices into pre-requisite and targeted knowledge and skills, but we also see them as intertwined and their development as iterative, as you noted. For example, in one CT standard focused on data interpretation, we decomposed the standard into subcomponents that also relate and build on other CT standards. However, we decided to decompose the standards so we could build in opportunities for students to demonstrate their learning as well as mastery of the standards.    

cool - we found that the overlap in practices made it very difficult to validate assessment items of each practice. We could get an aggregate measure of CT but haven't yet figured out how to measure the practices independently. Hope you have more luck! :)

Dear Valerie, Thanks for your interest in our project and for sharing these thoughts and questions. Regarding your question about gameplay and learning, we found that we needed a bridge between the CT standards and the gameplay mechanics, and that is what lead us to the companion application, called the Minecraft Factory Planner. When we started decomposing the CT standards and aligning them with components of gameplay in Minecraft, we noticed that there were a number of CT standards that could not be addressed (meaning that students would not have an opportunity to learn them) with just Minecraft alone. We needed to make their processing and actions visible to them. That is how we decided we needed the company application. This allows students (and teachers) to develop understanding and display their learning. Thanks again for your thoughtful questions. 

Hi Valerie! So great to hear from you. Thanks for checking out our video. Your comments are so thoughtful. I want to add on to Leanne's comment and focus on your second point about instrumental and conceptual learning. Part of our game development phase included decomposing components of the standards and the game separately and then integrating. I learned that games are made up of loops including the primary, secondary, and tertiary loops. The primary loops are rote behaviors (e.g., chopping trees, which in our game relates to collecting data) and are similar to instrumental learning; whereas the tertiary loops are high-level conceptual learning (e.g., optimizing the number of trees chopped using computational tools and comparing solution methods with other students in game). The game allows for us to integrate these different types of learning experiences. Thanks for commenting!

Hi Suzanne. Thank you so much for commenting on our video. Your questions are very thoughtful. We are aligning the game experience to several middle grades CSTA standards, which we identified in collaboration with our Educator Advisory Panel. Our goal is for the game to have broad applicability in mathematics, science and computer science courses. The game interfaces with Canvas so teachers will be able to view a summary of students' progress in game without entering Minecraft themselves. Teachers will be able to differentiate by assigning new or additional practice tasks based on evidence of learning and mastery. As former teachers, we completely understand the levels of overwork teachers experience, and we hope that Canvas will make it easy for them to use and monitor students' progress in game (without actually needing to enter the game). We recognize that we will need to provide support, which we would love to hear more from you on how this support might look, and we are hoping that use of the game will require minimal PD for teachers. Thank you so much for commenting!

Thank you so much, Ning! We appreciate you watching our video and commenting.

Hi Ning, SMU is lucky to have a top notch game design program. This STEM+C project is a collaboration between the following: Guildhall (Game Design), Lyle (Engineering and Computer Science), and Simmons (Education). We are also collaborating with Mineplex. We are currently in our second year of the project.

To add to Beth's response, we don't use Unity for this. Actually, we run a version of Minecraft directly on one of their EDU servers. We are able to access the data of a the server through queries and even manipulate the inventory and spawning in the world through API calls. We can create the game scenario by scenario, so the process is iterative in development.  

Hi Chad, thank you so much for watching our video and commenting. We were drawn to Minecraft for exactly the reasons you mention. Many students are already interested, and the game allows for "stealth" learning and an immersive state of flow. We are currently engaged in cognitive walkthroughs with novice players. I think for this reason, we have not seen the issues that you mention where players revert back to usual patterns. This information is helpful as we move forward this summer with cognitive walkthroughs with players who have more fluency with the game. We will keep it in mind that we might want to reach out you if we experience similar challenges to brainstorm solutions for keeping students on track. I can see why building a home with a massive swimming pool and big screen tv would be enticing! 

Thanks Chad! We agree. Right now we have been controlling the inventory of players pretty carefully with the game design. So at early stages, they do not have the materials or capability to simply play the game like they might be accustomed to at home (or in creative mode). This gives us some control over off-topic behavior, but its not complete control. This might become more of an issue for later stages of the storyline where players have more freedom. Any guidance or tips you have would be very helpful!

Thanks for watching our video, Blake. We appreciate your interest in our project.