Code Hunt challenges players to display their coding skills.

As bridge fanatics and crossword puzzle fans know, intellectual games breed competition, and Code Hunt was no exception. Soon, we were organizing weekend Code Hunt contests for students. Under the banner ofImagine Cup, we advertised the contests worldwide and offered them monthly from September 2014 through April 2015. Interest and involvement ran high.

Adam Żychowski of the Warsaw University of Technology typifies many contestants. “I really love Code Hunt Challenge and every month I wait for it with excitement,” says Żychowski. “In Poland, the challenge starts at 1:00 A.M., so it is very busy night for me.” Microsoft Imagine program lead John Scott Tynes adds, “Code Hunt has been very good for Imagine Cup—it drove thousands of new students to the main competition.” Our next set of challenges has a regional and local flavor, with contests specifically for China and India, as well as universities—starting with the University of Washington. 

Imagine Cup Code Hunt Contest winners and new Puzzle Masters Dawid Bugajewski (Military University of Technology, Warsaw) and Vincent Hsiao (University of Maryland) with Puzzle Master Nigel Horspool in background

Code Hunt has an integrated scoring system, which rewards better-written programs. Top scores attract attention, and one successful player, Alexey Kolesnichenko from ETH Zurich, has recently been hired by the Microsoft Trustworthy Computing group. Other tech companies have contests to find top coders, but Code Hunt is unique in that it mixes the game with real learning, teaching students how to manage perplexing coding problems. But how does one help students who are baffled by a problem? While others have conducted research into building hint systems, most of these systems provide specific, built-in hints—which may not address each of the thousands of players’ different needs.

By contrast, the Code Hunt hint system, built by Microsoft intern Daniel Perelman, takes advantage of the game’s foundation of symbolic execution and data in the cloud. Since we have access to close to a million attempts from our players, we use data mining and program synthesis to generate hints that are relevant to the student’s progress. We thus take advantage of all the solutions in the cloud to assist students working on any solution strategy, including ones the teacher might not have chosen as the reference—or may not even have been aware of. Of course, hints are turned off during contests!

Example of the hint system in action

Believe it or not, scale and success bring their own problems. It is a challenge for us to continually devise new problems. To help us, we invited the top coders themselves to start submitting puzzles, and we now have a cohort of eager young Puzzle Masters. When first approached to be a Puzzle Master, Josua Meier from Karlsruhe University of Technology responded, “Using a gaming approach to teach coding paradigms is a very amazing concept. Making a contribution to this, however small, would be great.”

Faculty and researchers design new puzzles and games for Code Hunt. 
Pictured from left to right: Rishabh Singh (Microsoft Research), Armando Solar-Lezama (MIT), Amey Karkare (University of Kanpur), Alexey Kolesnichenko (ETH Zurich), Alessandro Orso (Georgia Tech), Nikolai Tillmann (Microsoft), and Willem Visser (University of Stellenbosch)

Now, with all the data we have, we can analyze it in order to find out how students learn and improve. In early February at the Code Hunt Workshop, we presented our first results showing that such simple concepts as division can be real obstacles that discourage students from continuing to code. Now, we have publicly released a data set of 13,000 programs on GitHub so that academics can continue to analyze the coding process.

So don’t wait for a contest to come to your school or region. Join the community at one of our upcoming workshops, try Code Hunt on your own, or download the data set and get into some deep analysis of how students learn to code.

—Judith Bishop, Director of Computer Science, Microsoft Research