Code Hunt: creating a community with a game
Judith Bishop, Director of Computer Science, Microsoft ResearcJudith Bishop, Director of Computer Science, Microsoft Research
MAY 19, 2015 01:52 AM
A+ A A-
Launched a year ago, Code Hunt is a coding game that challenges players first to deduce a hidden problem from clues presented as unit tests and then to write a program to solve it. The game has been enormously successful, attracting more than 150,000 players from around the world and achieving amazing stickiness—99 percent of players return to the game within a week. In the process, it has become much more than an intriguing game. It has inspired Code Hunt competitions around the world and has become a vehicle for identifying top coding talent. It has also established itself as an educational presence, both as a tool for teaching coding and as a resource for analyzing how students learn to code.

Code Hunt challenges players to display their coding skills.
Code Hunt challenges players to display their coding skills.

As bridge fanatics and crossword puzzle fans know, intellectual games breed competition, and Code Hunt was no exception. Soon, we were organizing weekend Code Hunt contests for students. Under the banner ofImagine Cup, we advertised the contests worldwide and offered them monthly from September 2014 through April 2015. Interest and involvement ran high.

Adam Żychowski of the Warsaw University of Technology typifies many contestants. “I really love Code Hunt Challenge and every month I wait for it with excitement,” says Żychowski. “In Poland, the challenge starts at 1:00 A.M., so it is very busy night for me.” Microsoft Imagine program lead John Scott Tynes adds, “Code Hunt has been very good for Imagine Cup—it drove thousands of new students to the main competition.” Our next set of challenges has a regional and local flavor, with contests specifically for China and India, as well as universities—starting with the University of Washington. 

Imagine Cup Code Hunt Contest winners and new Puzzle Masters Dawid Bugajewski and Vincent Hsiao with Puzzle Master Nigel Horspool
Imagine Cup Code Hunt Contest winners and new Puzzle Masters Dawid Bugajewski (Military University of Technology, Warsaw) and Vincent Hsiao (University of Maryland) with Puzzle Master Nigel Horspool in background

Code Hunt has an integrated scoring system, which rewards better-written programs. Top scores attract attention, and one successful player, Alexey Kolesnichenko from ETH Zurich, has recently been hired by the Microsoft Trustworthy Computing group. Other tech companies have contests to find top coders, but Code Hunt is unique in that it mixes the game with real learning, teaching students how to manage perplexing coding problems. But how does one help students who are baffled by a problem? While others have conducted research into building hint systems, most of these systems provide specific, built-in hints—which may not address each of the thousands of players’ different needs.

By contrast, the Code Hunt hint system, built by Microsoft intern Daniel Perelman, takes advantage of the game’s foundation of symbolic execution and data in the cloud. Since we have access to close to a million attempts from our players, we use data mining and program synthesis to generate hints that are relevant to the student’s progress. We thus take advantage of all the solutions in the cloud to assist students working on any solution strategy, including ones the teacher might not have chosen as the reference—or may not even have been aware of. Of course, hints are turned off during contests!

Example of the hint system in actionExample of the hint system in action

Believe it or not, scale and success bring their own problems. It is a challenge for us to continually devise new problems. To help us, we invited the top coders themselves to start submitting puzzles, and we now have a cohort of eager young Puzzle Masters. When first approached to be a Puzzle Master, Josua Meier from Karlsruhe University of Technology responded, “Using a gaming approach to teach coding paradigms is a very amazing concept. Making a contribution to this, however small, would be great.”

Faculty and researchers design new puzzles and games for Code Hunt. Pictured from left to right: Rishabh Singh, Armando Solar-Lezama, Amey Karkare, Alexey Kolesnichenko, Alessandro Orso, Nikolai Tillmann, and Willem Visser
Faculty and researchers design new puzzles and games for Code Hunt. 
Pictured from left to right: Rishabh Singh (Microsoft Research), Armando Solar-Lezama (MIT), Amey Karkare (University of Kanpur), Alexey Kolesnichenko (ETH Zurich), Alessandro Orso (Georgia Tech), Nikolai Tillmann (Microsoft), and Willem Visser (University of Stellenbosch)

Now, with all the data we have, we can analyze it in order to find out how students learn and improve. In early February at the Code Hunt Workshop, we presented our first results showing that such simple concepts as division can be real obstacles that discourage students from continuing to code. Now, we have publicly released a data set of 13,000 programs on GitHub so that academics can continue to analyze the coding process.

So don’t wait for a contest to come to your school or region. Join the community at one of our upcoming workshopstry Code Hunt on your own, or download the data set and get into some deep analysis of how students learn to code.

Judith Bishop, Director of Computer Science, Microsoft Research

[%= name %]
[%= createDate %]
[%= comment %]
Share this:
Please login to enter a comment:

Computing Now Blogs
Business Intelligence
by Keith Peterson
Cloud Computing
A Cloud Blog: by Irena Bojanova
The Clear Cloud: by STC Cloud Computing
Computing Careers: by Lori Cameron
Display Technologies
Enterprise Solutions
Enterprise Thinking: by Josh Greenbaum
Healthcare Technologies
The Doctor Is In: Dr. Keith W. Vrbicky
Heterogeneous Systems
Hot Topics
NealNotes: by Neal Leavitt
Industry Trends
The Robotics Report: by Jeff Debrosse
Internet Of Things
Sensing IoT: by Irena Bojanova