Human Computation

Human Computation

Instructor: Prof. Haoqi Zhang
Monday: volunteer crowdsourcing: motivation, scale, and quality.
Wednesday: human coordination: design patterns and programming models.

Discussion Questions:

  • When is input agreement useful? when is output agreement useful? consider examples of problems that would be better solved with one vs the other
  • In ClickWorkers, how could the researchers verify that the work was any good?
  • What are the key design take-aways from how eBird evolved over time?
  • What are the various forms of motivations/incentives used in these human computation systems? What makes them good fits for their application’s data collection goals?
  • What are the kinds of tasks that have been solved with human computers that can’t be solved with digital computers?
  • What are the unique challenges of crowd computing over digital computing? How do the programming models suggested in Miller et. al.’s article address these challenges?
  • What is crash and re-run? Why is it useful?
  • Identify the design patterns used for programming humans on Mechanical Turk to produce good quality work. Can you see how these patterns can be useful for other problems? What are their shortcomings?

Readings and Media:



XKCD: Suspicion