November 26, 2019

RR 439: Human Powered Rails: Automated Crowdsourcing In Your RoR App with Andrew Glass

44 minutes

Andrew Glass is a Brooklyn based Rubyist operating a small independent devshop called Bang Equals. He has held many ‘enrichment jobs’, including being a ball person at US Open for 5 years, traveling for judging Guinness World Record attempts, and will be a balloon holder in the Macy’s Thanksgiving Day Parade this year. Today the panel is discussing his about his 2018 RailsConf talk, Human Powered Rails: Automated Crowdsourcing In Your Ruby on Rails App. In his talk, he shows the audience how to use Amazon Mechanical Turk. Amazon Mechanical Turk lets you post tasks, set a price point, and then people can go and complete the task. This is often done with tasks that can’t be done with machine learning and to train machine learning algorithms. In his talk he goes into What it is, how it’s used, and how we can use Ruby to automate the process. In his apps, he uses it for lead generation, qualification, enrichment, and some video and photo tagging. More specific uses include recording items from a picture of a shopping list, identifying specific things in a video, categorizing businesses and items, sentiment analysis of text or image. Overall, Mechanical Turk is used for things that machine learning can’t handle yet. The panel discusses some different uses for crowdsourcing and how to submit something to Mechanical Turk. There are multiple ways to ensure accuracy in your surveys, including setting up multiple stages to your task, having more than one person complete your task, and creating a qualified worker pool based on tests to determine their aptitude and skill. The panel discusses some of the controversy surrounding Mechanical Turk, citing an article in the New York Times (see links). The big issue is wages and worker rights. Wages can be very low, and it is ripe for abuse by companies as they could easily refuse all work and withhold pay. It is also important for the companies to give an accurate time estimate for the task and a reasonable reimbursement. Mechanical Turk attracts a variety of people, from people that do it for fun to people to actually do it for a living, so it is vital that companies use the tool responsibly. Andrew talks more about how his app works. His apps are built on RTurk, Turkee, and Mechanical Turk, and he talks about how they work. The tricky part is figuring out the logic for what answers they will accept. Andrew talks about how to get started with Mechanical Turk and how to validate the work you get back. To ensure you get accurate information, he suggest that you make it happy for your users, make the UX simple and usable, and use a lot of formatting in your forms so that you get good information in. They preface their results with an accuracy score to help determine what is true. Andrew talks about where he wants to go from he. His Turking days are behind him, but his days of coordinating the efforts of many using software show promise. Panelists