Login



Champion Sponsor



Donors










Best of MobileHCI 2012

Best Papers

  • An investigation into the use of tactile instructions in snowboarding. Daniel Spelmezan
  • MemReflex: Adaptive Flashcards for Mobile Microlearning. Darren Edge, Stephen Fitchett, Michael Whitney, James Landay

Best Demo

  • Tilt Displays: Display Surfaces with Multi-Axis Tilt and Actuation. Jason Alexander, Andrés Lucero, Sriram Subramanian

Demo honorable mention

  • PoI Poi: Point-of-Interest Poi for Multimodal Tethered Whirling. Michael Cohen

Social Media

The official hashtag for MobileHCI2012 is #mobilehci2012 and you can follow us on Twitter or Facebook.



MobileHCI 2012 Industry Tutorial 1

Designing and Deploying Mobile User Studies In-The-Wild: A Practical Guide

Instructor: Karen Church, Telefónica Research, Barcelona, Spain

The aim of this first invited industrial tutorial is to provide practical instructions and guidelines to designing and deploying mobile field studies, based on the learning outcomes Karen and her colleagues at Telefónica Research in Barcelona, Spain have gathered over the past few years. In this tutorial we will provide an overview about recent research in this domain from an industrial research perspective. We will discuss how to design effective mobile field studies, the importance of mobile prototyping, the impact of various design choices on the study setup and deployment, how to engage participants and how to avoid ethical and legal issues. The tutorial will include practical examples, hand-on design exercises, case studies from a range of mobile field studies as well as a set guidelines on “what not to do” when trying to gather insights from mobile users in-the-wild.

Overview

Mobile phones once deemed as simple communications devices, have now weaved themselves into our everyday life. We use them to communicate and connect with each other, to stay informed and up-to-date, to alleviate boredom, to shop, to navigate, to search, etc. As researchers, practitioners and developers, we’re rarely short of new ideas for engaging mobile services that we would love to design and deploy in the real world. After all the best way to test our ideas is with real users. However, in reality, designing and deploying mobile field studies in-the-wild is a non-trivial task. The mobile space represents a challenging environment even for the most experienced mobile designer, developer, researcher or user experience specialist. The unique characteristics of the mobile space — small screen devices, limited input and interaction capabilities, a diverse range of handset types, operating systems, mobile browsers and sensors, as well as dynamically changing mobile contexts such as location and time — make designing and successfully deploying user studies of mobile applications and/or services inthe- wild a difficult feat. The aim of this tutorial is to provide practical instructions and guidelines based on the learning outcomes we have gathered after carrying out a number live mobile field studies within an industrial research setting, in this case Telefonica Research. The topics covered in the tutorial include participant selection and recruitment, mobile prototyping and development, data capturing and logging, study deployment and participant engagement as well as the ethical concerns associated with conducting such user studies. The tutorial will include practical examples and case studies from a range of mobile field studies as well as a set guidelines on “what not to do” when trying to gather insights into mobile users in-the-wild. Furthermore, the tutorial will follow a highly interactive format with a number of design exercises and a chance to for attendees to discuss the challenges they face in conducting their mobile user studies. We hope to provide some answers / help with these questions.

Objective

This tutorial is intended to provide the attendees with the following: * An understanding of fundamental challenges in conducting live mobile field studies – what makes the mobile space so unique, why are mobile field studies so difficult and the barriers/challenges faced when trying to deploy such studies?

  • An understanding of the various components that need to be considered in deploying field studies in the wild – recruiting users, designing the application, prototyping, development, testing and piloting, logging of data, deployment, data analysis and classification, participant engagement and gathering subjective input.
  • An overview of rapid mobile prototyping – the tools available, the benefits and importance of prototyping and a hands-on guide to a few tools we use within Telefonica Research.
  • An understanding of the range of choices and various constraints for mobile development: natives apps versus mobile web apps and what these choices impact in the design of a user study.
  • An understanding of the complexities of user recruitment for mobile field studies and how the type of application/service, mobile platform, country and even phone type can impact on the demographics and underlying motivations of end-users.
  • The methods available for gathering users insights and which methods to choose in what scenario: quantitative vs. qualitative, large-scale surveys, online diary studies, contextual experience sampling techniques, interviews, mix-method approaches, etc.
  • Some lessons on how to cope with data logging and how to make life a little easier when it comes to data analysis.
  • Some tips on how to keep users engaged, how to communicate clearly with participants and how to outline the right expectations throughout the study.
  • A practical guide of what has worked and not worked for us in Telefonica Research in the past, which will be useful to attendees planning future mobile field studies – from methodologies chosen, choice of mobile development platforms, to interacting with users and analysing very large datasets.
  • Hands-on experience of designing a mobile field study: based on a group exercise to be conducted as part of the tutorial.
  • Answers to perhaps difficult questions – attendees will be given the opportunity to present some of the challenges they face in conducting field studies and will get feedback from other attendees and the instructor on how to overcome such difficulties. The goal is that the attendees will come away with some starting points to help them overcome the challenges they face in conducting mobile field studies.
  • Aside from scientific publications, a listing of non-academic resources related to mobile design, development and mobile user experience will be provided. This listing will include online articles, blogs and references to key influencers in the mobile space thus providing a more complete view of the resources and work being conducted in this space.

Tutorial Structure

Throughout the tutorial, examples will be taken from real studies of mobile users conducted by Telefonica Research over the past 4 years. The tutorial will consist of the following parts:

  • Introduction

The first part of the tutorial introduces the theme and goals. Attendees will be asked about their own background and what they would like to gain from the tutorial. An overview of the design exercise will be given so that attendees can use the rest of the lecture to think about what elements of designing mobile field studies interest them the most and which parts they would like to focus on in the second hands-on session.

  • What makes the mobile space so unique?

The second part of the tutorial discusses the unique and challenging characteristics of the mobile space. Factors such as mobile devices, platforms, mobile context and culture will be discussed. We will also present the strengths of mobile environments and the benefits of focusing on what we can do well in the mobile space. An overview of conducting field studies in the mobile world will be described.

  • Designing mobile field studies

The third part describes the various components of designing mobile field studies. Emphasis will be placed on setting goals for each user study, the benefits and how-to of rapid mobile prototyping, the study and research methodologies available and how/why to choose a certain methodology in certain situations. We will also provide an overview of mobile development platforms and tools.

  • Deploying mobile field studies

The fourth part of the tutorial will describe the factors involved in deploying a mobile field study. Emphasis will be placed on participant recruitment, timelines for field studies, piloting of mobile applications, problems with data logging and coping with participant engagement. A listing of the common challenges in this phase will be provided along with a set of guidelines of what can be done to avoid big problems from occurring. We will also discuss ethical and legal issues in mobile study deployment.

  • Case studies and practical principles

The fifth part of the tutorial will describe selected case studies, illustrating the principles discussed in the previous parts and how they were applied in real-world mobile field studies carried out by Telefonica Research. Emphasis will be place on the goals of each study, the methodologies chosen and why, the challenges faced before, during and after the study and how these challenges were overcome. Case study examples will include diary studies of mobile web behavior, field studies of social mobile applications and mobile search applications and large-scale, longitudinal studies of mobile information needs that utilize the nature of the mobile space and the unique characteristics of mobile phones to gather a vast amount of information about mobile users. Each case study will describe a listing of what worked and what we would do differently should the studies be conducted again.

  • Conclusions

In part six of the tutorial, the overall conclusions will be presented along with a listing of relevant papers and links to online resources. This part will also present some insights into future trends and novel approaches to conducting mobile field studies.

Who should attend?

We hope that by hosting this tutorial at MobileHCI, we can (1) bridge the gap between research conducted in industry versus academia in this research space and (2) provide a number of practical tips about how to conduct effective mobile field studies reporting not only the successes but also the failures we have experienced in the past. The tutorial will provide the Mobile HCI community with an industry outlook on this field and will help arm the attendees with practical knowledge to improve the way in which they design and conduct live field studies of mobile applications and services. Finally we hope that the tutorial will be a reference source for researchers working within the mobile hci space. The tutorial should be easily understood by anyone who has some background in user experience, hci, mobile computing or some prior interest in mobile human computer interaction.

How the tutorial will be conducted?

The format of this tutorial is 3 hours with 1.5 hour of lecture followed by a hands-on group exercise and discussion. The tutorial will be highly interactive, that is, participants will be encouraged to interact with the instructor at any time during the course. Demonstrations will be shown through the lecture and key lessons will be summarized and illustrated. The hands-on group exercise will involve groups of 3 or 4 attendees designing a mobile field study and addressing common but challenging problems in mobile user experience. The goal is that group members will choose problems they are currently working on or have faced in the past. The outcomes of this group sessions will be drawn by hand on large posters and paper prototyping kits and then presented to the group as a whole and discussed.

Materials

Powerpoint slides will be used along with short snippets of videos, mobile prototypes and design/development tools to enhance the lecture. A number of demos will also be shown. The tutorial notes will include copies of all the tutorial slides along with space so that the attendees can take notes. Tutorial notes will also include an abstract, an agenda, an instructor biography, information about Telefonica Research and an extensive bibliography/listing of online resources. Poster sheets, markers, pens, post-its and paper prototyping kits will be provided for the hands-on exercises. Furthermore a tutorial website will created which will include all the details above as well as the output/outcomes of the tutorial. For example screenshots of posters generated in the group exercises, etc.

Instructor

Karen Church is a Research Scientist with Telefonica Research in Barcelona, Spain. She received her PhD in Computer Science entitled, “A Study of Mobile Internet Usage and Implications for Mobile Search Interfaces” from University College Dublin, Ireland. Her research involves understanding mobile users and their online behaviours with the goal of devising new services that will enrich their experiences. Karen has conducted a diverse range of field studies in mobile environments focusing on mobile user needs, their information access behaviours and how they engage socially. Karen was awarded a Marie Curie Fellowship in 2010. The fellowship involves investigating future mobile information access behaviours and trends. Her current research focus is on social mobile search services.

Telefónica is one of the largest private Telecommunications companies in the world, with over 300 million customers in 25 countries across the globe. Telefónica Research was created in 2006 with the aim of carrying out mid to long-term research studies related to several technological areas of interest to the Telefónica Group. One strand of research relates to mobile computing and mobile HCI where our goal is to learn as much as we can about our mobile customers so that we can design new mobile services that are tailored to their unique needs. This research involves designing, deploying and evaluating mobile applications and services in-the-wild with real mobile customers. We use both quantitative and qualitative methods in an attempt to gain an in-depth understanding about our customers.