Champion Sponsor


Best of MobileHCI 2012

Best Papers

  • An investigation into the use of tactile instructions in snowboarding. Daniel Spelmezan
  • MemReflex: Adaptive Flashcards for Mobile Microlearning. Darren Edge, Stephen Fitchett, Michael Whitney, James Landay

Best Demo

Demo honorable mention

  • PoI Poi: Point-of-Interest Poi for Multimodal Tethered Whirling. Michael Cohen

Social Media

The official hashtag for MobileHCI2012 is #mobilehci2012 and you can follow us on Twitter or Facebook.


The preliminary MobileHCI 2012 program is detailed below. This will be updated as new material is added to the schedule.

Full paper talks should be 20 minutes long including time for questions. Short paper talks should be 10 minutes long.

Registration will be open at 8am daily.

Friday 21st September

09:00 - 17:00 Workshops
09:00 - 17:00 Tutorials
09:00 - 17:00 Doctoral Consortium

Saturday 22nd September

09:00 - 09:30 Welcome and Opening Announcements
09:30 - 10:30 Opening Keynote
10:30 - 11:10 Posters and Demos Break
11:10 - 12:20 PS1 - Patterns of Use PS2 - Touch Input
12:20 - 14:00 Lunch
14:00 - 15:15 Panel on Longitudinal Review of Mobile HCI research methods PS3 - Off and Around the Screen
15:15 - 15:50 Posters and Demos Break
15:50 - 17:00 PS4 - Trust and Privacy PS5 - Body Space and Motion
17:30 - end of day Reception

Sunday 23rd September

09:10 - 10:30 PS6 - Understanding Use PS7 - Mobile Augmented Reality
10:30 - 11:15 Posters and Demos Break
11:15 - 12:15 PS8 - Understanding Touch PS9 - Multiplexing
12:15 - 14:00 Lunch
14:00 - 15:15 Panel on Relationships between Industry and Academia PS10 - Non-visual Interaction
15:15 - 15:45 Posters and Demos Break
15:45 - 17:00 Space Challenges and Demos
17:30 - end Banquet

Monday 24th September

09:10 - 10:30 PS11 - Location PS12 - Collaboration and Sharing
10:30 - 11:10 Posters and Demos Break
11:15 - 12:15 PS13 - Learning and Training Posters and Demos
12:15 - 14:00 Lunch
14:00 - 15:15 Closing Keynote
15:15 - 15:45 Other announcements
15:45 - 16:15 MobileHCI 2013

Opening Keynote

Katherine Isbister

How to Stop Being a Buzzkill: Designing Yamove!, a mobile tech mash-up to truly augment social play

Most technology-supported dance games result in gameplay that looks very different than what we know as social dancing. For that matter, most 'social' games involve a lot of solo staring at screens. Our lab re-examined the role of technology in supporting the dance experience, working with indie game developers and dancers to understand how to truly augment instead of override the joyful social and physical qualities of dancing together. Along the way, we learned some valuable lessons about how to reframe the conceptualization and development of mobile apps meant to augment everyday social experience. In this talk, I'll share these insights, toward opening your eyes as you tackle these kinds of design and development challenges in next-generation mobile applications.

Dr Katherine Isbister is an associate professor jointly appointed between the NYU-Poly Computer Science Department and the NYU Game Center. Isbister is Research Director of the Game Innovation Lab at NYU-Poly, and an investigator in the NYU Games for Learning Institute. Her research in Human Computer Interaction focuses on enhancing the social and emotional range of everyday interaction with technology. Isbister’s book on game character design — Better Game Characters by Design: A Psychological Approach — was nominated for a Game Developer Magazine Frontline award. Her edited volume, Game Usability, brings together best practices in game playtesting and user research. In 1999, Isbister was selected as one of MIT Technology Review's Young Innovators, and she received a Humboldt Experienced Researcher award in 2011. Her research has been covered in Wired, Scientific American, and on NPR and BBC radio.

PS1 - Patterns of Use

Understanding Tablet Use: A Multi-Method Exploration
Hendrik Mueller, Jenny Gove, John Webb

Tablet ownership has grown rapidly over the last year. While market research surveys have helped us understand the demographics of tablet ownership and provided early insights into usage, there is little comprehensive research available. This paper describes a multi-method research effort that employed written and video diaries, in-home interviews, and contextual inquiry observations to learn about tablet use across three locations in the US. Our research provides an in-depth picture of frequent tablet activities (e.g., checking emails, playing games, social networking), locations of use (e.g., couch, bed, table), and contextual factors (e.g., watching TV, eating, cooking). It also contributes an understanding of why and how people choose to use tablets. Popular activities for tablet use, such as media consumption, shopping, cooking, and productivity are also explored. These multi-dimensional findings provide design implications and opportunities for enriching the tablet experience, and agendas for future research.

Exploring iPhone Usage: The Influence of Socioeconomic Differences on Smartphone Adoption, Usage and Usability
Ahmad Rahmati, Chad Tossell, Clayton Shepard, Lin Zhong, Philip Kortum, Mian Dong, Zhen Wang

Previous studies have found that smartphone users differ by orders of magnitude. We explore this variability to understand how users install and use native applications in ecologically-valid environments. A quasi-experimental approach is applied to compare how users in different socioeconomic status (SES) groups adopt new smartphone technology along with how applications are installed and used. We present a longitudinal study of 34 iPhone 3GS users. 24 of these participants were chosen from two carefully selected SES groups who were otherwise similar and balanced. Usage data collected through an in-device programmable logger, as well as several structured interviews, identify similarities, differences, and trends, and highlight systematic differences in smartphone usage. A group of 10 lower SES participants were later recruited and confirm the influence of socioeconomic diversity on device usage. Among our findings are that a large number of applications were uninstalled, lower SES groups spent more money on applications and installed more applications overall, and the lowest SES group perceived their iPhone usability poorly in comparison to the other groups. We further discuss the primary reasons behind this low score, and suggest design implications to better support users across every SES bracket.

A Note Paper on Note-Taking: Understanding Annotations of Mobile Phone Calls
Juan Pablo Carrascal Ruiz, Rodrigo de Oliveira , Mauro Cherubini

Note-taking has been largely studied in contexts of work meetings. However, often people need to remember information exchanged in informal situations, such as during mobile phone conversations. In this paper we present a study conducted with 65 subjects who had their phone calls semi-automatically transcribed for later annotation. Analysis of the 639 calls and the subjects' annotation behavior revealed that phone recall is indeed a relevant user need. Furthermore, identifying standard patterns in phone calls provide better indicators of annotation than variables related to the callers' profile, context of calls, or quality of service. Length and importance of calls were also relevant indicators of note-taking. Our findings suggest implications for the design of mobile phone annotation tools.

Patterns of Usage and Context in Interaction with Communication Support applications in Mobile Devices
Andreas Komninos, Athanassios Plessas, Vassilios Stefanis, John Garofalakis

Contact lists are one of the most frequently used applications on mobile devices. Users are reluctant to delete or remove contacts from their repositories and as modern smartphones provide an unlimited contact list storage space, these become increasingly large, sometimes measuring several hundred entries. In this paper we present our findings from two experiments with user-subjective and quantitative data concerning the use of mobile Contact Lists. We also propose a context oriented framework to aid the speed and efficacy of the information seeking and retrieval process during the use of the Contact List application.

PS2 - Touch Input

An Exploration of Inadvertent Variations in Mobile Pressure Input
Craig Stewart, Eve Hoggan, Laura Haverinen, Hugues Salamin, Giulio Jacucci

This paper reports the results of an exploratory study into inadvertent grip pressure changes on mobile devices with a focus on the differences between static lab-based and mo-bile walking environments. The aim of this research is to inform the design of more robust pressure input techniques that can accommodate dynamic mobile usage. The results of the experiment show that there are significant differences in grip pressure in static and walking conditions with high levels of pressure variation in both. By combining the pressure data with accelerometer data, we show that user movement is closely related to grip pressure.

The Fat Thumb: Using the Thumb's Contact Size for Single-Handed Mobile Interaction
Sebastian Boring, David Ledo, Xiang 'Anthony' Chen, Nicolai Marquardt, Anthony Tang, Saul Greenberg

Modern mobile devices allow a rich set of multi-finger interactions that combine modes into a single fluid act, for example, one finger for panning blending into a two-finger pinch gesture for zooming. Such gestures require the use of both hands: one holding the device while the other is interacting. While on the go, however, only one hand may be available to both hold the device and interact with it. This mostly limits interaction to a single-touch (i.e., the thumb), forcing users to switch between input modes explicitly. In this paper, we contribute the Fat Thumb interaction technique, which uses the thumb's contact size as a form of simulated pressure. This adds a degree of freedom, which can be used, for example, to integrate panning and zooming into a single interaction. Contact size determines the mode (i.e., panning with a small size, zooming with a large one), while thumb movement performs the selected mode. We discuss nuances of the Fat Thumb based on the thumb's limited operational range and motor skills when that hand holds the device. We compared Fat Thumb to three alternative techniques, where people had to precisely pan and zoom to a predefined region on a map and found that the Fat Thumb technique compared well to existing techniques.

The Hold-and-Move Pattern for Multi-touch Interfaces
Alexander Kulik, Jan Dittrich, Bernd Froehlich

We present the two finger hold-and-move pattern as an alternative to the disruptive long-tap which utilizes dwell times for switching from panning into object dragging mode in touchinterfaces. We make use of a second finger for object selection and manipulation while the regular workspace panning is performed by the first finger. Since object dragging and workspace panning can be done simultaneously, the cumbersome and hard-to-control workspace autoscrolling function is no longer needed when dragging an object beyond the currently visible viewport. Single finger panning and pinch-zoom still work as expected. A user study revealed that the use of the hold-and-move input sequence enables faster object dragging than the conventional dwell time approach and that it is preferred by most users.

Brush-and-Drag: A Multi-touch Interface For Browsing Digital Photo Collections
Seon Joo Kim, Hongwei Ng, Stefan Winkler, Peng Song, Chi-Wing Fu

Due to the convenience of taking pictures with various digital cameras and mobile devices, we often end up with multiple shots of the same scene with only slight variations. To enhance photo triaging, which is a very common photowork activity, we propose an effective and easy-to-use brush-and-drag interface that allows the user to interactively explore and compare photos within a broader scene context. First, we brush to mark an area of interest on a photo with our finger(s); our tailored segmentation engine automatically determines corresponding image elements among the photos. Then, we can drag the segmented elements from different photos across the screen to explore them simultaneously, and further perform simple finger gestures to interactively rank photos, select favorites for sharing, or to remove unwanted ones. This novel interaction method was implemented on a consumer-level tablet computer and demonstrated to offer effective interactions in a user study.

Panel 1

PS3 - Off and Around the Screen

EdgeSplit: A Technique to Facilitate the Selection of Off-Screen Objects
Zahid Hossain, Khalad Hasan, Hai-Ning Liang, Pourang Irani

Devices with small viewports (e.g., smartphones or GPS) result in interfaces where objects of interest can easily reside outside the view, into off-screen space. Researchers have addressed this challenge and have proposed visual cues to assist users in perceptually locating off-screen objects. However, little attention has been placed on methods for selecting the objects. Current designs of off-screen cues can result in overlaps that can make it difficult to use the cues as handles through which users can select the off-screen objects they represent. In this paper, we present EdgeSplit, a technique that facilitates both the visualization and selection of off-screen objects on small devices. EdgeSplit exploits the space around the device's borders to display proxies of off-screen objects and then partitions the border regions to allow for non-overlapping areas that make selection of objects easier. We present an effective algorithm that provides such partitioning and demonstrate the effectiveness of EdgeSplit for selecting off-screen objects.

Around device interaction for Multiscale Navigation (Nominated)
Brett Jones, Rajinder Sodhi, David Forsyth, Brian Bailey, Giuliano Maciocci

In the future mobile devices will have depth sensors. In this paper we study the design space of free-space interactions afforded by mobile depth sensors. Such interactions will have a greater working volume, more fluid control and avoid screen occlusion effects intrinsic to touch screens. This work contributes the first study to show that mobile free-space interactions can be as good as touch. We are also the first to analyze interaction volume usage, resulting in strong implications for how sensors should be placed on mobile devices. We describe a user study evaluating mobile free-space navigation techniques and the impacts of sensor orientation on user experience. Finally, we discuss guidelines for future mobile free-space interaction techniques and sensor design.

Dynamic visualization of large numbers of off-screen objects on mobile devices: an experimental comparison of Wedge and Overview+Detail
Stefano Burigat, Luca Chittaro, Andrea Vianello

Overview+Detail and Wedge have been proposed in the literature as effective approaches to resolve the off-screen objects problem on mobile devices. However, they have been studied with a small number of off-screen objects and in the majority of studies with static scenarios, in which users did not have to perform any navigation activity. In this paper, we propose improvements to Wedge and Overview+Detail specifically aimed at simplifying their use in dynamic scenarios that involve large numbers of off-screen objects. We compare the effectiveness of the two approaches in the considered scenario with a user study, whose results show that Overview+Detail allows users to be faster in searching for off-screen objects and more accurate in estimating their location.

How to Position the Cursor? An Exploration of Absolute and Relative Cursor Positioning for Back-of-Device Input
Khalad Hasan, Xing Dong Yang, Hai-Ning Liang, Pourang Irani

Observational studies indicate that most people use one hand to interact with their mobile devices. Interaction on the back-of-devices (BoD) has been proposed to enhance one-handed input for various tasks, including selection and gesturing. However, we do not possess a good understanding of some fundamental issues related to one-handed BoD input. In this paper, we attempt to fill this gap by conducting three studies. The first study explores suitable selection techniques; the second study investigates the performance and suitability of the two main modes of cursor movement: Relative and Absolute; and the last study examines solutions to the problem of reaching the lower part of the device. In this paper, we report the findings of these three studies and based on these findings provide guidelines for designing BoD interactions for mobile devices.

PS4 - Trust and Privacy

Soft Trust and mCommerce Shopping Behaviours
Serena Hillman, Carman Neustaedter, John Bowes, Alissa Antle

Recently, there has been widespread growth of shopping and buying on mobile devices, termed mCommerce. With this comes a need to understand how to best design experiences for mobile shopping. To help address this, we conducted a diary and interview study with mCommerce shoppers who have already adopted the technology and shop on their mobile devices regularly. Our study explores typical mCommerce routines and behaviours along with issues of soft trust, given its long-term concern for eCommerce. Our results describe spontaneous purchasing and routine shopping behaviours where people gravitate to their mobile device even if a computer is nearby. We found that participants faced few trust issues because they had limited access to unknown companies. In addition, app marketplaces and recommendations from friends offered a form of brand protection. These findings suggest that companies can decrease trust issues by tying mCommerce designs to friend networks and known marketplaces. The caveat for shoppers, however, is that they can be easily lured into a potentially false sense of trust.

Technology Enabled Social Contribution for Public Safety using M-Urgency
Shivsubramani Krishnamoorthy, Ashok Agrawala

M-Urgency is a public safety system that (1) redefines how emergency calls are made to a Public Safety Answering Point (PSAP) like the 911 system and (2) is designed to be context-aware of the situation in which it is used. M-Urgency enables mobile users to stream live audio and video from their devices to local PSAP along with the audio stream, the real time location and the relevant context information, enabling appropriate and prompt service. This paper presents a new feature, incorporated in M-Urgency, that enables social contribution whereby users in the vicinity of an emergency event can help operation conducted by the emergency personnel through verbal/visual information useful to them or by providing assistance. Our experiments show a very positive response from the participant to the capabilities of our system. In this paper, we also discuss the social implications such as privacy and security for this system.

“There Are No Secrets Here!”: Professional Stakeholders' Views on the Use of GPS for Tracking Dementia Patients
Yngve Dahl, Kristine Holbø

This paper investigates the attitudes of professional stakeholders involved in dementia care to GPS tracking of dementia patients. Data were gathered via focus groups that met in the context of a field experiment in which patients' spatial activities were tracked using GPS. Four main topics emerged: (1) different perspectives on the purpose of the measure; (2) privacy concerns and underlying premises for employing GPS technology in professional care, including; (3) knowledge about patients; and (4) routines for use. Our findings highlight the need to consider carefully which aspects of dementia patients' movements a GPS tracking system should provide to care workers, and how positioning information should be presented. We found that the level of detail required is intimately linked to the purpose of use. Positioning data that were regarded as being irrelevant for the immediate situation could be perceived as violations of patient privacy and damaging for the system's efficiency.

But I don't trust my friends. Ecofriends - A mobile application for reflective grocery shopping
Jakob Tholander, Anna Ståhl, Mattias Jacobsson, Lisen Schultz, Sara Borgström, Maria Normark, Carolina Johansson, Elsa Kosmack-Vaara

The Ecofriends application was designed to engage people in reflecting on their everyday grocery shopping from social and ecological perspectives. Ecofriends portrays the season of various groceries as socially constructed, emphasizing subjective dimensions of what it means for a product to be in season, rather than attempting to communicate it as an established fact. It provides the user with unexpected information (news, weather, blog posts and tweets) about the place where the product was grown, and visualizes how the product's popularity shifts throughout the year, among the user's friends, among chefs and other food experts, and among the general public. A number of key findings from users' first encounters with the system are presented. In particular we discuss aspects of trust, information fragments as catalysts, and how several of the participants were provoked by the system's portrayal of season.

PS5 - Body Space and Motion

A Recognition Safety Net: Bi-Level Threshold Recognition for Mobile Motion Gestures
Matei Negulescu, Jaime Ruiz, Edward Lank

Designers of motion gestures for mobile devices face the difficult challenge of building a recognizer that can separate gestural input from motion noise. A threshold value is often used to classify motion and effectively balances the rates of false positives and false negatives. We present a bi-level threshold recognizer that is built to lower the rate of recognition failures by accepting either a tightly thresholded gesture or two consecutive possible gestures recognized by a looser model. We evaluate the technique with a pilot study in order to gauge effectiveness as a recognition safety net for users who have difficulty activating a motion gesture. Lastly, we suggest the use of bi-level thresholding to scaffold learning of gestures.

Tilt Displays: Designing Display Surfaces with Multi-axis Tilting and Actuation
Jason Alexander, Andrés Lucero, Sriram Subramanian

We present a new type of actuatable display, called Tilt Displays, that provide visual feedback combined with multi-axis tilting and vertical actuation. Their ability to physically mutate provides users with an additional information channel that facilitates a range of new applications including collaboration and tangible entertainment while enhancing familiar applications such as terrain modelling by allowing 3D scenes to be rendered in a physical-3D manner. Through a mobile 3×3 custom built prototype, we examine the design space around Tilt Displays, categorise output modalities and conduct two user studies. The first, an exploratory study examines users' initial impressions of Tilt Displays and probes potential interactions and uses. The second takes a quantitative approach to understand interaction possibilities with such displays, resulting in the production of two user-defined gesture sets: one for manipulating the surface of the Tilt Display, the second for conducting everyday interactions.

m+pSpaces: Virtual workspaces in the spatially-aware mobile environment
Jessica Cauchard, Markus Löchtefeld, Mike Fraser, Antonio Krüger, Sriram Subramanian

We introduce spatially-aware virtual workspaces for the mobile environment. The notion of virtual workspaces was initially conceived to alleviate mental workload in desktop environments with limited display real-estate. Using spatial properties of mobile devices, we translate this approach and illustrate that mobile virtual workspaces greatly improve task performance for mobile devices. In a first study, we compare our spatially-aware prototype (mSpaces) to existing context switching methods for navigating amongst multiple tasks in the mobile environment. We show that users are faster, make more accurate decisions and require less mental and physical effort when using spatially-aware prototypes. We furthermore prototype pSpaces and m+pSpaces, two spatially-aware systems equipped with pico-projectors as auxiliary displays to provide dual-display capability to the handheld device. A final study reveals advantages of each of the different configurations and functionalities when comparing all three prototypes. Drawing on these findings, we identify design considerations to create, manipulate and manage spatially-aware virtual workspaces in the mobile environment.

Extending a Mobile Device's Interaction Space through Body-Centric Interaction
Xiang 'Anthony' Chen, Nicolai Marquardt, Anthony Tang, Sebastian Boring, Saul Greenberg

Modern mobile devices rely on the screen as a primary input modality. Yet the small screen real-estate limits interaction possibilities, motivating researchers to explore alternate input techniques. Within this arena, our goal is to develop Body-Centric Interaction with Mobile Devices: a class of interaction techniques that allow a person to position her mobile device to navigate and manipulate digital contents anchored in the space on and around the body. To achieve this goal, we explore such interaction in a bottom-up path of prototypes and implementations. From our experiences, as well as by examining related work, we discuss and contribute three recurring themes that characterize how these interactions can be realized. We illustrate how these themes can inform the design of Body-Centric Interactions by applying them to the design of a novel mobile browser application. This further contributes a class of mobile input techniques where interactions are not limited to small screen input, but instead leverage the person's movement of the device on and around their body to enable novel interactions.

PS6 - Understanding Use

From Cameraphone to Mediaphone: Re-thinking Smartphones for Rural Developing Regions
David Frohlich, Simon Robinson, Kristen Eglinton, Matt Jones, Elina Vartiainen

Technology tends to be shaped by its socio-cultural context of use, either explicitly through design-oriented research in its development or implicitly through feedback from the market into which it is sold. In this paper we consider the current and future use of cameraphones in the context of rural South Africa, where many people do not have access to the latest models and ICT infrastructure is poor. We report a new study of cameraphone use in this setting, and the design and testing of a novel application for creating rich multimedia narratives and materials. We argue for a shift towards high-quality multimedia computing on mobile platforms in this region, and indicate a number of ways in which this might be used for both personal and community benefit.

AutoWeb: Automatic Classification of Mobile Web Pages for Revisitation
Jie Liu, Yuanchun Shi

Revisitation in mobile Web browsers takes more time than that in desktop browsers due to the limits of mobile phones. In this paper, we propose AutoWeb, a novel approach to speed up revisitation in mobile Web browsing. In AutoWeb, opened Web pages are automatically classified into different groups based on Web content. Users can quickly revisit an intended opened Web page by narrowing down search scope into a group of pages that share the same topic. We evaluated the classification accuracy and computing resource consumption. A controlled experiment and a semi-open experiment were conducted to investigate revisitation performance in three tasks. Results show AutoWeb can save significant time for revisitation by 29.5%, especially for long time Web browsing, and that it improves overall mobile Web revisitation experience.

Enough Power to Move: Dimensions for Representing Energy Availability
Anders Lundström, Cristian Bogdan

Energy and design of energy-feedback are becoming increasingly important in the mobile HCI community. Our application area concerns electric vehicles, we thus depart from home and workplace appliances and address range and energy anxiety caused by short driving distance capabilities and long charging times in mobile settings. Meanwhile some research has been done on energy management of mobile devices, less has been done on mobility devices like electric vehicles. We explore this topic by letting conventional fuel car drivers reflect on their current driving habits through an exploration tool that we developed. Our results demonstrate three dimensions related to energy availability to consider for design of energy dependent mobility devices and explain how these dimensions could be utilize in our design through energy visualizations. With this we contributed not only by demonstrating important aspects of energy availability and mobility, but also through opening up for new interesting possibilities and inquires in our and possibly other domains.

PS7 - Mobile Augmented Reality

Revisiting Peephole Pointing: A Study of Target Acquisition with a Handheld Projector
Bonifaz Kaufmann, David Ahlström

Peephole pointing is a promising interaction technique for large workspaces that contain more information than can be appropriately displayed on a single screen. In peephole pointing a window to the virtual workspace is moved in space to reveal additional content. In 2008, two different models for peephole pointing were discussed. Cao, Li and Balakrishnan proposed a two-component model, whereas Rohs and Oulasvirta investigated a similar model, but concluded that Fitts' law is sufficient for predicting peephole pointing performance. We present a user study performed with a handheld projector showing that Cao et al.'s model only outperforms Fitts' law in prediction accuracy when different peephole sizes are used and users have no prior knowledge of target location. Nevertheless, Fitts' law succeeds under the conditions most likely to occur. Additionally, we show that target overshooting is a key characteristic of peephole pointing and present the implementation of an orientation aware handheld projector that enables peephole interaction in unconstrained environments.

Mobile Augmented Reality: Exploring design and Prototyping Techniques
Marco de Sa, Elizabeth F. Churchill

As mobile devices are enhanced with more sensors, powerful embedded cameras, and increased processing power and features, new user experiences become possible. A good example is the recent emergence of Augmented Reality (AR) applications that are designed for personal use while people are on-the-go. However, designing effective and usable AR experiences for mobile devices poses challenges for the design process. In this paper we outline reasons why simulating a compelling, mobile AR experience with sufficient veracity for effective formative design is a challenge, and present our work on prototyping and evaluation techniques for mobile AR. An experiment within the context of an ongoing design project (Friend Radar) is presented along with resulting findings and guidelines. We reflect on the benefits and drawbacks of low, mixed and high fidelity prototypes for mobile AR by framing them into a set of analytic categories extracted from the existing literature on prototyping and design.

Playing it Real: Magic Lens and Static Peephole Interfaces for Games in a Public Space
Jens Grubert, Ann Morrison, Helmut Munz, Gerhard Reitmayr

Magic lens and static peephole interfaces are used in numerous consumer mobile phone applications such as Augmented Reality browsers, games or digital map applications in a variety of contexts including public spaces. Interface performance has been evaluated for various interaction tasks involving spatial relationships in a scene. However, interface usage outside laboratory conditions has not been considered in depth in the evaluation of these interfaces. We present findings about the usage of magic lens and static peephole interfaces for playing a find-and-select game in a public space and report on the reactions of the public audience to participants' interactions. Contrary to our expectations participants favored the magic lens over a static peephole interface despite tracking errors, fatigue and potentially conspicuous gestures. Most passersby did not pay attention to the participants and vice versa. A comparative laboratory experiment revealed only few differences in system usage.

Integrating the Physical Environment into Mobile Remote Collaboration (Nominated)
Steffen Gauglitz, Cha Lee, Matthew Turk, Tobias Höllerer

While largely successful when dealing with verbal communication and digital data, current remote collaboration systems hit severe limitations when real-world objects are involved. In this paper, we describe a framework and prototype implementation for unobtrusive mobile remote collaboration on tasks that involve the physical environment. Our system leverages model-free, markerless visual tracking to facilitate decoupled, live updated views of the environment and world-stabilized telepointers while supporting a moving camera and unknown, unprepared environments. In order to evaluate our concept and prototype, we conducted a user study with 48 participants in which a remote expert instructed a local user to operate a mock-up airplane cockpit. Users performed significantly better with our prototype (40.8 tasks completed on average) as well as with static telepointers (37.3) than without telepointers (28.9). 79% of the users preferred our prototype despite noticeably imperfect tracking.

PS8 - Understanding Touch

Understanding Touch Performance on Soft Smart Phone Keyboards
Shiri Azenkot, Shumin Zhai

Text entry on smart phones is far slower and more error-prone than on traditional keyboards, despite sophisticated detection and auto-correct algorithms. We explore touch performance on soft QWERTY keyboards, the de-facto standard for smart phone text entry, to better understand touch behavior and causes of error. In a lab study with 32 participants, we collected data from input with an index finger, two thumbs, and one thumb. We describe touch accuracy and precision for different keys, showing distinct patterns among the three hand postures. This suggests keyboards should adapt to different postures. We also discovered that participants' touch precision was relatively high given typical key dimensions, but there were pronounced and consistent touch offsets. We describe our empirical findings and discuss implications for design that may serve as a foundation for improving soft keyboards.

Understanding Unconscious Slipping for One-Handed Thumb Use on Touchscreen-Based Mobile Devices
Wenchang Xu, Jie Liu, Chun Yu, Yuanchun Shi

There is usually an unaware screen distance between initial contact and final lift-off when users click on touchscreen-based mobile devices with their fingers, which may affect users' target selection accuracy, gesture performance, etc. In this paper, we summarize such case as unconscious slipping and give its models under both static and dynamic scenarios. We then conducted two user studies to understand unconscious slipping for the widely-adopted one-handed thumb use on touchscreen-based mobile devices under both scenarios respectively. Our findings shed light on the following four questions: 1) what are the factors that affect unconscious slipping; 2) what is the distance range of the slipping; 3) how is the distance varying over time; 4) how are the unconscious points distributed around the initial contact point. These results not only explain certain touch inaccuracy, but also provide important reference for optimization and future design of UI components, gestures, input techniques, etc.

Design and Evaluation of a Pie Menu Augmented Soft Keyboard for Chinese Pinyin Input Methods (Nominated)
Ying Liu, Lingzhi Wang, Xiantao Chen, Shen Li, Hequan Zhang, Shumin Zhai

Soft keyboard for Chinese pinyin input methods are rarely studied although it is one of the default methods on devices with touch screens. Via an analysis of the diagraph frequency of the pinyin system, we discovered a unique characteristic of the pinyin system: only 10 Roman letters are needed for the subsequent characters in a pinyin syllable after the leading letter. Making use of this feature and existing knowledge on layout optimization of soft keyboard, pie menu and ShapeWriter, we designed a pie menu augmented keyboard. We conducted a user study to compare user performance to test if the pie menu can help to increase user performance with a working prototype. We found that after about 2 hours use of the pie menu augmented quasi-QWERTY keyboard, users can reach a speed of 25 Chinese characters per minute with slightly lower error rate. Moreover, users can well remember the layout of the pie menu after about two hours use of it.

PS9 - Multiplexing

An Empirical Investigation into How Users Adapt to Mobile Phone Auto-Locks in a Multitask Setting
Duncan Brumby, Vahab Seyedi

Auto-locks are a necessary feature on modern day mobile phones, but can they sometimes have detrimental consequences? In this paper we investigate how auto-locks can affect multitasking behavior. A study is conducted in which participants had to enter text using a touch-screen interface while driving a simulated vehicle in a lab setting. Different auto-lock mechanisms were implemented on the secondary task, manipulating both the duration of the lockout threshold (i.e., the period of inactivity before the auto-lock was initiated) and the complexity of the unlock procedure (i.e., how easy it was for the user to unlock the device once it had locked). Results showed that lane-keeping performance on the primary driving task was worse when there was a shorter lockout threshold. The reason for this was two-fold: (1) participants took fewer long pauses between typing actions, so as to avoid being locked out of the device, and (2) when the device did lock, unlocking it took time and further distracted the driver. In support of this latter finding, we also found that a more complex unlock procedure, which required a pin code to be entered, resulted in worse lane-keeping performance than when the device could be unlocked by making a simple button press. The implications of this research for mobile HCI are discussed.

What was I Doing? Where is My Mind? The Costs of Mobile Application Interruptions
Luis A. Leiva, Matthias Böhmer, Sven Gehring, Antonio Krüger

Smartphone users might be interrupted while interacting with an application, either by intended or unintended circumstances. In this paper, we report on a large-scale observational study that investigated mobile application interruptions in two scenarios: (1) intended back and forth switching between applications and (2) unintended interruptions caused by incoming phone calls. Our findings reveal that these interruptions rarely happen (at most 10% of the daily application usage), but when they do, they may introduce a significant overhead (can lag up to 4 times a task completion). We conclude with a discussion of the results, their limitations, and a series of implications for the design of mobile phones.

Visual search on a mobile device while walking
Ji Lim, Cary Feria

Previous studies have examined the effects of walking and user interface elements on mobile task performance, using physical target selection tasks. The present study investigates the effect of walking and user interface elements on visual search on a mobile device, isolating the effects on perceptual and cognitive processes. The effects of object size, contrast, and target location on mobile devices while walking and standing were examined. A serial visual search using “T” and “L” shapes on a mobile device was conducted, which controlled for the physical target selection involvement. The results showed that walking, bigger object size and the target position in the outer area of the mobile device display slowed the visual search reaction time. Walking slows perceptual and cognitive processes involved in visual search on a mobile device. Placing major contents in the center of the mobile display improves visual search performance.

Bridging Waiting Times on Web Pages
Florian Alt, Alireza Sahami Shirazi, Albrecht Schmidt, Richard Atterer

Nowadays, high-speed Internet connectivity makes browsing a convenient task. However, there are many situations in which surfing the web is still slow: bandwidth may be limited or servers may be slow to respond due to high load or complex queries. As a consequence, loading web pages can take several seconds, making (mobile) browsing cumbersome and time-consuming. In this paper, we present an approach which makes use of the time spent on waiting and loading the next page, by bridging the wait with extra cached or preloaded content. We show how the content (e.g., news, Facebook activity, ads) can be adapted to the user's interests as well as to the context of use, hence making mobile surfing more comfortable. We compare two approaches: In the time-multiplex mode, the entire screen displays bridging content until the requested page has finished loading. In the space-multiplex mode, content is displayed alongside the requested content while it loads. To add content, we use a special HTTP proxy, which intercepts requests and inserts JavaScript code, which allows the bridging content from websites of our choice to be loaded. The approach has been evaluated with 15 participants, assessing suitable content, usability, and application scenarios.

Panel 2

PS10 - Non-visual Interaction

Thermal Icons: Evaluating Structured Thermal Feedback for Mobile Interaction (Nominated)
Graham Wilson, Stephen Brewster, Martin Halvey, Stephen Hughes

This paper expands the repertoire of non-visual feedback for mobile interaction, established through Earcons and Tactons, by designing structured thermal cues for conveying information. Research into the use of thermal feedback for HCI has not looked beyond basic 'yes-no' detection of stimuli to the absolute identification of those stimuli. We first designed thermal icons that varied along two parameters to convey two pieces of information. We also designed intramodal tactile icons, combining one thermal and one vibrotactile parameter, to test perception of different tactile cues and so evaluate the possibility of expanding existing vibrotactile displays with thermal feedback. Thermal icons were identified with 83% accuracy, while intramodal icons had 97% accuracy, suggesting thermal icons are a viable means of conveying information in mobile HCI, when audio and/or vibrotactile feedback is not suitable or desired.

Touching the Micron: Tactile Interactions with an Optical Tweezer
Stuart Lamont, Richard Bowman, Matthias Rath, John Williamson, Roderick Murray-Smith, Miles Padgett

A tablet interface for manipulating microscopic particles is augmented with vibrotactile and audio feedback.The feedback is generated using a novel real-time synthesis library based on approximations to physical processes, and is efficient enough to run on mobile devices, despite their limited computational power. The feedback design and usability testing was done with a realistic simulator on appropriate tasks, allowing users to control objects more rapidly, with fewer errors and applying more consistent forces. Users commented that the feedback makes the interaction more tangible, giving the user more awareness of changes in the characteristics of the optical tweezers as the number of traps changes.

BrailleTouch: An Evaluation of Mobile Eyes-Free Touchscreen Text Entry for the Visually Impaired
Caleb Southern, James Clawson, Brian Frey, Gregory Abowd, Mario Romero

We present the evaluation of BrailleTouch, an eyes-free soft keyboard for touchscreen smartphones. We developed BrailleTouch as an assistive technology for the visually impaired. Based on the Perkins Brailler, BrailleTouch presents a standard six-key chorded braille keyboard. In our laboratory evaluation, eleven blind participants typed for 165 twenty-minute sessions on three mobile devices: 1) BrailleTouch on a smartphone; 2) a soft braille keyboard on a touchscreen tablet; and 3) a commercial braille keyboard with physical keys. Expert blind users typed at an average of 23.2 words per minute (wpm) on the smartphone using BrailleTouch. The fastest participant, a touchscreen novice, achieved 32.1 wpm during his first session. We report the speed for braille text entry on three mobile devices, an in depth error analysis, and the lessons learned for the design and evaluation of accessible and eyes-free soft keyboards.

PocketMenu: Non-Visual Menus for Touch Screen Devices
Martin Pielot, Anastasia Kazakova, Wilko Heuten, Susanne Boll

We present PocketMenu, a menu optimized for non-visual, in-pocket interaction with menus on handheld devices with touch screens. By laying out menu items along the border of the touch screen its tactile features guide the interaction. Additional vibro-tactile feedback and speech allows identifying the individual menu items non-visually. In an experiment, we compared PocketMenu with iPhone's VoiceOver. Participants had to control an MP3 player while walking down a road with the device in the pocket. The results provide evidence that in the above context the PocketMenu outperforms VoiceOver in terms of completion time, selection errors, usability. Hence, it enables usage of touch screen apps in mobile contexts (e.g. walking, hiking, or skiing) and limited interaction spaces (e.g. device resting in a pocket).

Clicking Blindly: Using Spatial Correspondence to Select Targets in Multi-Device Environments
Krzysztof Pietroszek, Edward Lank

We propose spatial correspondence targeting to support interaction between devices in multi-device environments when network connectivity fails. In spatial correspondence targeting, for a given target on surface A, an end-user envisions the relative position of that target on surface B and interacts on surface B without any visual depiction of the target on surface B. The targeting task relies on human spatial visualization ability, i.e. the ability to relate the spatial position of objects on one display to their scale-invariant position on another display. We provide experimental evidence that demonstrates that users may be able to target up to 25 discrete targets using a smartphone screen even in the absence of a depiction of the target on the smartphone screen. We argue that the accuracy of spatial correspondence targeting is sufficient for the technique to have many practical applications.

Demos & Space Challenges

PS11 - Location

Urban Exploration Using Audio Scents
Andreas Komninos, Peter Barrie, Vassilios Stefanis, Athanassios Plessas

We describe the design and evaluation of an audio-based mixed reality navigation system that uses the concept of audio scents for the implicit guidance of tourists and visitors of urban areas, as an alternative to turn-by-turn guidance systems. A field trial of our prototype uncovers great potential for this type of implicit navigation and is received positively by our participants. We discuss the technical implementation of our prototype, detailed findings from quantitative and subjective evaluation data gathered during the field trial and highlight possible strands for further research and development

A real-world study of an audio-tactile tourist guide
Delphine Szymczak, Kirsten Rassmus-Gröhn, Charlotte Magnusson, Per-Olof Hedvall

This paper reports on the in-context evaluation of an audio-tactile interactive tourist guide – one test was done in a medieval city center, and the other was done at an archaeological site. The activity theory framework was used as a perspective to guide design, field-study and analysis. The evaluation shows that the guide allows users to experience an augmented reality, while keeping the environment in focus (in contrast with the common key-hole like experience that on-screen augmented reality generates). The evaluation also confirms the usefulness of extending the vibrational feedback to convey also distance information as well as directional information.

TraceViz: 'Brushing' for Location Based Services
Yung-Ju Chang, Pei-Yao Hung, Mark Newman

The popularization of Location Based Services (LBS) has created new challenges for interaction designers in validating their design of applications. Several existing tools designed to play back GPS location traces data streams have shown potential for testing LBS applications for supporting rapid and reflective prototyping. However, selecting a useful set of location traces from among a large collection of location traces remains a difficult task. In this paper, we present TraceViz, the first system that is purposed for supporting LBS designers in exploring, filtering, and selecting location traces with the support of brushing. We devise Reselect, Intersection, and Union modes to allow LBS designers to flexibly adjust their trajectory filter criteria to find the location traces of interest. We believe that TraceViz can potentially encourage LBS designers to validate their LBS design with a larger variety of location traces, which in turn can produce a higher quality of application.

iFitQuest - A Mobile Location-Aware Exergame for Adolescents
Andrew Macvean, Judy Robertson

Exergames, games that encourage and facilitate physical exercise, are growing in popularity thanks to progressions in ubiquitous technologies. While initial findings have confirmed the potential of such games, little research has been done on systems which target the needs of adolescent children. In this paper we introduce iFitQuest, a mobile location-aware exergame designed with adolescent children in mind. In an attempt to understand how exergames can be used to target adolescent children, and whether they can be effective for this demographic, we outline the results of a school based field study with twenty-five users. Through a detailed analysis of our results, we conclude that a) iFitQuest appeals to twelve to fifteen year olds and causes them to exercise at moderate to vigorous levels; b) children from a variety of gaming backgrounds enjoyed the game, although boys were more likely to rate it highly than girls c) users were capable of appropriately manipulating the difficulty level of the game and their effort and enjoyment remained consistent in response to losing the game, and d) in order to develop effective systems that can dynamically adapt to the adolescent users, an in-depth understanding of users' behavior is required.

Tacticycle: Supporting Exploratory Bicycle Trips
Martin Pielot, Benjamin Poppinga, Wilko Heuten, Susanne Boll

Going on excursions to explore unfamiliar environments by bike is a popular activity in many places in this world. Investigating the nature of exploratory bicycle trips, by studying tourists on their excursions on a famous vacation island, we found that existing navigation systems are either not helpful or discourage exploration. We therefore propose a conceptual prototype of a user interface for a bicycle navigation systems. Relying on a minimal set of navigation cues only it helps staying oriented while supporting spontaneous navigation and exploration at the same time. In cooperation with a local bike rental we rented the prototype to tourists who took it on their actual excursions. The results show that they always felt oriented and encouraged to playfully explore the island, providing a rich, yet relaxed travel experience.

PS12 - Collaboration and Sharing

Displaying Mobile Feedback during a Presentation
Jaime Teevan, Daniel Liebling, Ann Paradiso, Carlos Carcia Jurado Suarez, Curtis von Veh, Darren Gehring

Smartphone use in presentations is often seen as distracting to the audience and speaker. However, phones can encourage people to participate more fully in what goes on around them and build stronger ties with their companions. In this paper, we describe a smartphone interface designed to help audience members engage fully in a presentation by providing real time mobile feedback. This feedback is then aggregated and reflected back to the group via a projected visualization, with notifications provided to the presenter and the audience on interesting feedback events. We deployed this system in a large enterprise meeting, and collected information about the attendees' experiences with it via surveys and interaction logs. We find that providing mobile phone-based feedback is convenient, helps attendees pay close attention to the presentation, and enables them to feel connected with other audience members.

MobiComics: Collaborative Use of Mobile Phones and Large Displays for Public Expression
Andrés Lucero, Jussi Holopainen, Tero Jokela

We explore shared collocated interactions with mobile phones and public displays in an indoor public place. We introduce MobiComics, an application that allows a group of collocated persons to flexibly create and edit comic strip panels using their mobile phones. The prototype supports ad hoc sharing of comic strip panels between people and onto two public displays by taking the spatial arrangement of people into account, measured with a radio tracking technology integrated in their mobile phones. MobiComics also includes game-like elements to foster social interaction between participants. Our evaluations show that people enjoyed creating panels collaboratively and sharing content using the proposed interaction techniques. The included game-like features positively influenced social interaction.

I wanted to settle a bet! - Understanding Why and How People Use Mobile Search in Social Settings
Karen Church, Antony Cousin, Nuria Oliver

Recent work in mobile computing has highlighted that conversations and social interactions have a significant impact on mobile Web and mobile search behaviours. To date, however, this social element has not been explored fully and little is known about why and how mobile users search for information in social settings. The goal of this work is to provide a deeper understanding of social mobile search behaviours so that we may improve future mobile search experiences that involve a social component. To this end we present the results of two studies: a survey involving almost 200 users and a two-week diary and follow-up interview study of 20 users. Our results extend past research in the mobile search space, by exploring the motivations, circumstances and experiences of using mobile search in social settings to satisfy group information needs. Our findings point to a number of open research challenges and implications for enriching the search experiences of mobile users.

Investigating Collaborative Annotation on Slate PCs (Nominated)
Jennifer Pearson, George Buchanan, Harold Thimbleby

Mobile reading is becoming evermore popular with the introduction of eInk devices such as the Kindle, as well as the many reading applications available on slate PCs and cellular handsets. The portable nature and large storage capacity of these modern mobile devices is making reading a more technology orientated activity. One aspect of mobile reading that has been given surprisingly little attention, however, is the notion of collective reading - an activity commonly undertaken on paper. We investigate the support of group reading using slate PCs, focussing on collective annotation. User evaluations of our new system, BuddyBooks demonstrate that the slate PC form factor can provide advantages to paper for group reading; an area where desktop PCs have proved inferior. While annotation practices change with the new format, coordinating within the group can be improved when touch-interaction is carefully exploited.

Degrees of Sharing: Proximate Media Sharing and Messaging by Young People in Khayelitsha
Marion Walton, Gary Marsden, Silke Hassreiter, Sena Allen

This paper explores the phone and mobile media sharing relationships of a group of young mobile phone users (N=46) in Khayelitsha, South Africa. Intensive sharing took place within peer and intimate relationships, while resource sharing characterized relationships with a more extensive circle, including members of the older generation. Phones were kept open to others to avoid inferences of stinginess, disrespect, or secretiveness and the use of privacy features (such as passwords) was complicated by conflicts between communal values and the protection of individual property and privacy. Collocated phone use trumped online sharing but media on phones constituted public personae similar to social media 'profiles'. Proximate sharing within close relationships allowed social display, relationship-building and deference to authority. We suggest changes to current file-based interfaces for Bluetooth pairing, media 'galleries', and peer-to-peer text communication to better support proximate exchanges of media and messaging.

PS13 - Learning and Training

An investigation into the use of tactile instructions in snowboarding (Nominated)
Daniel Spelmezan

In many sports, athletes do not receive instructions while practicing an exercise because they are spatially separated from their coach. We present the findings of an in-the-wild study that demonstrate the potential for teaching sport skills with realtime tactile instructions. Our study focused on snowboard training. Ten amateur snowboarders learned an unfamiliar riding technique with tactile instructions during the descent. A wearable system provided these instructions in sync with the movements of the snowboard. These instructions signaled which body movements to perform. We found that the addition of tactile instructions can help snowboarders to improve their skills. We provide insights into the snowboarders' opinion and give recommendations for teaching sport skills with tactile instructions. Our findings help to identify the conditions under which realtime tactile instructions can support athletes in sports training.

TipTapTones: Mobile Microtraining of Mandarin Sounds (Nominated)
Darren Edge, Kai-Yin Cheng, Michael Whitney, Yao Qian, Zhijiie Yan, Frank Soong

Learning a second language is hard, especially when the learner's brain must be retrained to identify sounds not present in his or her native language. It also requires regular practice, but many learners struggle to find the time and motivation. Our solution is to break down the challenge of mastering a foreign sound system into minute-long episodes of 'microtraining' delivered through mobile gaming. We present the example of TipTapTones - a mobile game with the purpose of helping learners acquire the tonal sound system of Mandarin Chinese. In a 3-week, 12-user study of this system, we found that an average of 71 minutes gameplay significantly improved tone identification by around 25%, regardless of whether the underlying sounds had been used to train tone perception. Overall, results suggest that mobile microtraining is an efficient, effective, and enjoyable way to master the sounds of Mandarin Chinese, with applications to other languages and domains.

MemReflex: Adaptive Flashcards for Mobile Microlearning (Nominated)
Darren Edge, Stephen Fitchett, Michael Whitney, James Landay

Flashcard systems typically help students to learn facts (e.g., definitions, names, and dates), relying on intense initial memoriztion with subsequent tests delayed up to days later. This approach does not exploit the short, sparse, and mobile opportunities for microlearning throughout the day, nor does it support learners who need the motivation that comes from successful study sessions. In contrast, our MemReflex system of adaptive flashcards gives fast-feedback by retesting new items in quick succession, dynamically scheduling future tests according to a model of the learner's memory. We evaluate MemReflex across three user studies. In the first two studies, we demonstrate its effectiveness for both audio and text modalities, even while walking and distracted. In the third study of second-language vocabulary learning, we show how MemReflex enhanced learner accuracy, confidence, and perceptions of control and success. Overall, the work suggests new directions for mobile microlearning and 'micro activities' in general.

Mobilizing Education: Evaluation Of a Mobile Learning Tool in a Low-Income School
Vanessa Frias-Martinez, Jesus Virseda, Aldo Gomero

The pervasiveness of feature phones in emerging economies has contributed to the advent of mobile learning applications for low-income populations. However, many of these tools lack the proper evaluation required to understand their educational impact. In this paper, we extend the state of the art by presenting the evaluation of a game-based mobile learning tool – EducaMovil– in both formal and informal settings at a low-income school in Lima, Peru. We show that EducaMovil improves knowledge acquisition in the formal environment of a classroom. In addition, use of the tool in more informal settings such as school breaks enhances the level of knowledge, as long as there is continuous engagement over time. We also demonstrate that EducaMovil can be used as a paperless complement to homework. Finally, we provide teachers with a set of guidelines for a successful deployment of a mobile learning program at their schools.

Closing Keynote

Ken Goldberg, Professor, IEOR, EECS, Information, UC Berkeley

Cloud Robotics, Google Goggles, and the Rashomon Project

The Cloud provides access to vast resources of data, software, computation, as well as access to unprecedented levels of human interaction and creativity. Cloud Robots will use advances in statistical machine learning to be more aware than oblivious, more social than solitary, and more like companions than tools. The Cloud also allows human designers and researchers worldwide to form new networks to identify and solve important challenges. I'll present several examples including a new project integrating a robot with the Google Goggles mobile object recognition system.

I'll then describe a new project motivated by the November 2011 clashes between students and police on UC campuses that were documented with mobile smartphone videos. The “Rashomon Project” will develop an open-source, online “toolkit” that helps activists and the public develop and analyze “Multi-Perspective Chronology” by arranging these videos so that they are precisely time aligned along a time axis.

Ken Goldberg is an inventor working at the intersection of art, robotics, and social media. At UC Berkeley, Ken teaches and supervises research in Robotics, Automation, Art, and New Media. Ken was awarded dual degrees in Electrical Engineering and Economics from the University of Pennsylvania (1984) and MS and PhD degrees from Carnegie Mellon University (1990). He joined the UC Berkeley faculty in 1995 where he is craigslist Distinguished Professor of New Media. He is a Professor of Industrial Engineering and Operations Research, with secondary appointments in Electrical Engineering/Computer Science, Art Practice, and the School of Information. Ken also holds an appointment in the UC San Francisco Medical School's Department of Radiation Oncology.

Ken has published over 170 peer-reviewed technical papers on algorithms for robotics, automation, and social information filtering; his inventions have been awarded eight US Patents. He is Editor-in-Chief of the IEEE Transactions on Automation Science and Engineering (T-ASE), Co-Founder of the African Robotics Network (AFRON), Co-Founder of the Berkeley Center for New Media (BCNM), Co-Founder and CTO of Hybrid Wisdom Labs, Co-Founder of the Moxie Institute, and Founding Director of UC Berkeley's Art, Technology, and Culture Lecture Series.

Ken's art installations, based on his research, have been exhibited internationally at venues such as the Whitney Biennial, Berkeley Art Museum, SF Contemporary Jewish Museum, Pompidou Center, Buenos Aires Biennial, and the ICC in Tokyo. Ken co-wrote three award-winning Sundance documentary films, “The Tribe”, “Yelp”, and “Connected: An Autoblogography of Love, Death, and Technology.” His artwork is represented by the Catharine Clark Gallery in San Francisco.

Ken was awarded the Presidential Faculty Fellowship in 1995 by President Clinton, the National Science Foundation Faculty Fellowship in 1994, the Joseph Engelberger Robotics Award in 2000, and elected IEEE Fellow in 2005.

Ken lives in Mill Valley, California with his daughters and wife, filmmaker and Webby Awards founder Tiffany Shlain.