CS295J/Contributions for class 12

A predictive, fully-integrated model of user workflow which encompasses low-level tasks, working spheres, communication chains, interruptions and multi-tasking.

Owner: Andrew Bragdon

1. Traditionally, software design and usability testing is focused on low-level task performance. However, prior work (Gonzales, et al.) provides strong empirical evidence that users also work at a higher, working sphere level. Su, et al., develops a predictive model of task switching based on communication chains. Our model will specifically identify and predict key aspects of higher level information work behaviors, such as task switching. We will conduct initial exploratory studies to test specific instances of this high-level hypothesis. We will then use the refined model to identify specific predictions for the outcome of a formal, ecologically valid study involving a complex, non-trivial application.
2. Impact: Current information work systems are almost always designed around the individual task, and because they do not take into account the larger workflow context, these systems arguably do not properly model the way users work. By establishing a predictive model for user workflow based on individual task items, larger goal-oriented working spheres, multi-tasking behavior, and communication chains, developers will be able to design computing systems that properly model users and thus significantly increase worker productivity in the United States and around the world.
3. 3-week Feasibility Study: To ascertain the feasibility of this project we will conduct an initial pilot test to investigate the core idea: a predictive model of user workflow. We will spend 1 week studying the real workflow of several people through job shadowing. We will then create two systems designed to help a user accomplish some simple information work task. One system will be designed to take larger workflow into account (experimental group), while one will not (control group). In a synthetic environment, participants will perform a controlled series of tasks while receiving interruptions at controlled times. If the two groups perform roughly the same, then we will need to reassess this avenue of research. However, if the two groups perform differrently then our pilot test will have lent support to our approach and core hypothesis.
4. Risks/Costs: Risk will play an important factor in this research, and thus a core goal of our research agenda will be to manage this risk. The most effective way to do this will be to compartmentalize the risk by conducting empirical investigations - which will form the basis for the model - into the separate areas: low-level tasks, working spheres, communication chains, interruptions and multi-tasking in parallel. While one experiment may become bogged down in details, the others will be able to advance sufficiently to contribute to a strong core model, even if one or two facets encounter setbacks during the course of the research agenda. The primary cost drivers will be the preliminary empirical evaluations, the final system implementation, and the final experiments which will be designed to support the original hypothesis. The cost will span student support, both Ph.D. and Master's students, as well as full-time research staff. Projected cost: $1.5 million over three years.

A mixed-initiative system for interface design

Owner: Eric

Summary

Note: click here for flowchart.

We propose a framework for interface evaluation and recommendation that integrates behavioral models and design guidelines from both cognitive science and HCI. Our framework behaves like a committee of specialized experts, where each expert provides its own assessment of the interface, given its particular knowledge of HCI or cognitive science. For example, an expert may provide an evaluation based on the GOMS method, Fitts's law, Maeda's design principles, or cognitive models of learning and memory. An aggregator collects all of these assessments and weights the opinions of each expert based on past accuracy, and outputs to the developer a merged evaluation score and a weighted set of recommendations.

Different users have different abilities and interface preferences. For example, a user at NASA probably cares more about interface accuracy than speed. By passing this information to our committee of experts, we can create interfaces that are tuned to maximize the utility of a particular user type.

We evaluate our framework through a series of user studies. Interfaces passed to our committee of experts receive evaluation scores on a number of different dimensions, such as time, accuracy, and ease of use for novices versus experts. We can compare these predicted scores to the actual scores observed in user studies to evaluate performance. The aggregator can retroactively weight the experts' opinions to determine which weighting would have given the best predictions of user behavior for the given interface, and observe whether that weighting generalizes to other interface evaluations.

Inputs

• The task the user is trying to accomplish
• The GUI he/she is using to perform this task
• The utility a user gets for values of different performance metrics (time, cognitive load, fatigue, etc.)
• The predicted and/or actual trace of a user using this GUI

Outputs

• An evaluation of the GUI, in terms of the individual metric values (i.e. time, cognitive load, etc.), and the overall utility for this as expressed by the utility function.
• Suggested improvements for the GUI, in two forms:
  • Immediate transformations that can be automatically applied to the GUI
  • Higher level suggestions/guidelines that would have to be made by a developer

Contributions

• The creation of a language for abstractly representing user interfaces in terms of the layout of graphical components and the functional relationships between these components.
• A system for generating interaction histories within user interfaces to facilitate individual and collaborative scientific discovery, and to enable researchers to more easily document and analyze user behavior.
• A system that takes user traces and creates a GOMS model that decomposes user actions into various cognitive, perceptual, and motor control tasks.
• The development of other evaluation methods using various cognitive/HCI models and guidelines.
• A design tool that can provide a designer with recommendations for interface improvements. These recommendations can be made for a specific type of user or for the average user, as expressed by a utility function.

A Unified Model of Cognitive Perception and Processing in HCI

Steven

• We plan to develop a unified model of the conscious and subconscious mental processes which direct human behavior in interaction with digital interfaces. We will draw upon existing theoretical conceptions of these processes, including distributed cognition, activity theory, etc. This model will be empirically validated in a manner similar to that used in assessing interface efficiency via CPM-GOMS.

• Impact: This contribution will provide a standard model of mental processes upon which effective tools for interface evaluation can be built and by which they can be justified. Such a model would eliminate the need for costly wide-ranging testing of new interface suggestions.

A Quantitative, Cognition-Based Framework for Analyzing and Predicting User Performance

Owner: Trevor O'Brien

Summary

Established guidelines for designing human computer interfaces are based on experience, intuition and introspection. Because there is no integrated theoretical foundation, many rule-sets have emerged despite the absence of comparative evaluations. We propose to develop a theoretical foundation for interface design, drawing on recent advances in cognitive science – the study of how people think, perceive and interact with the world. We will distill a broad range of principles and computational models of cognition that are relevant to interface design and use them to compare and unify existing guidelines. To validate our theoretical foundation, we will use our findings to develop a quantitative mechanism for assessing interface designs, identifying interface elements that are detrimental to user performance, and suggesting effective alternatives. Results from this system will be explored over a set of case studies, and the quantitative assessments output by this system will be compared to actual user performance.

Within this framework, we will introduce novel techniques for collecting and filtering user interaction histories with respect to user goals. In addition, we propose to integrate data from multiple sensing technologies -- including pupil-tracking, muscle-activity monitoring and auditory recognition -- with these interaction histories to analyze the explicit contributions of perception, cognition and motor skills with respect to user performance. This data will be used to define a CPM-GOMS model of cognition for a given interface, providing a quantitative, predictive, cognition-based parameterization of usability. From empirically collected data, user trajectories through the model (critical paths) will be examined with respect to the cognitive principles obtained from our theoretical findings, highlighting bottlenecks within the interface, and offering suggested alterations to the interface to induce more optimal user trajectories.

Contributions

• Design and user-study evaluation of novel techniques for collecting and filtering user traces with respect to user goals.
• Extensible, low-cost architecture for integrating pupil-tracking, muscle-activity monitoring, and auditory recognition with user traces in existing applications.
• System for isolating cognitive, perceptual, and motor tasks from an interface design to generate CPM_GOMS models for analysis.
• Design and quantitative evaluation of semi-automated techniques for extracting critical paths from an existing CPM_GOMS model.
• Novel algorithm for analyzing empirically-collected critical paths and suggesting optimized paths based on established research in cognitive science.

Specific Aims

• Incorporate interaction history mechanisms into a set of existing applications.
• Perform user-study evaluation of history-collection techniques.
• Distill a set of cognitive principles/models, and evaluate empirically?
• Build/buy sensing system to include pupil-tracking, muscle-activity monitoring, auditory recognition.
• Design techniques for manual/semi-automated/automated construction of CPM-GOMS model from interaction histories and sensing data.
• Design system for posterior analysis of interaction history w.r.t. CPM_GOMS model, evaluating critical path trajectories.
• Design cognition-based techniques for detecting bottlenecks in critical paths, and offering optimized alternatives.

Input

• User traces: interaction histories coupled with contextual information about the interface itself and the application to which it belongs.
• User-sensing data: head-tracking data, pupil-tracking data, muscle-activity monitoring, auditory recognition.
• High-level user goals: set of overarching user intents.
• Set of cognitive guidelines/heuristics (used to analyze and optimize critical paths)

Output

• CPM-GOMS-style breakdown of perceptual, motor and cognitive tasks.
• Empirically observed critical path trajectories.
• Cognition-based breakdown of user trajectories. Highlights bottlenecks, offers suggestions to improve bottlenecks. (parallelization?)

Significance

• Evaluate utility of cognitive principles in HCI.
• Provide a quantitative, cognition-based mechanism for task analysis.
• Software system/architecture that allows other researchers and developers to analyze interface design effectively.
• In the case of scientific applications, aid in the adoption of tools by domain scientists.

A Unified Framework of Cognition in Analysis and Prediction in HCI

Owner: E J Kalafarski 14:25, 17 April 2009 (UTC)

Summary

Established guidelines for designing human-computer interfaces are based on experience, intuition and introspection. Because there is no integrated theoretical foundation, many rulesets have emerged despite the absence of comparative evaluation. We propose a theoretical foundation for interface design, drawing on recent advances in cognitive science—the study of how people think, perceive and interact with the world.

We will distill a broad range of principles and computational models of cognition that are relevant to interface design and use them to compare and unify existing guidelines. To validate our theoretical foundation, we will use our findings to develop a quantitative mechanism for assessing interface designs, identifying interface elements that are detrimental to user performance, and suggesting effective alternatives. Results from this system will be explored over a set of case studies, and the quantitative assessments output by this system will be compared to actual user performance.

Cognition, in this broad sense, cannot be limited to internal processing. A central focus of our work will be to broaden the range of cognitive theories that are used in HCI design, and we will explore the effects of improvement on all "links" in the interaction "chain"—cognition, perception, and motor skills, the CPM of the so-called CPM-GOMs evaluation method. Few low-level theories of perception and action, such as Fitts's law, have garnered general acceptance in the HCI community due to their simple, quantitative nature, and widespread applicability. Our aim is to produce similar predictive models that apply to lower levels of perception as well as higher levels of cognition, including vision, learning, memory, attention, reasoning and task management.

We will focus on generating an extensible, generalizable framework that can be applied to a broad range of interface design challenges. Interfaces exist everywhere in the modern world, and much research has accumulated regarding how people manage multiple tasks, how machines distribute tasks and work, and how devices communicate and anticipate user action effective. The promise of such a foundation is rich, and the potential benefits game-changing. We intend to do no less than provide new methodology for the examination of the human-computer system as a unified cognitive entity. E J Kalafarski 14:58, 17 April 2009 (UTC)

Input

• Interface specification (within, to begin with, narrow and realistic parameters)
• User traces, including user-sensing data: interaction histories coupled with contextual information about the interface itself and the application to which it belongs.
• High-level user goals and intent
• Set of cognitive guidelines/heuristics (used to analyze and optimize critical paths)

Output

• CPM-GOMS-style breakdown of perceptual, motor and cognitive tasks.
• Empirically observed critical path trajectories.
• Cognition-based breakdown of user trajectories. Highlights bottlenecks, offers suggestions to improve bottlenecks.

Contributions

• The creation of a language for abstractly representing user interfaces in terms of the layout of graphical components and the functional relationships between these components.
• A system for generating interaction histories within user interfaces to facilitate individual and collaborative scientific discovery, and to enable researchers to more easily document and analyze user behavior.
• A design tool that can provide a designer with recommendations for interface improvements, based on a unified matrix of cognitive principles and heuristic design guidelines.

Specific Aims

• Incorporate interaction history mechanisms into a set of existing applications.
• Perform user-study evaluation of history-collection techniques.
• Distill a set of cognitive principles/models, and evaluate empirically?
• Build/buy sensing system to include pupil-tracking, muscle-activity monitoring, auditory recognition.
• Design techniques for manual/semi-automated/automated construction of CPM-GOMS model from interaction histories and sensing data.
• Design system for posterior analysis of interaction history w.r.t. CPM_GOMS model, evaluating critical path trajectories.
• Design cognition-based techniques for detecting bottlenecks in critical paths, and offering optimized alternatives.

Significance

• Merges disparate-but-related fields of usability and cognition.
• Provide a quantitative, cognition-based mechanism for task analysis.
• Software system/architecture that allows other researchers and developers to analyze interface design effectively.
• Revise and provide theoretical foundations for industry-standard human usability guidelines.

"Big World" Goals

• Broader understanding of basic, reliable usability principles and more prevalent application of them.
• Efficient, profitable, and compatible usability application methods in business.
• A set of usability principles that supersedes Human-Computer Interaction and is applicable in all varieties of product development.

Interface Evaluation Using Parallel Cognitive Models (Adam and Jon)

Summary

Existing approaches for using cognitive models to evaluate user interfaces are based on extensive cognitive architectures that incorporate that integrate much of cognitive theory. The disadvantage of this approach is that modifying and using these models is resource intensive

We propose a distributed approach whereby simple cognitive models of specific behaviors each evaluate an interface independently, and those scores are combined to give a general evaluation. This approach is more flexible, and permits easier modification and use at multiple levels of detail.

Input

• Functions that the interface performs
• Interface description
• User traces

Sample Modules

• Exogenous attention
  • Question: To what extent does the interface draw your attention to what you need?
  • Need Interface display; user traces.

• Automaticity
  • Question: How often is executive attention needed?
  • Need: Predictability given previous actions and information displayed

• Fitts Law
  • Question: Time to move pointer from one operation location to the next?
  • Need a physical arrangement of elements and transition matrix (user traces?)

• Memory Load
  • Question: How many things are remembered simultaneously?
  • Need Interval (average?) between presentation and use. User traces.

• Affordances
  • Question: Are interactions with the interface “natural”?
  • Elements, their associated actions, and their functions

• Interruptions
  • Question: How long for interruption recovery?
  • Need User traces and predictability given information displayed

Output

• Efficiency curve