CS295J/Literature class 2.11

• Project Ernestine: Validating a GOMS Analysis for Predicting and Explaining (File:Project Ernestine Paper.pdf) A research project from way back that demonstrated that modeling cognition plus motor plus perceptual tasks by telephone operators could predict the efficiency of a new user interface. The efficiency turned out to be lower than the old, low-tech version, which was a surprise. This paper is just the kind of result I'd like to be able to publish about more complex user interfaces. (Owner: David Laidlaw, discussion: Stephen Brawner, discussant: ?)

• Cognitive Strategies and Eye Movements for Searching Hierarchical Computer Displays This paper uses predictive modeling and eye-tracking data together to explain search behavior in hierarchical or non-hierarchical layouts. The layouts were lists of items organized in labeled groups, with the labels being either useful (hierarchical condition) or random (non-hierarchical condition). The research question is about whether people use different strategies when searching for a target item in each condition. They compared their model's predictions to observed eye movements and found them to be a pretty good fit, and therefore characterized search strategies using the model. (Owner: Caroline Ziemkiewicz, discussion: Chen, discussant: Diem Tran)

• Mapping Human Whole-Brain Structural Networks with Diffusion MRI The authors use diffusion MRI to create network maps with significantly larger detail than previous models of physical connectivity. Their methods allow them to study live humans and model the interconnectivity of neuronal groups as networks with thousands of nodes versus previous methods with less than 100 nodes studied from post-mortem animal subjects. Based on these new experimental methods they demonstrate that the brain network is in the form of a small world. (Owner: Stephen Brawner, discussant: Jenna Zeigen, discussant: Caroline Ziemkiewicz)

• Toward a Perceptual Theory of Flow Visualization, Colin Ware, CG&A March 08

This paper is a good entry point for Ware's other work on neural modeling for visualization. It describes how spatial receptor patterns in the visual cortex enable contour interpretation and related visualization tasks (e.g., particle advection in flow fields). There's also some good discussion about a perception-based approach to visualization, validating visual mappings with perceptual theories. (Owner: Steven Gomez, discussion: Chen, discussant: Nathan)

• Demonstrating the Feasibility of Using Forearm Electromyography for Muscle-Computer Interfaces Saponas-2008-DFU (Posted by Michael Spector)

The authors of this paper discuss the feasibility of muscle-computer interaction by demonstrating that simple finger gestures can be accurately detected using EMG data gathered from the upper forearm. While very accurate EMG data can be obtained with more invasive methods, the paper focuses on the applications of devices that can be used with ease in everyday situations: the example given is an armband type sensor. Muscle-computer interfaces like these are relatively undeveloped but could provide an intuitive means for interacting with computer systems, and through them, complicated visual representations of information. Interfaces like the one proposed in this paper are particularly good for scenarios where the user needs to be able to multitask quickly between the computer system and other activities, an important quality for scientific problem solving. While this paper does not explicitly touch upon cognitive aspects of HCI, its primary focus is to develop a system that lessens the cognitive strain on the user, with practicality and ease of use. The technical details of the experiment are less pertinent to our project, but the underlying message remains relevant. (Owner: Michael Spector, Discussant: Clara Kliman-Silver, Discussant: ?)

• Audio-Visual Intent-To-Speak Detection For Human-Computer Interaction, Cuetos-2000-ISD. (Posted by Michael Spector)

Discusses a speech detection system that uses both auditory and visual cues to more accurately detect speech commands. It aims to recognize the user's intention to speak, and to ignore background noise, or speech recognized as not being directed at the system. Although it is fairly dated, this paper is relevant in that it discusses applications of cognition/perception to HCI. (Owner: ? Discussant: Caroline Ziemkiewicz Discussant: ?)

• This is your brain on interfaces: enhancing usability testing with functional near-infrared spectroscopy Hirshfield-2011-BOI

The authors use a brain measuring device to detect activations in the brain when a user is performing a task. In this way, the authors are able to measure a range of cognitive workload states, known as subjective factors, which are difficult to measure using qualitative studies. They quantify workloads of users in three different UI tasks and point out the usability of a UI design choice as well as the low-level cognitive resources in the brain correspond to a task. How this paper relates to our project - As this paper presents a novel approach to measure effectiveness of a UI, it is relevant that we consider this in our process to evaluate the tools we develop. (Owner: Diem Tran, Discussant: Hua Guo, Discussant: Michael Spector)

• Measuring effectiveness of graph visualizations: a cognitive load perspective Huang-2009-JIV

This paper has proposed the following three items:

1. The concept of "mental effort" as a measurement for effectiveness of visualization,
2. The formulation that "visualization efficiency" can be measured in terms of cognitive gain and cognitive loss, where cognitive gain is reflected through response accuracy, and cognitive cost is reflected through response time and mental effort.
3. A model of factors related to cognitive load. In this model, domain complexity, data complexity, task complexity, visual complexity, demographics complexity, time complexity and between-factors complexity are considered as casual factors of cognitive load, and mental effort, response time and response accuracy are considered as assessment factors of cognitive load.
4. A model of the relationship between task performance, mental effort and cognitive load.

The formula for visualization efficiency intends to put more emphasis on "how much cognitive load is required when working with a certain visualization" in addition to the traditional emphasis on user performance. Taking cognitive load into account helps distinguish the effectiveness of two visualizations where the user performance is roughly equal, but the required cognitive workload is different when working with the two visualizations.

A user study is performed to (in)validate whether the proposed casual factors are significantly correlated with the proposed assessment factors.

Some thought about how this paper relates to our proposal:

1. "Mental effort" is a good indicator for visualization effectiveness, if combined properly with other indicators and measured accurately.
2. The author mentioned, though now fully expanded, in the paper that when considering cognitive load, we may want to distinguish between "useful cognitive load" which either contributes positively to user reasoning or is inevitable, such as data complexity and task complexity, and "wasted cognitive load" which is unnecessary for problem solving, such as visual complexity. This can be further explored to be incorporated into future cognitive modeling, or serve as design guidelines : (Owner: Hua Guo, Discussant: Clara Kliman-Silver, Discussant: Jenna Zeigen)

• Could I have the Menu Please? An Eye Tracking Study of Design Conventions McCarthy-2003-CMP

This article discusses search performance strategies in user interfaces per the results of an eye-tracking study. Specific attention is given to menu organization in the context of Web interfaces; the study asks whether people can learn to navigate menu environments that differ from standard layouts. Results show that there able to adapt to new layouts, even they violate previous expectations. Although substantial progress has been made in the past decade, the article draws attention to relevant design issues and concepts, especially as eye tracking methodologies continue continue to grow and improve. It also serves to reinforce the importance of eye tracking in HCI research and how exactly people analyze the data, techniques we can take into account as we seek new approaches for our own HCI research. (Owner: Clara Kliman-Silver, Discussant: Michael Spector, Discussant: Wenjun Wang)

• Cognitive Models of the Influence of Color Scale on Data Visualization Tasks -Breslow-2009-CMI

Discusses the ways color scales and differences can be influential in the optimization of data visualization and analysis. Discusses the differences between absolute identification and relative comparison tasks and the implications of different types of color scales on the performance of such tasks. The authors create computational models of the processes and then compare their predictions to the results of two experiments. This paper is relevant because the visualizations we create probably will involve color scales, and the analysis of our visualizations probably will involve both types of tasks described. (Owner: Jenna Zeigen, Discussant: Steven Gomez, Discussant: Stephen Brawner)

• Computational visual attention systems and their cognitive foundations: A survey Frintrop-2010-CVA

This paper "provides an extensive survey of the grounding psychological and biological research on visual attention as well as the current state of the art of computational systems".

Specifically, it:

1. Provides a brief overview of the human visual system and the concept of visual attention
2. Presents and discusses a number of different psychological and cognitive models of attention
3. Reports on the most common approaches to visual attention used by computational systems
4. Discusses the evaluation of these systems, their applications in computer vision and robotics, and open questions in the field

This relates to our project in that:

1. It provides an overview, and lots of references, to models of attention, which is generally useful for understanding human-computer interaction.
2. More specifically, predictive models could be used to identify regions of (particular) interest to users of the software, for example to provide additional details, auto-focus, or auto-select. These models, and their predictions, could be verified and compared with eye-tracking data.

(Owner: Nathan Malkin, Discussant: Hua Guo , Discussant: Wenjun Wang)

• Wizard of Oz for Participatory Design: Inventing a Gestural Interface for 3D Selection of Neural Pathway Estimates, Akers-2006-WOP.

The paper decribes a participatory design process which is used to invent an interface for neuroscientists to select neural pathways from existing brain images by mouse. Neuroscientists are frastrated selecting pathways. In this design process, they are asked to invite their own gestures, with a prototype including two basic gesture, which is unknown by the neuroal scientist. They use mouse to make a gesture and tell the prototype what they want to do, like drawing a circle to indicate they want pathways through the circle. The prototype achieves the goal by combining basic gesture operations, and showing the results to the scientist right away. It's better to use the prototype because people can obtain the results immediately. This design process is more productive than those not using a prototype. ( Owner:Chen, Discussant: Nathan, Discussant: Diem Tran)

• The Laws of Simplicity by John Maeda

An interesting work on the efficiency of minimalist design. Quick read for those interested.

A set of design guidelines some of which we may be able to build on in automating interface evaluation; will certainly apply to manual evaluations (Owner: Wenjun Wang, Discussant: Steven Gomez, Discussant: ?)