CS295J/Literature

Perception

• Colin Ware: Information Visualization: Perception for Design

Insight into some of the theory of perception as it pertains to building visual interfaces (David)

• Some unindentified paper(s)/book(s) about Gestalt theories of perception and cognition wikipedia page

These theories, from the 40's inform visual design and may provide an analogy for integration of theory and practice. They describe some characteristics of perception that have been used as evaluative rules in UI design. (David)

• Feeling Bumps and Holes without a Haptic Interface: the Perception of Pseudo-Haptic Textures Lecuyer-2004-FBH

A cool technique on "hacking" human perception by modifying the control/display ratio of visible elements to simulate haptic feedback for the user. Strong analysis of which parts of haptic feedback are useful (e.g., vertical elements can be discarded). Pseudo-haptic feedback is implemented by combining the use of visible feedback with the changing sensitivity of a passive input device (e.g., a mouse). E J Kalafarski 16:02, 26 January 2009 (UTC)

• Research issues in perception and user interfaces Encarnacao-1994-RIP

"The authors focus at on three things: presentation of information to best match human cognitive and perceptual capabilities, interactive tools and systems to facilitate creation and navigation of visualizations, and software system features to improve visualization tools." First and third points sound relevant. E J Kalafarski 16:02, 26 January 2009 (UTC)

• The Uniqueness of Individual Perception Whitehouse-1999-ID

Focuses on the commonalities of perception. Rough overview of sensory mechanisms, and strong anecdotal support of not adapting completely to the user, but rather requiring the user to adapt as well. Identifies some common perceptual problems with particular groups of EUs (e.g., blind people). E J Kalafarski 16:02, 26 January 2009 (UTC)

• Guided search 2. 0. A revised model of visual search Wolfe-1994-GS2

A theory of visual search that builds on the distinction between visual targets that you need to search for in a field of distractors and those that "pop out" at you. (Adam) 16:22, 26 January 2009 (UTC)

• Perceptual causality and animacy Scholl-2000-PCA

Discusses some of the automatic interpretation in our perception, focusing on inferring causal relations and animacy.

• Technology Affordances Gaver-1991-TAF

Affordances are actions that are appropriate for an object and that come to mind when perceiving the object. (Adam) 16:22, 26 January 2009 (UTC)

• Affordance, conventions, and design Norman-1999-ACD

How the original concept of affordances differs from how it has been used in HCI. (Adam) 16:22, 26 January 2009 (UTC)

• The Ecological Approach to Visual Perception Gibson-1986-EAV

Outlines direct perception and the original theory of affordances. (Jon) 14:07, 3 February 2009.

• Ecological interface design: Theoretical foundations Vicente-1992-EID

Theory of how interfaces can avoid forcing processing at a higher level than the task requires. (Jon) 14:56, 3 February 2009.

• The Ecology of Human-Machine Systems II: Mediating Direct Perception in Complex Work Domains Vicente-2000-EHM

Taking advantage of fast perceptual processes to reduce cognitive demands as applied to the design of a thermal-hydraulic system. (Jon) 14:56, 3 February 2009.

• Acting on a visual world: The role of multimodal perception in HCI. Wolff-1998-AVW.

Experiment that has implications for gesture interpretation module development.

• Effects of motor scale, visual scale, and quantization on small target acquisition difficulty Chapuis-2011-EMS

In the first class, we talked about the problem with Windows Start menus -- how hard it is to navigate and select the right one. This study provides empirical evidence for this problem, confirming the difficulty of acquiring small-sized targets (like the menus) and identifying motor and visual sizes of the targets as limiting factors. (Nathan Malkin, 12 September 2011)

• Do predictions of visual perception aid design? Rosenholtz-2011-DPV

This paper asks the question: does the use cognitive and perceptual models actually help (in this case, the process of design)? They find that "the models can help, but in somewhat unexpected ways": ""goodness" values were not very useful" but it "seemed to facilitate communication ... about design goals and how to achieve those goals". (Nathan Malkin, 12 September 2011)

• Crowdsourcing Graphical Perception: Using Mechanical Turk to Assess Visual Design, Heer and Bostock, CHI '10

This paper explores crowdsourcing as a viable method for conducting visualization perception evaluations. They replicate some results of Cleveland and McGill's 1984 graphical perception paper, and do some analysis on cost and performance of using MTurk for these studies on static, chart-type visualizations. (Steven Gomez)

Cognition

• Colin Ware: Visual Thinking: For Design

Insight into some of the theory of cognition as it pertains to building visual interfaces (David)

• Wikipedia's Seven Plus or Minus Two page

A clear description of one part of human thinking; will probably provide pointers to other things to read (David)

• Wikipedia's Hick's Law page

Hick's law describes the relationship between the decision-making time and the number of possible choices. (Hua)

• A cognitive model for the perception and understanding of graphs Lohse-1991-CMP

Describes a computer program that predicts response time to a query from assumptions from eye-tracking, short-term memory capacity, and the amount of information that can be absorbed from the query in each "glance." Attempts to lay the foundation for explaining several steps of human cognition, including input, memory, and processing. E J Kalafarski 16:02, 26 January 2009 (UTC)

• Cognitive load during problem solving: Effects on learning John Swiller

Older article but referenced in a lot of newer ones; looks at how conventional problem-solving is ineffective as a learning device. (lisajane)

• Dimensional overlap: Cognitive basis for stimulus–response compatibility—A model and taxonomy Kornblum-1990-DOC

People are more effective at a task when the stimulus and response representations are compatible and they don't require "translation". (Adam) 16:22, 26 January 2009 (UTC)

• Tracking Neuropsychological Recovery Following Concussion in Sport

This paper discusses the neurological basis for the ImPact test given to athletes after they've suffered a concussion. It provides testing and quantitative measures for verbal memory, visual memory, and reaction times. These simple measures of cognition may be useful to incorporate in an HCI study. (Trevor)

A Framework of Interaction Costs in Information Visualization

ABSTRACT: Interaction cost is an important but poorly understood factor in visualization design. We propose a framework of interaction costs inspired by Norman’s Seven Stages of Action to facilitate study. From 484 papers, we collected 61 interaction-related usability problems reported in 32 user studies and placed them into our framework of seven costs: (1) Decision costs to form goals; (2) System-power costs to form system operations; (3) Multiple input mode costs to form physical sequences; (4) Physical-motion costs to execute sequences; (5) Visual-cluttering costs to perceive state; (6) View-change costs to interpret perception; (7) State-change costs to evaluate interpretation. We also suggested ways to narrow the gulfs of execution (2–4) and evaluation (5–7) based on collected reports. Our framework suggests a need to consider decision costs (1) as the gulf of goal formation.

Includes some ideas for quantitatively evaluating information visualization interfaces (David)

• Distributed Cognition as a Theoretical Framework for HCI (1994) Christine A. Halverson [1]

• The Extended Mind Clark-1998-TEM

Cognition can be thought to be distributable across mediums (outside of the skull). How might we off-load "cognitive" processes to computer systems? (Gideon - Owner)

I think that Ware gets into this in some of his writing about information visualization (or in his second book, thinking with visualization). We can build in external "caches" or other constructs to be part of our cognitive model. It seems like most of an analytical user interface is part of the external cognitive process. (David)

• Sources of Flexibility in Human Cognition: Dual-Task Studies of Space and Language HermerVazquez-1999-SFC

Our use of language serves as a higher-order cognitive system which can be utilized as "scaffolding" in human thought, supporting goal-driven tasks. (Gideon)

• In two minds: dual-process accounts of reasoning Evans-2003-ITM

It is hypothesized that there are two distinct systems of reasoning in the mind. System 1 is innate and fast, system 2 is controlled and slow. Knowledge of this might help us determine which tasks are candidates for one system or another. (Gideon)

• Priming against your will: How accessible alternatives affect goal pursuit Shah-2002-PAY

The authors demonstrate how priming the means to achieving a goal also primes the goal, but inhibits alternative means to achieving the same goal. It means that making the means of achieving a goal salient in an interface will make it more likely that people pursue that goal, and less likely that they will think of other means to pursue it. (Adam)

• Computational visual attention systems and their cognitive foundations: A survey Frintrop-2010-CVA

This paper "provides an extensive survey of the grounding psychological and biological research on visual attention as well as the current state of the art of computational systems". It should make for good background reading if we want to work with visual attention (detecting regions of interest in images). (Nathan Malkin, 12 September 2011)

Owner: Nathan Malkin

• Getting inspired!: understanding how and why examples are used in creative design practice Herring-2009-GIU

A user study on the use of examples to improve creativity. Results show that examples are very useful to inspire designers of new ideas. Surprisingly, inspiring examples are not limited to the ones in the design domain, but are expanded to other areas too.

• Contemporary cognitive load theory research: The good, the bad and the ugly Kirschner-2010-CCL

This review summarizes and critiques 16 papers on cognitive load theory (CLT) and its impact on learning and ability to navigate different environments. It also discusses the difficulties inherent to the study of cognitive and moves made to attack them. While the paper does not contribute directly to our research, it does provide background on some of the issues of usability and "tolerance" of HCI systems that we discussed last week. Clara Kliman-Silver 13:54, 18 September 2011 (EDT)

• Models of attention in computing and communication: from principles to applications Horvitz-2003-MAC

Talks about efforts to make UIs "aware" of their user's ability to attend and comprehend. Jenna Zeigen 10:50, 19 September 2011 (EDT)

• The construction of preference

• The role of memory and restricted context in repeated visual search Kunar-2008-RMR

Why don't people who have to perform repeated visual search (searching through an unchanging display for hundreds of trials) use their memory to speed up their tasks? Several experiments reported in this paper "show that participants choose *not* to use a memory strategy because, under these conditions, repeated memory search is actually less efficient than repeated visual search, even though the latter task is in itself relatively inefficient." However, if you restrict where in the image the target stimuli may appear, using memory becomes more efficient.

(Nathan Malkin, 19 September 2011)

• A Theory of Graph Comprehension Steven Pinker

Caroline Ziemkiewicz 17:52, 19 September 2011 (EDT)

• On Distinguishing Epistemic from Pragmatic Actions Kirsh and Maglio

Caroline Ziemkiewicz 17:52, 19 September 2011 (EDT)

• Galvanic skin response (GSR) as an index of cognitive load Shi-2007-GSR -- Jenna Zeigen

Discusses how GSR can be used to evaluate users' stress and arousal levels while using different forms of an interface. This is relevant to the reviewers comments that there lacked a plan to evaluate how well the visualizations would jive with cognitive principles. If we are using them properly, then stress and cognitive load levels would be low.

• Automatic cognitive load detection from speech features Yin-2007-ACL -- Jenna Zeigen

Again, discusses a way to evaluate a user's cognitive load. Again, this is relevant to the reviewers comments that there lacked a plan to evaluate how well the visualizations would jive with cognitive principles. If we are using them properly, then cognitive load levels would be low.

• Cognitive Task Analysis Schraagen et.al

"Cognitive task analysis is a broad area consisting of tools and techniques for describing the knowledge and strategies required for task performance. The topics to be covered by this work include: general approaches to cognitive task analysis, system design, instruction, and cognitive task analysis for teams." I think this book may be relevant in that some of the cognitive tasks analysis methods in the book may serve as reference when designing cognitive performance evaluation benchmark tests for the proposed brain visualization system, but I am not too sure. Hua

HCI

• A diary study of mobile information needs

A detailed study into how people use mobile devices. (Andrew Bragdon - OWNER for Assignment 2)

• Feasibility and pragmatics of classifying working memory load with an electroencephalograph

Examines how practical it is to use electroencephelographs to measure cognitive load, and discusses domain-specific knowledge needed.

• Dynamic Detection of Novice vs Skilled Use

Used a learning classifier, trained on low-level mouse and keyboard usage patterns, to identify novice and expert use dynamically with accuracies as high as 91%. This classifier was then used to provide differrent information and feedback to the user as appropriate.

• Bubble Cursor

Example of a paper which demonstrated a novel interaction technique still obeys Fitts's law.

• Sloppy Selection

Utilized a quantiative model of user performance which used curvature to predict the speed of a pen as it moved across a surface to help disambiguate target selection intent.

• Disruption and Recovery of Computing Tasks

Studied task disruption and recovery in a field study, and found that users often visited several applications as a result of an alert, such as a new email notification, and that 27% of task suspensions resulted in 2 hours or more of disruption. Users in the study said that losing context was a significant problem in switching tasks, and led in part to the length of some of these disruptions. This work hints at the importance of providing affordances to users to maintain and regain lost context during task switching.

• A diary study of task switching and interruptions

Showed that task complexity, task duration, length of absence, and number of interruptions all affected the users' own perceived diffuclty of switching tasks. (Andrew Bragdon)

• John M. Carroll: HCI Models, Theories, and Frameworks: Toward a Multidisciplinary Science

A gargantuan book with chapters by many folks describing some of the models and theories from HCI that may relate back to cognition; may need to create individual (David)

• Project ernestine: validating a GOMS analysis for predicting and explaining real-world task performance

A study in which the GOMS method is used to correctly predict the performance of call center operators using a new workstation. Might be interesting because of the methodology used to decompose the task into basic cognitive and perceptual actions, and then measuring these actions to evaluate the new interface. (Eric) The CPM (Critical Path Modeling) aspect used handles the parallel nature of several human components of HCI and seems to very accurately model the low level tasks from this study. (David)

• The Growth of Cognitive Modeling in Human-Computer Interaction Since GOMS

• The limits of expert performance using hierarchic marking menus

Marking menus naturally facillitate the transition from novice to expert performance for command invocation, and have been quite influential over the years to research into menu techniques. (Andrew Bragdon)

• Manual and gaze input cascaded (MAGIC) pointing

This is a system which combines gaze input (coarse-grained) and mouse input (fine-grained) to quickly target items. This is important because it "kind of" gets around Fitt's law by using gaze input to "warp" the cursor to the general vicinity of what the user wants to work on. (Andrew Bragdon)

• If not now, when?: the effects of interruption at different moments within task execution Adamczyk-2004-INN

Presents task models of user attention. (Andrew Bragdon) (Adam - owner; Gideon - discussant) DISCUSSANT: E J Kalafarski 22:58, 28 January 2009 (UTC)

• Ecological Interface Design: Progress and challenges Vicente-2002-EID

Discusses the implications of Ecological Interface Design (EID), a theoretical HCI framework, for designing human-computer interfaces and compares performance of EID-informed designs to other comtemporary approaches. (Owner: Jon)

• "Constant, constant, multi-tasking craziness": managing multiple working spheres. Gonzalez-2004-CCM

Empirical study of how information workers spend their time. Puts forward a theory of how users organize small individual tasks into "working spheres." (Andrew Bragdon - OWNER; Adam Darlow - Discussant; Steven Ellis - Discussant)

Any visualization which is used extensively by a user over a period of time will be used in the context of that user's daily workflow. It is therefore essential to understand this larger workflow context to design the visualization application appropriately to fit the needs of real world users. This paper studies in detail the daily workflow tasks and patterns of work of analysts, managers and software developers in a medium-sized software company. This paper provides strong empirical evidence that users, rather than working on discrete and well-defined tasks, in reality, switch tasks on average every two to three minutes, and instead, work on larger thematically connected units of work (working spheres). In addition, the study found that users switched between these larger working spheres on average every 12 minutes. Thus, it is strongly indicated by this paper that many information workers are in a constant state of rapid fire multi-tasking. This suggests that for a visualization to be relevant to any of these information workers, it would need to fit into, and support, this workflow. This is just a first step towards understanding how users interact with visualizations in particular, however; future work that studies how users interact with visualizations as part of their larger daily work patterns is warranted, and would be an important component of a broad theory of visualization.

• The state of the art in automating usability evaluation of user interfaces Ivory-2000-SAA

Presents a new taxonomy for automating usability analysis. Advantages of automated evaluation are purported to be advantages linked to efficiency, such as comparing alternate designs, uncovering more errors more consistently, and predicting time/error costs across an entire design. Breaks down a taxonomy with individual benefits and drawbacks of each method, and checks observations against existing guidelines (e.g. Smith and Mosier guidelines, Motif style guidelines, etc). Introduces several visual tools. Looks extremely relevant as a comprehensive survey of existing techniques. OWNER: E J Kalafarski 16:02, 26 January 2009 (UTC) Discussant: --- Trevor O'Brien 23:22, 28 January 2009 (UTC) Discussant: Steven Ellis

• User Interface Evaluation in the Real World: A Comparison of Four Techniques Jeffries-1991-UIE

Overview of the four major UI evaluation methods: heuristic evaluation, usability testing, guidelines, and cognitive walkthrough, followed by a comparison in their application to a case study. E J Kalafarski 16:02, 26 January 2009 (UTC)

• Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces Polson-xxxx-CWM

Presents the concept of performing a hand walkthrough of the cognitive process, based on another theory of "learning by exploration." Strong results for a limited evaluation timeframe and little or no time for formal instruction of the interface for the user. The reviewer considers each behavior of the interface and its resultant effect on the user, attempting to identify actions that would be difficult for the "average" user. Claims that a given step will not be difficult must be supported with empirical data or theory. The application of cognitive theory early in the design process seems useful in avoiding costly redesigns when problems are identified later. E J Kalafarski 16:02, 26 January 2009 (UTC)

• Finding usability problems through heuristic evaluation Nielsen-1992-FUP

Emphasis on heuristic evaluation. Shockingly, usability experts are found to be better at performing this type of evaluation. Usability problems relating to elements that are completely missing from the interface are difficult to identify with this method when evaluating unimplemented designs. E J Kalafarski 16:02, 26 January 2009 (UTC)

• The Use of Eye Movements in Human-Computer Interaction Techniques: What You Look at is What you Get Jacob-1991-UEM

One of the first research papers to introduce eye tracking as a viable HCI technique. (Trevor)

• Real-Time Eye Tracking for Human Computer Interfaces Amarnag-2003-RTE

Technical details about the implementation of a recent real-time eye-tracking system. (Trevor)

• Semantic Web HCI: Discussing Research Implications Degler-2007-SWH

A workshop discussion from CHI 2007 discussing the idea of a "semantic internet" and its relevance to the HCI community. Discusses things like adaptive web interfaces, mashups, dynamic interactions, etc. (Trevor)

• Implicit Human Computer Interaction Through Context Schmidtt-2000-IHC

A highly cited paper discussing the notion of implicit HCI, including semantic grouping of interactions, and some perceptual rules. (Trevor - OWNER; Andrew Bragdon - discussant; DISCUSSANT: E J Kalafarski 22:57, 28 January 2009 (UTC))

• Cognitive Strategies for the Visual Search of Hierarchical Computer Displays Anthony J. Hornof

This article investigates the cognitive strategies that people use to search computer displays. Several different visual layouts are examined. (lisajane)

• Unseen and Unaware: Implications of Recent Research on Failures of Visual Awareness for Human-Computer Interface Design D. Alexander Varakin; Daniel T. Levin; Roger Fidler

This article reviews basic and applied research documenting failures of visual awareness and the related metacognitive failure and then discuss misplaced beliefs that could accentuate both in the context of the human-computer interface. (lisajane)

• Shneiderman, Plaisant: Designing the User Interface

My textbook for an HCI class, has many good lists of guidelines. Especially Ch.2 pp 59-102. (lisajane)

• Robert Mack, Jakob Nielsen: Usability Inspection Methods (Ch. 1 Executive Summary)

Provides an overview of main usability inspection methods, a fair introduction to the industrial applications, as well as certain costs and benefits, of the methods as well as suggestions for expansive research. (Steven)

• Jock Mackinlay, Automating the design of graphical presentations of relational information, ACM Transactions on Graphics (TOG), 5(2):110-141, 1986. (Jian)

The first paper talked about how to automatically generate *good* graphs.

• Jock Mackinlay, Pat Hanrahan, Chris Stolte, Show Me: Automatic Presentation for Visual Analysis IEEE TVCG 13(6): 1137-1144, Nov-Dec, 2007 (Jian)

Extend their previous paper to analytics tasks.

• Jarke J. van Wjik, The value of visualization, IEEE visualization 2005. (Jian)

discuss vis from a variety of angles as for art, science and technology and question and quantify the utility of visualization.

• Johnny Accot and Shumin Zhai, Beyond Fitts' law: models for trajectory-based HCI tasks, CHI 97. (Jian)

Extend Fitts' to trajectory based tasks.

• Saraiya, P., North, C., Lam, V., and Duca, K.A, An Insight-Based Longitudinal Study of Visual Analytics, TVCG 12(6): 1511-1522, 2006.(Jian)

the first paper that quantifies what insight is by comparing several infoVis tools for bioinformatics.

• An Insight-Based Methodology for Evaluating Bioinformatics Visualizations, Saraiya, North, and Duca, TVCG 2005

• David H. Laidlaw, Michael Kirby, Cullen Jackson, J. Scott Davidson, Timothy Miller, Marco DaSilva, William Warren, and Michael Tarr.Comparing 2D vector field visualization methods: A user study. IEEE Transactions on Visualization and Computer Graphics, 11(1):59-70, 2005. (Jian)

an application-specific comparison of visualization method. a cool paper.

• Rosenholtz, Li, Mansfield, and Jin, Feature Congestion: A Measure of Display Clutter, CHI 2005. (Jian)

quantify visual complexity from a statistics point of view.

• The GOMS Family of User Interface Analysis Techniques: Comparison and Contrast (Trevor)

This paper offers an analysis of four types of GOMS (Goals, Objects, Methods and Selection) based interaction techniques. GOMS is a widely used UI paradigm, made popular by Card et al in The Psychology of Human-Computer Interaction (1983).

• Automatic Facial Expression Interpretation: Where Human-Computer Interaction, Artificial Intelligence and Cognitive Science Intersect (Trevor)

Using advanced computer vision/AI techniques, this work aims to discern and make use of users' emotions in UI design.

• Automatically Personalizing User Interfaces (Trevor)

Discusses some techniques and design decisions for constructing adaptable and customizable user interfaces. There are some useful references in the paper on using HMMs and RMMs (Relational Markov Models) for interaction prediction.

• Exploring the Design Space for Adaptive Graphical User Interfaces (Trevor)

This paper presents comparative evaluations of three methods for implementing adaptable user interfaces. The evaluation methodology gives rise to three key concepts that affect the performance of adaptable UIs: frequency of adaptation, accuracy of adaptation, and the impact of predictability.

• Conceptual Modeling for User Interface Development - David Benyon, Diana Bental, and Thomas Green

Proposes a new set of terminology for describing and comparing existing and future cognitive models of HCI. (Steven)

• The Skull beneath the Skin: Entity-Relationship Models of Information Artefacts T. R. G. Green, D. R. Benyon

A paper in form of prelude to the above, gives a good overview of the ERMIA method. (Steven)

• Wizard of Oz for Participatory Design: Inventing a Gestural Interface for 3D Selection of Neural Pathway Estimates, Akers-2006-WOP.

Designs an interactive visualization interface for 3D selection of neural pathways of human brains. The mouse based interface helps neuroscientists to select neural pathways more efficiently and intuitively. (--- Chen Xu 15:04, 13 September 2011 (EDT) -OWNER)

• Could I have the Menu Please? An Eye Tracking Study of Design Conventions McCarthy-2003-CMP

This article examines eye tracking techniques as they pertain to improving search performance in user interfaces. Specific attention is given to menu organization in the context of Web interfaces. Although substantial progress has been made in the past decade, the article draws attention to relevant design issues and concepts, especially as eye tracking methodologies continue continue to grow and improve. (Clara, 11 September 2011--OWNER; Chen -- Discussant)

• Audio-Visual Intent-To-Speak Detection For Human-Computer Interaction, Cuetos-2000-ISD.

Discusses a speech detection system that uses both auditory and visual cues to more accurately detect speech commands. It aims to recognize the user's intention to speak, and to ignore background noise, or speech recognized as not being directed at the system. Although it is fairly dated, this paper is relevant in that it discusses applications of cognition/perception to HCI. Michael Spector 13:19, 13 September 2011 (EDT)

• An exploration of relations between visual appeal, trustworthiness and perceived usability of homepages Lindgaard-2011-ERB

This paper is interesting and relevant to cognition+HCI because it attempts to differentiate between "judgments differing in cognitive demands (visual appeal, perceived usability, trustworthiness)" and see whether those tasks with more cognitive demand have different results. (The paper includes a model to account for these.)

Also, this is interesting to Steve and me in the context of some of our discussions this past spring. (Apparently, yes: "all three types of judgments [including, crucially, trustworthiness] are largely driven by visual appeal".) (Nathan Malkin, 12 September 2011)

• Activity Theory: Implications for Human-Computer Interaction Kaptelinin--1996-ATI

This article discusses activity theory, an alterate to present theories surrounding HCI. In particular, it examines the principle differences between activity theory and cognitive theory, applies it to HCI, and suggests implications for the field. While not directly relevant to the proposal, it offers an alternate framework for some of the issues that we discuss. (Clara, 12 September 2011)

• Let's Get Real: a Position Paper on the Role of Cognitive Psychology in the Design of Humanely Usable and Useful Systems Landauer-1991-LGR

Perhaps less useful, only because it's 20 years old, but an interesting read nonetheless: this paper questions the "modern" relevance of cognitive psychology to human-computer interaction design. The primary issue, it argues, is that human-computer systems are entirely unpredictable, and thus, some of the modern understanding of cognition (and, indeed, HCI theory) simply cannot apply given the erratic behavior of computer systems. Instead, he addresses some of the more "useful models," including Fitt's law and theories of visual perception, to define a new space for emerging research in HCI. (Clara, 12 September 2011)

• Mid-air pan-and-zoom on wall-sized displays Nancel-2011-MPZ

The paper describes approaches to perform pan and zoom tasks in mid-air: bimanual & unimanual, linear & circular gestures.

• LiquidText: a flexible, multitouch environment to support active reading Tashman-2011-LER

A technique utilizing multitouch to improve reading efficiency.

• Rethinking 'multi-user': an in-the-wild study of how groups approach a walk-up-and-use tabletop interface Marshall-2011-RMU

An ethnographic study that explores how groups of users approach tabletop interfaces in real environments. Some results contradict existing findings.

• Demonstrating the Feasibility of Using Forearm Electromyography for Muscle-Computer Interfaces Saponas-2008-DFU

Discusses the merits of HCI (here called muCI, for muscle-computer interaction) by detection of forearm muscle activity rather than manipulation of an object such as a mouse or keyboard. Michael Spector 13:19, 13 September 2011 (EDT)

• HUMAN-COMPUTER INTERACTION: Psychological Aspects of the Human Use of Computing Olson-2003-HCI

Overview of issues in psychology in HCI (Jenna Zeigen, 9/12/11)

• Integrating cognitive load theory and concepts of human–computer interaction Hollender-2010-ICL

This paper compares existing models of cognitive load theory as it applies to HCI, reviews present literature, and discusses current problems and potential advances. Relevant to our work but ventures into theory that is heavier than necessary given our purposes. (Clara Kliman-Silver 15:58, 18 September 2011 (EDT))

• Gesture Recognition : A Survey IEEE

This article provides a survey on gesture recognition with particular emphasis on hand gestures and facial expressions. Applications involving hidden Markov models, particle filtering and condensation, finite-state machines, optical flow, skin color, and connectionist models are discussed in detail. (Wenjun Wang , 18 September 2011 (EDT))

• A person–artefact–task (PAT) model of flow antecedents in computer-mediated environments Finneran-2003-PAT

Re-evaluates flow theory within the HCI framework and proposes a model that fits best in the field Jenna Zeigen 11:50, 19 September 2011 (EDT)

• Now Where Was I? Psychologically-Triggered Bookmarking, Pan et al., CHI '11

Presents an interaction paradigm for implicitly bookmarking application progress/media during user interruptions (e.g., phone ringing) using galvanic skin response (GSR) to identify interruptions, or the orienting response (OR), automatically. The authors evaluate how well GSR works for identifying user ORs, and describe a few experiments using an audiobook listener application that creates bookmarks (stored and represented in a GUI) automatically when GSR peaks in response to controlled stimuli. In our project, we're looking at predicting user states (or effect on task performance), so this is an example of that kind of predictive affect-tracking that has been engineered into a usability feature. (Steven Gomez 17:12, 19 September 2011 (EDT))

• Ripples: Utilizing Per-Contact Visualizations to Improve User Interaction with Touch Displays, Wigdor-2009-PCV

Demonstrates a system for visual feedback on touch screen interfaces, designed to reduce the ambiguity that can arise in the absence of feedback. The system changes the feedback based on the task at hand and claims to provide information in an intuitive way as to how the touch screen is registering the inputs. Video demonstration Michael Spector 17:30, 19 September 2011 (EDT)

• Synchronous interaction among hundreds: an evaluation of a conference in an avatar-based virtual environmentCHI-2011

This paper presents the first in-depth evaluation of a large multi-format virtual conference. The conference took place in an avatar-based 3D virtual world with spatialized audio, and had keynote, poster and social sessions. (Wenjun Wang)

Cognitive Modeling

• Activity Theory: Implications for Human-Computer Interaction (Trevor)

Discusses the notion of Activity Theory as the basis for HCI research. The most interesting part of this paper for me was the introduction which expressed the need for a Theory of HCI.

• Norman's Seven stages of action

Presents Norman's seven stages of action, as well as his model of evaluation. (Steven)

• Analyzing Human-Computer Interaction as Distributed Cognition: The Resources Model (Trevor)

Creates a compelling argument for why distributed cognition research fits in with HCI, and what types of impacts it may have on the HCI community.

• Eye Tracking in Human-Computer Interaction and Usability Research: Ready to Deliver the promises Jacob-2003-ETH

This paper (book chapter) looks beyond the relevance of eye tracking methodologies to HCI and instead addresses the data produced. It examines various approaches to analysis and the implications and conclusions that can be drawn. Given that eye tracking is often coupled with other inputs, such as a mouse or a keyboard, analysis is rarely clear-cut: other variables, such as error, saccades, and speed must be factored in. Moreover, eye movements are far less deliberate than mechanical (i.e. mouse) input, and so errors must be handled differently. The chapter discusses each of these issues and subsequently offers solutions. In general, the article argues for the importance of eye tracking, considering it as a central component of HCI methodology. (Clara, 11 September 2011)

• Studying Context, A Comparison of Activity Theory, Situated Action Models, and Distributed Cognition Bonnie A. Nardi

Defines the task of the HCI specialist as the application of psychological and anthropological principles to specific design problems. It posits an inherent feud between the accurate study of relative contexts and the necessary, but more general, development of comparative models and results. Gives a coherent overview of activity theory, situated action models, and distributed cognition; finds that activity theory presents the best overall framework. There is little reason given for this ranking, however, and the description of activity theory is the most theoretical and least developed of the three.

Having spent quite a bit of time studying Soviet psychology (from which came activity theory) last semester, I question the validity of the paper’s claim, as its description of activity theory bears the artifacts of the oppressive regulations which the Soviet government imposed on psychologists. Although the theory may sound more practical, it seems fairly weak as a basis for empirical design analysis.

The paper’s strongest point is the criticisms which follow descriptions, in which theoretical shortcomings of each perspective are discussed. (Owner: Steven, Discussant: --- Trevor O'Brien 23:22, 28 January 2009 (UTC))

• Chunking and Phrasing and the Design of Human-Computer Dialogues

High-level theory of human-computer dialogues. (Andrew Bragdon)

• Polson, P. and Lewis, C. Theory-Based Design for Easily Learned Interfaces. Human-Computer Interaction, 5, 2 (June 1990), 191-220.

This is a cognitive model of how users find and learn commands in an unfamiliar user interface. This could potentially be adapted to be a piece of a theory of visualization. (Andrew Bragdon)

• Activity Theory vs Cognitive Science in the Study of Human-Computer Interaction

Provides a brief history of Cognitive Science and HCI, then compares the effectiveness of the aforementioned theories in aiding design and development. (Owner - week 2 : Steven)

• International Journal of Human-Computer Studies - Special Issue on Cognitive Modeling

Articles all concerning various issues of cognitive modeling as relates to HCI. (Steven)

• The role of cognitive theory in human–computer interface. Chalmers, Patricia A, 2003.

Was scared again, but no need to be. Touches only on a subset of cognitive theories (Schema theory, Cognitive load, and retention theories) and undertakes a survey of some software design theories, but does not attempt an explicit mapping between the two. E J Kalafarski 13:48, 20 February 2009 (UTC)

• Design Guidelines. Marshall, Nelson, and Gardiner, 1987.

Attempt to apply cognitive psychology to user-interface design. Here, the opposite problem is seen—the author's make no significant attempt to take existing heuristic guidelines into account. E J Kalafarski 13:48, 20 February 2009 (UTC)

• Cognitive Design Solutions, Inc.

Training and consulting firm that claims to take advantage of Cognitive Design in making design and performance improvements. E J Kalafarski 13:50, 20 February 2009 (UTC)

• Actability Principles in Theory and Practice. Agerfalk, Par J, 2003.

Presents a set of nine contemporary principles for the evaluation of IT systems ("social tools to perform communicative action") based explicitly on cognitive principles. Introduces a notion comparable to usability called actability. Presents a mapping for some basic usability principles to some seminal sets of guidelines. E J Kalafarski 14:29, 20 February 2009 (UTC)

• Wikipedia page for ACT-R

ACT-R is a cognitive architecture developed at CMU. It aims to define the basic cognitive and perceputal operations of human mind. (Hua)

• Wikipedia page for Soar

Soar is another cognitive architecture developed at CMU, now maintained in UMich. (Hua)

• Brain-Computer Interfaces and Human-Computer Interaction

(Not sure what heading this ought to go under!) Brain-Computer Interfaces (BCI) offer a neurological corollary to human-computer interfaces: users use their thoughts to signal to machines, instead of relying on physical movements. Thus, the areas activated are purely cognitive, not motor. The article provides an overview of the differences between HCI and BCI, the implications thereof, and the directions that their interaction may take. Relevant to some of the issues and concepts raised in the proposal, in addition to being a rather interesting idea! (Clara, 12 September 2011)

• Spanning Seven Orders of Magnitude: A Challenge for Cognitive Modeling, John Anderson, 2002

The paper argues that high-level human behavior can be understood by analyzing the chain of fast, low level activity (from 10ms up) in the perceptual/cognitive bands that compose larger behaviors. It gives an intro to ACT-R and variants and some compelling examples for cognitive modeling and eye-tracking. (Steven Gomez)

• Cyberpsychology: A Human-Interaction Perspective Based on Cognitive Modeling Emond-2003-CHI

This paper talks about for the applicability of cognitive modeling to cyberpsychology, the study of the impact of computer and Internet interaction on humans. (Jenna Zeigen, 9/12/11)

• Cognitive Computing, Modha-2011-CC

Unite neuroscience, supercomputing, and nanotechnology to discover, demonstrate, and deliver the brain’s core algorithms. (--- Chen Xu 17:25, 13 September 2011 (EDT) -OWNER)

• Measuring effectiveness of graph visualizations: a cognitive load perspective Huang-2009-JIV

(This item can also go into the Evaluation category) This paper discusses the cognitive load theory and its application in measuring effectiveness of graph visualization. A model of user task performance, mental effort and cognitive load has been proposed and experiments have been conducted to refine the model. This seems to be an attempt along the line of defining quality metrics for visualization through cognitive modeling, which then closely relates to our proposal. (Hua)

• Supporting Collaboration and Distributed Cognition in Context-Aware Pervasive Computing Environments Fischer-2004-SCD

(Not exactly sure where this paper belongs.) The paper looks at several issues in HCI: 1) collaborative design (multiple users accessing, manipulating, and addressing information, in what they call "large computational spaces"), 2) mobile technology (mobile phones, wireless, etc.), and 3) smart objects (seems to be largely mobile phones and similar devices). This paper is dated and parts of it are very interesting but equally irrelevant. Nonetheless, it asks some important questions about how to deliver information to the user, how to manage search techniques and memory systems (particularly with searching) in HCI, and how to access information, all of which are crucial to "modern" HCI research. (Clara Kliman-Silver 17:15, 18 September 2011 (EDT))

• Attention, Habituation and Conditioning: Toward a Computational Model Balkenius-2000-AHC
  • "The central claim of this article is that attention can be controlled in the same way as actions using similar learning mechanisms and by related areas of the brain."
  • "A computational model of attention is presented that uses habituation as well as classical and instrumental conditioning to explain a number of attentional processes."
  • "Computer simulations are presented that illustrates the operation of the model."

(Nathan Malkin, 19 September 2011)

• CAEVA: Cognitive Architecture to Evaluate Visualization Applications Juarez-Espinosa, 2003

(May also be categorized in Evaluation) This paper, indicated by the author, is the first paper that uses cognitive modeling to evaluate computer application. This paper presents a rather high-level picture for a proposed cognitive architecture that is intended to be used to evaluate visualization applications. The proposed architecture has two main components: interoperability model and cognitive model. The cognitive model simulates a human interacting with the visualization system. It has two components, one on domain-dependent knowledge and one on domain-independent knowledge. The knowledge again is divided into two categories: declarative knowledge and procedural knowledge. The interoperability model describes how the cognitive model communicates with the visualization application.

This paper does not provide detailed implementation or experiment with the proposed cognitive architecture. It seems like the proposed cognitive architecture is not fully implemented to the extent of fully functioning when the paper is published.

(Hua, Sep. 19, 2011)

• Implementing a Cognitive Model in Soar and ACT-R: A Comparison Muller

This paper presents an implementation of a cognitive model in the cognitive architecture Soar. It also presents a comparison of this implementation with a previous implementation in ACT-R. The cognitive task is not quite relevant to our research; but it may help to get an idea of what it is like to implement a cognitive task in ACT-R/Soar. (Hua, Sep. 19, 2011)

• Computational Model of Facial Attractiveness Judgements Bronstad-2008-CMF

Talk about the computational models of facial attractiveness judgements, one model uses partial least squares to identify facial images and attractive ratings, the second model uses manually derived measures of facial features as input. Results are discussed. The paper also concludes that averageness and sexual dimorphism are important for facial attractiveness judgements.(Chen)

• Visual Reasoning Model as an Analysis Tool for Different Tasks Related to Design Abilities Kim-2009-VRM

Describe how the visual reasoning model could be used to identify characteristics in various tasks related to design ability such as missing view problem task, mental synthesis task, and conceptual design task. (Chen)

• Examining the feasibility of a case-based reasoning model for software effort estimation Mukhopadhyay-1992-EFC

A case-based reasoning model, called Estor, was developed based on the verbal protocols of a human expert solving a set of estimation problems.(Chen)

• COSIMO: A cognitive simulation model of human decision making and behavior in accident management of complex plants Cacciabue-1992-SMC

"A cognitive simulation model (COSIMO) which simulates the behavior of an operator controlling a complex system during the management of accidents is described." (Hua)

• Visual variability analysis for goal models Gonzales-2004-VVA

This work proposes a visual variability analysis technique and describes an implemented tool that supports it. We can investigate how goal maintaince can be put into practice. (Chen)

• The cognitive and neuroanatomical correlates of multitasking Burgess-2000-CNC

As a reviewer points out, the pre-proposal section dealing with "goal maintenance" is vague and low on specifics. While this paper does not suggest a specific solution, we can use its findings ("that there are three primary constructs that support multitasking: retrospective memory, prospective memory, and planning") to come up with specific techniques to help with individual correlates.

(Nathan)

• Sensing cognitive multitasking for a brain-based adaptive user interface Solovey-2011-SCM

Addressing the same reviewer comment, this paper reports the results of specific experiments in differentiating cognitive multitasking processes. We could cite this as something we would build on in our project, since "this prototype system [in the paper] serves as a platform to study interfaces that enable better task switching, interruption management, and multitasking", and that's exactly what we're trying to build.

(Nathan)

• Embodied models as simulated users: introduction to this special issue on using cognitive models to improve interface design Ritter-2002-EMS

A reviewer worries, "The primary risk in this proposal is that the cognitive models which can be created are too weak to support the design process." We can provide evidence that this risky part is feasible by pointing to the findings cited in this article (and others from the issue that the title refers to) of successful uses of cognitive models for developing and designing interfaces.

(Nathan)

Design

• Wikipedia articles on "A Pattern Language" and "Design Pattern"

see summary for Alexander below (David)

• UI Design principles (feedback, etc -- find ref)

• Alexander: A Pattern Language: Towns, Buildings, Construction

The original design pattern source; what makes a human space work, ineffable best practices, ~250 rules is enough to do communities and house-sized artifacts; could be a good metaphor for making; could be a good metaphor for making human virtual space work? (David)

• On tangible user interfaces, humans, and spatiality Sharlin-2004-TUI

Considers a range of user interfaces, ranging from the ordinary computer mouse to the cognitive cube, and the heuristics that underly their use. The article covers the logic that lies behind tactile user interfaces, with an eye to the cognitive systems and spatial relations involved. (Clara, 11 September 2011)

• Tangible Bits: Towards Seamless Interfaces between People, Bits, and Atoms Ishii-1997-TBT

A specific UI proposal, but has nice relevant discussion on how we perceive "foreground" items and "background" items and their relationship, taking advantage of this "parallel" processing of perception. Includes the use of visual metaphors, phicons, and a notion they invent called "digital shadows," in which the shadow projected by an object conveys some information on its contents. E J Kalafarski 16:02, 26 January 2009 (UTC)

• Slanty Design Beale-2007-SD

Design method with emphasis on discouraging undesirable behavior, by perhaps forcing the user to adapt to the interface, giving equal weight to user goals, user "non-goals," and wider goals of stakeholders besides the immediate user. The important insight seems to be that these wider goals can enhance the user's experience with the larger system in the long run, if not in the immediate timeframe. Five major design steps. E J Kalafarski 16:02, 26 January 2009 (UTC)

• Designing Interactions by Bill Moggridge

Really awesome book on the evolution of interactions with technology. (Trevor)

• Sketching User Experiences by Bill Buxton

Another great book on the practices of interaction design. (Trevor)

• The Laws of Simplicity by John Maeda

An interesting work on the efficiency of minimalist design. Quick read for those interested. (Steven)

A set of design guidelines some of which we may be able to build on in automating interface evaluation; will certainly apply to manual evaluations David Laidlaw

• Ben Shneiderman, The Eyes Have It: User Interfaces for Information Visualization, UM, tech-report, CS-TR-3665. (Jian)

the paper talks about visualization mantra.

• A pattern approach to interaction design Borchers-2001-PAI

A highly-cited work on the development of a language for defining design patterns for use in interface development, with an emphasis on communication between application developers and application domain experts. E J Kalafarski 16:37, 3 February 2009 (UTC)

• Understanding Interaction Design Practices, Goodman et al., CHI '11 (Owner: Diem Tran - Steven, if you already own this, let me know)

This is a position paper describing the disconnect between HCI research and real interaction design practices. It analyzes approaches for studying design practice (e.g., reported practice, anecdotal descriptions, first-person research), and argues a need for generative theories of design in order to address practice. (Steven Gomez)

• Transparent layered user interfaces: an evaluation of a display design to enhance focused and divided attention Harrison-1995-TLU

Proposes a framework for classifying and evaluating user interfaces with semi-transparent windows. Comes out of research investigating graphical user interfaces from an attentional perspective. Jenna Zeigen 11:31, 19 September 2011 (EDT)

• Design and Evaluation of a Mobile Art Guide on iPod Touch

This paper evaluates the design principles behind an iPod app with respect to minimizing cognitive load and maximizing usability. Trade-offs between HCI technology and cognitive load are discussed. (Clara Kliman-Silver 12:29, 19 September 2011 (EDT)--OWNER for Week 3)

• Sensing cognitive multitasking for a brain-based adaptive user interface Solvey-SCM-2011

The paper describes interface designs that are meant to modulate task switching. Given that users are likely to multitask when using applications, it is important to have an interface that provides support for different multitasking strategies and adapt to users' techniques. Scientists designed experiments to develop a better understanding of human multitasking abilities when using interface technology, and apply their conclusions to a human-robot system. Potentially, this article can lay groundwork for an improved understanding of the cognitive demands that the interface we will build places on users, and how we can address them (Hua's comment). (Clara Kliman-Silver 22:40, 25 September 2011 (EDT))

Thinking, analysis, decision making

• Morgan D. Jones: The Thinker's Toolkit: Fourteen Powerful Techniques for Problem Solving

Set of methods for solving problems that might be incorporated into tools for thinking (David)

• Keim, Shazeer, Littman: Proverb: The Probabilistic Cruciverbalist

An automatic crossword-puzzle solver; the software framework for building this program may be a metaphor for some thinking groupware with plug-in modules. (David)

• Thomas, Cook: Illuminating the Path

a research agenda for tools for intelligence analysts; not sure of relevance (David)

• Richard Thaler, Cass Sunstein: Nudge - Improving Decisions About Wealth, Health, and Happiness

A great, easy read for someone who isn't familiar with the psychological perspective. Focuses mainly on public policy issues, but certain sections (on developing a better social security website, for example) relate specifically to digital design. (Steven)

• Graphical Histories for Visualization: Supporting Analysis, Communication, and Evaluation (InfoVis 2008)

Work by Jeff Heer of Stanford (formerly Berkley) on using Graphical Interaction Histories within the Tableau InfoVis application. This is a great recent example of "workflow analysis" that we've been discussing in class. Though geared toward two-dimensional visualizations with clearly defined events, his work offers some very useful design guidelines for working with interaction histories, including evaluations from the deployment of his techniques within Tableau. (Trevor)

• Characterizing Users' Visual Analytic Activity for Insight Provenance

The authors look into combining user triggered and automatically generated vis. histories.

• An exploratory Study on Visual Information Analysis

The authors run a user study to find the tasks involved in collaborative evidence aggregation

• Collaborative Synthesis of Visual Analytic Results

Same as the previous one.

• The sensemaking process and leverage points for technology

The authors propose a model of analysis and identify leverage points for visualization.

• Autonomous robots: from biological inspiration to implementation and control

Autonomous robots are intelligent machines capable of performing tasks in the world by themselves, without explicit human control. This book examines the underlying technology, including control, architectures, learning, manipulation, grasping, navigation, and mapping. Living systems can be considered the prototypes of autonomous systems. ( Wenjun Wang)

• Brainframe: A Knowledge Visualization System for the Neurosciences Shaw-2009-KVS

This paper begins with a brief overview of a problem plaguing the field of neuroscience today-- namely, that there is so much data available that it can't be synthesized in a useful way by researchers-- and the resulting negative effects that arise because of it. The authors propose BrainFrame, a "knowledge management system," designed to streamline this massive amount of data in a way that is sensitive to the limitations of human cognition and perception. Michael Spector 00:36, 20 September 2011 (EDT)

• Uncovering cognitive processes: Different techniques that can contribute to cognitive load research and instruction Van Gog-2009-UCP

This article reviews modern eye tracking techniques as they aim to monitor different aspects of cognitive load. While the techniques employed are aimed at instruction, they are relevant to the project in that users will, presumably, have to learn to use the tools we build. It reviews different methodologies currently employed in eye tracking and cognitive load research. Cognitive load plays a significant role in any sort of data visualization, and therefore, information from this article should inform many parts of the proposal (Clara Kliman-Silver 23:23, 25 September 2011 (EDT)).

• Low-Level Components of Analytic Activity in Information Visualization, Amar, Eagan, and Stasko; InfoVis 2005

The paper presents a taxonomy of ten primitive analysis task types for information visualization, providing a language and vocabulary to discuss and evaluate these tools. The taxonomy was created in part by analyzing student questions about data visualizations; the task types are motivated by real analysis needs. Using a taxonomy like this, or building one, would be important in the proposed "Task Analysis" section Caroline mentioned in her reviews response. We need to be more explicit about the task types we can model, both at the specific level of brain diagram tasks and in a more generalized form. (Steven Gomez)

Visualization

• Data, Information, and Knowledge in Visualization, Chen et al., CG&A Jan 09

• Volume composition and evaluation using eye-tracking data Lu-2010-VCE

Brain data sets are huge, and rendering all the information they contain (at the same time) is almost impossible. To deal with this problem, we could use the approach proposed in this paper (with different data): choosing rendering parameters based on where the user's attention is focused, using eye tracking data to determine that. (Nathan Malkin, 12 September 2011)

• Neural modeling of flow rendering effectiveness

This paper provides a comparison and discussion of "the relative strengths of different flow visualization methods for the task of visualizing advection pathways". This could be useful in selecting visualization methods for the brain circuits software. (As an added bonus, they cite Laidlaw et al.'s "advection task" right there in the abstract.) (Nathan Malkin, 12 September 2011)

• Toward a Perceptual Theory of Flow Visualization, Colin Ware, CG&A March 08

This paper is a good entry point for Ware's other work on neural modeling for visualization. It describes how spatial receptor patterns in the visual cortex enable contour interpretation and related visualization tasks (e.g., particle advection in flow fields). There's also some good discussion about a perception-based approach to visualization, validating visual mappings with perceptual theories. (Steven Gomez)

• An Approach to the Perceptual Optimization of Complex Visualizations, House, Bair, and Ware, TVCG, 2006

This paper describes a humans-in-the-loop architecture for guiding layered visualizations with multiple visual parameters toward optimal tunings. They use a genetic algorithm to iteratively produce new "genomes" of visual parameters that are evaluated by humans (and either passed along or terminated in the genetic process). Finally they do some analysis on the surviving visualization space (though for me, this was less interesting than the generative visualization method using humans and the GA). (Steven Gomez)

• Evaluating 2D and 3D visualizations of spatiotemporal information Kjellin-2008-E23

A frequent topic of interest to brain scientists is longitudinal data: how does the brain change over time? If the brain circuits software were to support answering this kind of question, we might evaluate different approaches to visualization using the methods in this paper. (Nathan Malkin, 12 September 2011)

• Cognitive Models of the Influence of Color Scale on Data Visualization Tasks -Breslow-2009-CMI

Discusses the ways color scales and differences can be influential in the optimization of data visualization and analysis (Jenna Zeigen, 9/12/11) -- Owner: Jenna Zeigen

• Brain structure visualization using spectral fiber clustering Klein-2006-BSV

Present a novel algorithm that allows for visualizing white matter fiber tracts in real time. And the result is more accurate. This visualizing algorithm might be adopted in our proposal. ( --- Chen Xu 17:21, 13 September 2011 (EDT) -OWNER)

• Image Based Flow Visualization van Wijk-2002-IBF

Talked about a two-dimensional fluid flow visualization method which was based on advection and decay of dye. It was called IBFV, with IBFV, a wide variety of visualization techniques can be emulated. It can visualize flow, generate arrow plots, streamlines, particles and topological images, and handle unsteady flows.(Chen)

• The State of the Art in Flow Visualization: Dense and Texture-Based Techniques Laramee-2004-SAF

Discussed dense, texture-based flow visualization techniques. (Chen)

• Interactive Visualization of Small World Graphs van Ham-2004-IVS

The brain can be considered to be a small world type network. This paper "present[s] a method to create scalable, interactive visualizations of small world graphs, allowing the user to inspect local clusters while maintaining a global overview of the entire structure." Jenna Zeigen 11:03, 19 September 2011 (EDT)

• Readings in information visualization: using vision to think By Stuart K. Card, Jock D. Mackinlay, Ben Shneiderman. ( Wenjun Wang) 11:09, 19 September 2011 (EDT)

• Visualization Toolkit: An Object-Oriented Approach to 3-D Graphics ( Wenjun Wang) 11:15, 19 September 2011 (EDT)

• Human factors in visualization research Tory-2004-HFV (Diem Tran 15:15, 19 September 2011 (EDT))

The paper discuss three following aspects about human factors in research:1) review known methodology for doing human factors research, with specific emphasis on visualization, 2) review current human factors research in visualization to provide a basis for future investigation, 3) identify promising areas for future research.

• Data Visualization Optimization via Computational Modeling of Perception Pineo-2012-DVO

This article, to be published next year, develops paradigms for optimizing data visualization in neural-perceptual models. The algorithms aid in visualizations of different areas of the brain and attempt to offer the best projections of neural systems and neural activity. Given that these models are new and highly advanced, they reinforce and extend much of the work in the proposal (Jenna's comment). (Clara Kliman-Silver 23:16, 25 September 2011 (EDT))

• Supporting the Analytical Reasoning Process in Information Visualization Shrinivasan-2008- SAR -- (Jenna Zeigen)

Present a visualization framework that incorporates and supports research on analytic reasoning. Relevant to responses questioning the applicability of cognitive principles to the project.

• ManyNets: an interface for multiple network analysis and visualization Freire-2010-MNI (Diem Tran 21:24, 26 September 2011 (EDT))

The paper proposes an interface to visualize multiple networks at once.

This is an example of how a vis tool is developed and evaluated: through case studies and user feedback.

• Useful junk?: the effects of visual embellishment on comprehension and memorability of charts

The authors explore the effectiveness of embellished graphs versus plain ones by measuring users' memory. Results show that embellished graphs used in the study are more effective.

This is an example of how memory is measured. This is a method to evaluate user performance for our analysis tool. Diem Tran 21:20, 26 September 2011 (EDT)

Evaluation and Metrics

• Creativity factor evaluation: towards a standardized survey metric for creativity support Carroll-2009-CFE

One alternative to evaluating visualizations and other tools based on the amount of time they save is evaluating them based on how much they help creativity. This paper presents a survey metric for creativity support tools. (Nathan Malkin, 12 September 2011)

• Measuring multitasking behavior with activity-based metrics Bebunan-Fich-2011-MMB

When designing interfaces for scientists, we must be mindful of the fact that (like all users) they will be multitasking -- both in terms of cognitive tasks (drawing from multiple sources, evaluating different hypotheses, etc.) and (if the interface allows it) tasks within the software. This paper proposes a definition of multitasking and provides a set of metrics for (computer-based) multitasking. (Nathan Malkin, 12 September 2011)

• A Salience-based Quality Metric for Visualization Jänicke1-Chen-2010

This paper describes a method for defining quality metrics for visualization based on the distribution of salience over a visualization image. (Hua)

• The Challenge of Information Visualization Evaluation Plaisant-2004

This paper surveys the field of information visualization evaluation - current practices, challenges and possible next steps. It is a relatively old article, though, so it may be replaced by a more concurrent survey. (Hua, Sep.12, 2011)

• Heuristics for Information Visualization Evaluation Zuk et al-2006

This paper attempts to leverage some well-known heuristic evaluation used in HCI to Information Visualization. (Hua, Sep.12, 2011)

• Toward Measuring Visualization Insight

Do current approaches for evaluating visualizations provide measures of insight? This viewpoint identifies critical characteristics of insight, argues the fundamental reasons why traditional controlled experiments with benchmark tasks on visualizations do not effectively measure insight, and offers a new approach to controlled experiments that can better capture the notion of insight. ( Wenjun Wang)

• The Challenge of Information Visualization Evaluation Plaisant-2004-CVE (Diem Tran 15:15, 19 September 2011 (EDT))

The paper describes current evaluation practices being used in visualization research and challenges researchers are facing. It then suggests possible steps to improve visualization evaluation.

• Data Visualization Optimization Computational Modeling of Perception Ware-2011-TVCG

This paper presents a computational model of human vision that can be used to optimize and evaluate visualization systems. I think this is a great example of the application of cognitive modeling on visualization. (Hua, Sep.19, 2011)

• Learning Factors Analysis – A General Method for Cognitive Model Evaluation and Improvement Cen-2006-LFA

Propose a semi-automated method for improving a cognitive model called Learning Factors Analysis that combines a statistical model, human expertise and a combinatorial search. The method is used to evaluate an existing cognitive model and to generate and evaluate alternative models. (Chen)

• Cognitive work analysis and the design of user interfaces Westrenen-2011-CWA

Outlines "cognitive work analysis," a synthesis of two known theoretical tools: cognitive task analysis (CTA) and hierarchical task analysis (HTA). Provides a good example of cognitive metrics informing user interface design. Relevant to Hua's response in the panel summary. (Michael)

• Cognitive Design Principles for Visualizations Heiser-2003-CDP

Claims that the design of visualizations can be informed by cognitive principles and, based on those principles, be automated. Their research has implications in any domain in which visualization in useful, including our proposed project. This is relevant to Jenna's response to reviewer #4, by showing that cognitive principles can and have been implemented in the design of visualizations. (Michael)

• Cognitive Dimensions of Notations, Green, 1989

Green gives a framework and vocabulary to discuss usability dimensions for design. Green suggests at the end that the framework can be tied to models of user activity, and that a given model might dictate system requirements in these dimensions. This aligns with our project in the sense that we want a principled way of assessing cognitive load, both in the tool-design process and in building user-centered features. Relevant to (mine, Hua's) comments about profiling user cognitive abilities with benchmark tasks. (Steven Gomez)

• A Cognitive Dimensions Questionnaire Optimised for Users, Blackwell and Green, 2000

This paper discusses an interesting follow-up to Green's "cognitive dimensions" framework. One application of cognitive dimensions is to build general questionnaires that allow users to evaluate systems and environments in these 'dimensions', rather than having an HCI expert guide the evaluation. Relevant to (mine, Hua's) comments about profiling user cognitive abilities with benchmark tasks. A questionnaire used in the development 'spiral' of the brain application would be a principled way of evaluating 'cognitively optimized' tools. (Steven Gomez)

• Integrating statistics and visualization: case studies of gaining clarity during exploratory data analysis Perer-2008-ISV ( Diem Tran 21:20, 26 September 2011 (EDT))

This paper tempts to integrate statistics methodology and visualizations in interactive exploratory tools. The authors conduct long term case studies to evaluate this approach, which give positive results about the effectiveness of the approach.

The paper is an example of how exploratory tool can be evaluated in a long-term manner.

• Evaluating Visualizations based on the Performed Task Juarez-2000-EVPT

Another paper demonstrating evaluation methods of information visualization software. Could be useful for our evaluation purposes. Also shows applicability to other fields of research. (Michael)

Development

• Parallel prototyping leads to better design results, more divergence, and increased self-efficacy Dow-2010-PPL

Not too relevant to the cognition aspects of the proposals, but provides some empirical support for "fast iteration" and related software design techniques, whose virtues are extolled in the proposals. The lesson: if we're going to prototype something for this class, and we want "better design results, more divergence, and increased self-efficacy", we should do it in parallel! (Nathan Malkin, 12 September 2011)

• MULTI-LABEL CLASSIFICATION OF MUSIC INTO EMOTIONS ISMIR 2008

Humans, by nature , are emotionally affected by music. This paper is an interdisciplinary one, which approaches towards automated emotion detection in music. Four algorithms are evaluated and compared in this task (Wenjun Wang, 19 September 2011)

• Sustainable software development

This brief paper describes the shortcomings of typical usability analysis when evaluated against a much different culture. It describes how the software development process and specifically HCI research has focused on values developed through Western societal values and it illustrates how this does not transfer to African nations, specifically Namibia. They ran a simple usability study and found that results of the questionnaires were contradictory to observed evidence. Using this examples and ideas taken from this article, we could develop a plan to help improve the sustainability of this software in a more heterogeneous community. (Stephen Brawner, 26 September 2011)

• From Open Source to long-term sustainability: Review of Business Models and Case studies

Provides several business models of sustainable open source development. It provides case studies for each of their business models. This could easily be relevant if the researchers are looking to create long-term tools that need to be supported long after the research funding ends.

• ‘Libre’ software: turning fads into institutions? What they term as 'Libre' software is open source software that is free to modify and re-distribute and is used here for its more relevant French definition. The paper discusses reasons why communities develop around this software and why they choose to contribute when there are few economic theories that could actually account for this. If certain key ideas are taken from this paper, we can format them to help counter reviewers perceived lack of a software sustainability plan. (Stephen Brawner, 26 September 2011)

Visual Analysis

• Utilizing Saliency Operators for Image Matching Fraundorfer-2003-USO

This paper from the field of computer vision describes mathematical and geometric techniques for identifying regions of saliency and "sub-saliency" in images.

(Nathan Malkin, 19 September 2011)

• Contextual Priming for Object Detection Torralba-2003-CPO

This is another vision paper; it describes how we can use the context in which a region of an image appears to help with object detection in that region.

(Nathan Malkin, 19 September 2011)

• Knowledge Visualization: Towards a New Discipline and Its Fields of Applications Describes the process of visualing knowledge as opposed to simply information. This paper proposes the idea and presents recent research on the subject. Several examples are presented of different types of visualizations. They begin to make the case about non-2d graphical visualization and interactive methods. (Stephen Brawner, 21 September 2011)

• The Art of Seeing: Visual Perception in Design and Evaluation of Non-Photorealistic Rendering A Ph.D thesis describing methods for presenting non-photorealistic images concisely for the purpose of presenting relevant information. It corresponds nicely to our discussion about how to simplify the information well. (Stephen Brawner, 21 September 2011)

• Action Reaction Learning: Automatic Visual Analysis and Synthesis of Interactive Behaviour (Wenjun Wang)

• Predicting human gaze using low-level saliency combined with face detectionNIPS-2007

Under natural viewing conditions, human observers shift their gaze to allocate processing resources to subsets of the visual input. Many computational models try to predict such voluntary eye and attentional shifts.This paper here demonstrate that a combined model of face detection and low-level saliency significantly outperforms a low-level model in predicting locations humans fixate on(Wenjun Wang)