CS295J/Research proposal

Project summary

We propose to integrate theories models of cognition, models of perception, rules of design, and concepts from the discipline of human-computer interaction to develop a predictive model of user performance in interacting with computer software for visual and analytical work. Our proposed model comprises a set of computational elements representing components of human cognition, memory, or perception. The collective abilities and limitations of these elements can be used to provide feedback on the likely efficacy of user interaction techniques.

The choice of human computational elements will be guided by several models or theories of cognition and perception, including Gestalt, Gibson, ???(where pathway, when pathway)???, ???working-memory???, ..., and ???. The list of elements will be extensible. The framework coupling them will allow for experimental predictions of utility of user interfaces that can be verified against human performance.

Coupling the system with users will involve a data capture mechanism for collecting the communications between a user interface and a user. These will be primarily event based, and will include a new low-cost camera-based eye-tracking system

During early development, existing interfaces will be evaluated manually to characterize their

(we need some way to specify interaction techniques...)

Specific Contributions

1. A model of human cognitive and perceptual abilities when using computers
   1. Demonstration of the model in predicting human performance with some interfaces.
   2. ???
2. Something about design rules collected and merged
   1. Something comparing these collected rules to a baseline (establishing their value)
   2. ???
3. A low-overhead mechanism for capturing event-based interactions between a user and a computer, including web-cam based eye tracking. (should we buy or find out about borrowing use of pupil tracker?)
   1. Accuracy study of eye tracking (2 cameras? double as an input device?)
   2. ???
4. A set of critiques of existing software used for visual and analytical work based on design rules
   1. ???

Specific Aims

1. build X
2. build Y
3. run experiment Z
4. compare X with existing approach Q

Background

Models of cognition

There are several models of cognition, ranging from fundamental aspects of neurological processing to extremely high-level psychological analysis. Three main theories seem to have become recognized as the most helpful in conceptualizing the actual process of HCI. These models all agree that one cannot accurately analyze HCI by viewing the user without context, but the extent and nature of this context varies greatly.

Activity Theory, developed in the early 20th century by Russian psychologists S.L. Rubinstein and A.N. Leontiev, posits the existence of four discrete aspects of human-computer interaction. The "Subject" is the human interacting with the item, who possesses an "Object" (e.g. a goal) which they hope to accomplish by using a tool. The Subject conceptualizes the realization of the Object via an "Action", which may be as simple or complex as is necessary. The Action is made up of one or more "Operations", the most fundamental level of interaction including typing, clicking, etc.

A key concept in Activity Theory is that of the artifact, which mediates all interaction. The computer itself need not be the only artifact in HCI - others include all sorts of signs, algorithmic methods, instruments, etc.

A longer synopsis of Activity Theory may be found at this website.

The Situated Action Model focuses on emergent behavior, emphasizing the subjective aspect of human-computer interaction and the therefore-necessary allowance for a wide variety of users. This model proposes the least amount of contextual interaction, and seems to maintain that the interactive experience is determined entirely by the user's ability to use the system in question. While limiting, this concept of usability can be very informative when designing for less tech-savvy users.

Distributed Cognition proposes that the computer (or, as in Activity Theory, any other artifact) can be used and ought to be thought of as an extension of the mental processing of the human. This is not to say that the two are of equal or even comparable cognitive abilities, but that each has unique strengths and that recognition of and planning around these relative advantages can lead to increased efficiency and effectiveness. The rotation of blocks in Tetris serves as a perfect example of this sort of cognitive symbiosis.

Workflow Context (Andrew Bragdon - OWNER)

There are, at least, two levels at which users work (Gonzales, et al., 2004). Users accomplish individual low-level tasks which are part of larger working spheres; for example, an office worker might send several emails, create several Post-It (TM) note reminders, and then edit a word document, each of these smaller tasks being part of a single larger working sphere of "adding a new section to the website." Thus, it is important to understand this larger workflow context - which often involves extensive levels of multi-tasking, as well as switching between a variety of computational and non-computational devices.

Gibsonianism

(relevant stuff about Gibson's theory) We will build on top of this theory/model by ... .

Design guidelines

There are several sets

User interface evaluations

Cost-based analyses

Heidi Lam, "A Framework of Interaction Costs in Information Visualization," IEEE Transactions on Visualization and Computer Graphics, vol. 14, no. 6, pp. 1149-1156, Nov./Dec. 2008, doi:10.1109/TVCG.2008.109

I'm pretty sure that this paper referred to some earlier paper with a very similar title.

Research plan

We can speculate here about a longer-term research plan, but it may not be necessary to actually flesh out this part of the "proposal"