CS295J/Proposal reviews from class 8

These are reviews and intellectual contributions for the research proposal, as specified in assignment 7, part A.

Name

Intellectual merit

(your paragraph here)

Gaps

• Your
• Bullet
• List
• Here

Andrew Bragdon

Intellectual merit

(your paragraph here)

Gaps

• Your
• Bullet
• List
• Here

Gideon

Intellectual merit

A meta-analysis of a subset of the cognitive psychology and human-computer interaction literature presents evidence that interactions between humans and computers can be improved by taking into account the cognitive resources required for different types of tasks. It is well known that humans and computers excel at different types of tasks, but the field has not made an explicit effort to standardize a set of guidelines that interface designers may use when developing computer systems. For people and computers to function in an optimized, complementary fashion, we still need a systematic way of distributing tasks amongst them.

It is often the case that what computers excel at, humans have difficulty with (e.g., arithmetic). While, the opposite is also true (e.g., pattern-recognition). After a preliminary search of the literature, we've explored common tasks in software today that have neglected consideration of this performance dichotomy. Designers have not been appropriately addressed these gaps in the computer industry due to a lack of multidisciplinary research. Our meta-analysis presents data that supports our view on two different tasks: 3D shape-rotation and face recognition.

We have demonstrated in just a 30-hour study that computer assisted 3D Shape Rotation is consistently preferred over human-only mental rotation. Complementarily, humans consistently outperform modern computer systems in face recognition. Sometimes these performance gaps are obvious due to a lack of technology or common-sense. However, that is not only the case. Our preliminary study clearly demonstrates that systems benefit from consistent and rule-based task distribution guidelines. Furthermore, the literature is rich with other types of tasks which await our systematic exploitation. Upon further study, the community will benefit from a tested and systematic approach for designing improved human-computer interfaces.

Gaps

• Your
• Bullet
• List
• Here

EJ

Intellectual merit

While attempts have been made in the past to apply cognitive theory to the task of developing human-computer interfaces, there remains much work to be done. No standard and widespread model for the cognitive interaction with a computer exists. The roles of perception and cognition, while examined and studied independently, are often at odds with empirical and successful design guidelines in practice. Methods of study and evaluation, such as eye-tracking and workflow analysis, are still governed primarily by the needs at the end of the development process, with no quantitative model capable of influencing efficiency and consistency in the field.

We demonstrate in wide-ranging preliminary work that cognitive theory has a tangible and valuable role in all the stages of interface design and evaluation: models of distributed cognition can exert useful influence on the design of interfaces and the guidelines that govern it; algorithmic workflow analysis can lead to new interaction methods, including predictive options; a model of human perception can greatly enhance the usefulness of multimodal user study techniques; a better understanding of why classical strategies work will bring us closer to the "holy grail" of automated interface evaluation and recommendation. We can bring the field further down many of the only partially-explored avenues of the field in the years ahead.

Gaps

• Project summary needs expanding and context, but this may be impossible before we solidify other sections
• Background is strong, and does not have an excess of context, but that may be desirable here. Some notes/outlines/questions need to be answered/expanded/removed.
• Is "Significance" supposed to be significantly (ha) different from a projection of the influence and effects of the study? Perhaps these estimates should be parsed out of contributions and placed here.
• Need stronger distinction/clarification between "Aims" and "Contributions."

Eric

Intellectual merit

With the emergence of documented interaction histories in scientific visualization comes a new source of data for predicting user interactions. Correct prediction and corresponding UI modifications allow for a more personalized interface that can improve the user's efficiency in data exploration and enables groups of researchers working on the same type of task to more efficiently learn from one another.

In a 30-hour preliminary study, we have implemented a basic interaction prediction module using a relational markov model and shown through a series of user studies that it predicts on average 35% better than chance. We have also created a module that provides basic recommendations to the user based on these interaction predictions, and have shown that the user clicks on a recommended action 20% of the time, leading to an average task speedup of 8%.

The tasks for future work are twofold: first, we will improve and generalize our prediction module to allow for more accurate predictions in a wide variety of interfaces, including those with a larger number of possible actions and states. Second, we will further study the question of, given predictions of future interactions, how to modify the interface beyond giving basic recommendations. Ultimately, research in both of these directions will allow researchers to more efficiently glean information from complex data, enabling them to more quickly and easily contribute to their respective fields.

Gaps

• A few places in the proposal are cut off mid-sentence or are otherwise incomplete, (e.g. some unfinished lists with "???" bullet points)
• The project summary makes it sound like our projects are more integrated than perhaps they are. Maybe we need to either re-evaluate our project summary, or better unify our individual projects somehow.

Adam

Note: EJ and I have generated joint specific aims and contributions and plan to generate unified preliminary results.

Second Note: According to other examples I misunderstood when I construed this section as a review of intellectual merit instead of writing an intellectual merit section. I'll add one soon.

Intellectual merit

The intellectual merit of the program is currently hard to determine because it is so fractionated. It seems to me that we have two or three goals:

1. Using cognitive principles to develop new design guidelines, especially ones that are above the level of the individual task. The content related to working spheres and distributed cognition mostly belongs here.
2. Using cognitive principles to make existing design principles more quantifiable. This could involve building an integrated cognitive model, but I think it's more plausible to generate separate measures for different cognitive/design principles.
3. Improving methods for assessing interfaces. To the degree that this involves using cognitive simulations, it overlaps with goal number 2. Development of an eye-tracking method or an information integration framework would be unrelated to goal number 2. I don't think anyone has been pursuing either of these ideas and I would recommend dropping them.

I think reorganizing our efforts so that they all fit together, either along these lines or some other way, is essential at this point.

Gaps

• Currently I perceive more gaps than content. See above.

Steven

Intellectual merit

Cognitive theories of human-computer interaction are neither few nor far between, but appear to have received minimal attention in the HCI community and less, if not absolutely no, implementation in the design of digital interfaces. There is not one weak link in the chain from cognitive HCI theories, models, guidelines, etc., but a general dearth of scholarly agreement at every stage of this progression. Scholars maintain antiquated, esoteric models and have not used their combined knowledge of proven cognitive processes and relevant current design practices to improve the human-computer relationship.

There exist multiple gaps in the spectrum of cognitive HCI theory and design, the most pressing of which we have demonstrated a capacity to fill. We have demonstrated in several preliminary studies and experiments the tangible prospect of not only developing a knowledge base for the advancement of cognitive HCI theory and design but for the development of sets of empirically verified models and guidelines which may aid program design at every level of the theoretical/technical spectrum. Our project thus promises not only to bring the field of cognitive HCI up to date but to provide a foundation for myriad related research topics.

Gaps

• Specific aims ought to be revised in light of the further development and specification of our individual goals
• Perhaps we should have a background section on prominent HCI concepts – the fact that GOMS isn’t mentioned once in the proposal seems like an oversight
• Constrain specific aims to preliminary results - It seems they ought to be a bit more congruent

Jon Ericson

Intellectual merit

Most existing theories of human-computer interaction have failed to integrate design principles with research on human cognition. This much-needed integration would facilitate the development and testing of a predictive model of human-computer interaction that would decrease reliance on individual user studies when evaluating candidate interfaces, and thereby aid convergence on optimal interfaces for a given task. This would result in a significant reduction in the amount of time and money invested in conducting individual user studies.

We demonstrate that a computational model based on partial integration of design and cognitive principles successfully predicts the (relative) amount of time required to perform a scientific visualization task when using various alternative interfaces. In order for the predictive ability of this model to generalize to a wide variety of other tasks it will be necessary to incorporate more cognitive and design principles and test the model on a number of other tasks. Further research will also focus on developing a model component capable of suggesting improvements to candidate interfaces that will further streamline user interactions.

Gaps

• The section on Gibsonianism could be improved/expanded considerably.
• There is a certain amount of overlap between some of the specific contributions and our user studies; perhaps some of them could be merged.
• We need to run an experiment in which we use a rudimentary model of human cognitive abilities to predict some aspect of user interaction and compare those predictions to actual performance.