CS295J/Final contributions

Specific Contributions

[David: I have removed some unattributed stuff that wasn't in the format described in assignment 13. If that was in error, go back to an earlier version to recover the text, attribute it, and put in the correct format.]

Thursday 2pm version of final contribution

Workflow, Multi-tasking and Interruptions

• Owner: Andrew Bragdon

Aims Coming Soon

Contributions

• Scientific Study of Multi-tasking Workflow and the Impact of Interruptions

1. We will undertake detailed studies to help understand the following questions:

1. How does the size of a user's working set impact interruption resumption time?
2. How does the size of a user's working set, when used for rapid multi-tasking, impact performance metrics?
3. How does a user interface which supports multiple simultaneous working sets benefit interruption resumption time?

• No Dependencies

• Meta-work Assistance Tool

1. We will perform a series of ecologically-valid studies to compare user performance between a state of the art task management system (control group) and our meta-work assistance tool (experimental group) [Didn't your study show that there is nothing close to a killer app in this department? - Steven]

• Dependent on core study completion, as some of the specific design decisions will be driven by the results of this study. However, it is worth pointing out that this separate contribution can be researched in parallel to the core study.

• Baseline Comparison Between Module-based Model of HCI and Core Multi-tasking Study

1. We will compare the results of the above-mentioned study in multi-tasking against results predicted by the module-based model of HCI in this proposal; this will give us an important baseline comparison, particularly given that multi-tasking and interruption involve higher brain functioning and are therefore likely difficult to predict

• Dependent on core study completion, as well as most of the rest of the proposal being completed to the point of being testable

Recording User-Interaction Primitives

• Owner: Trevor O'Brien

Aims

• Develop system for logging low-level user-interactions within existing applications. By low-level interactions, we refer to those interactions for which a quantitative, predictive, GOMS-like model may be generated.[How do you propose to get around the problem Andrew brought up of accurately linking this low-level data to the interface/program's state at the moment it occurred? - Steven]
• At the same scale of granularity, integrate explicit interface-interactions with multimodal-sensing data. i.e. pupil-tracking, muscle-activity monitoring, auditory recognition, EEG data, and facial and posture-focused video.

Contributions

• Extensible, multi-modal, HCI framework for recording rich interaction-history data in existing applications.[Perhaps something should be said about sectioning this data off into modules - i.e. compiling all e-mail writing recordings to analyze separately from e-mail sorting recordings in order to facilitate analysis. - Steven]

Demonstration

• Techniques for pupil-tracking, auditory-recognition, and muscle-activity monitoring will be evaluated with respect to accuracy, sampling rate, and computational cost in a series of quantitative studies. [Is there an explicit, quantifiable demonstration you can propose? Such some kind of conclusive comparison between automated capture techniques and manual techniques? Or maybe a demonstration of some kind of methodical, but still low-level, procedure for clustering related interaction primitives? E J Kalafarski 15:02, 29 April 2009 (UTC)] [Should this take into account the need to match recordings to interface state? - Steven]

Dependencies

• Hardware for pupil-tracking and muscle activity monitoring. Commercial software packages for such software may be required.

Background and Related Work

• The utility of interaction histories with respect to assessing interface design has been demonstrated in . Can you briefly summarize the utility with one sentence? Or is that what the next bullet does? (Jon)
• In addition, data management histories have been shown effective in the visualization community in , providing visualizations by analogy and offering automated suggestions , which we expect to generalize to user interaction history.

'review'

• This is recording everything possible during an interaction. It is necessary to do for our project. (Gideon)
• Two thumbs up. - Steven

Semantic-level Interaction Chunking

• Owner: Trevor O'Brien

Aims

• Develop techniques for chunking low-level interaction primitives into semantic-level interactions, given an application's functionality and data-context. (And de-chunking? Invertible mapping needed?)
• Perform user-study evaluation to validate chunking methods.

Contributions

• Design and user-study evaluation of semantic-chunking techniques for interaction.

Demonstration

• These techniques will be validated quantitatively with respect to efficiency and user error rate on a set of timed-tasks over a series of case studies. In addition, qualitative feedback on user satisfaction will be incorporated in the evaluation of our methods. [As above, how will this be validated quantitatively? Comparing two methodologies for manual chunking? What would these differing methodologies be? E J Kalafarski 15:06, 29 April 2009 (UTC)]

Dependencies

• Software framework for collecting user interactions.[I'd like to see this fleshed out in a theoretical manner. - Steven]
• Formal description of interface functionality.
• Description of data objects that can be manipulated through interaction.

review

• With what level of accuracy can this be performed. I like the idea, its worthy of a nice project in and of itself. (Gideon)

Can you include something about the broader impact of this contribution? (Jon)

Reconciling Usability Heuristics with Cognitive Theory

• Owner: E J Kalafarski 14:56, 28 April 2009 (UTC)

Contribution: A weighted framework for the unification of established heuristic usability guidelines and accepted cognitive principles.

[Andrew: Better word choice needed; technical terms that you are inventing should be defined -- for example, what is a "matricized analogue"? According to Wikipedia, matricization is defined as tensor unfolding - no doubt this is not what you intendend. Reviewers will not be gentle on points such as this, as they go to credibility.]

[David: I'm having trouble parsing this contribution. How do you weight a framework? Also, I'd like to have a little more sense of how this might fit into the bigger picture. The assignment didn't ask for that, but is there some way to provide some of that context?] [I agree that the wording is is a little confusing. The framework is not weighted, but rather it weights the different guidelines/principles. It might also be worth explaining how this is a useful contribution, e.g. does it allow for more accurate interface evaluation? -Eric]

Demonstration: Three groups of experts anecdotally apply cognitive principles, heuristic usability principles, and a combination of the two.

• A "cognition expert," given a constrained, limited-functionality interface, develops an independent evaluative value for each interface element based on accepted cognitive principles.
• A "usability expert" develops an independent evaluative value for each interface element based on accepted heuristic guidelines.
• A third expert applies several unified cognitive analogues from a matrix of weighted cognitive and [...HCI design principles? -Eric]
• User testing demonstrates the assumed efficacy and applicability of the matricized analogues versus independent application of analogued principles.

[There is still the question of how these weights are determined. Is this the job of the third expert? Or is the third expert given these weights, and he just determines how to apply them? -Eric]

Dependency: A set of established cognitive principles, selected with an eye toward heuristic analogues.

Dependency: A set of established heuristic design guidelines, selected with an eye toward cognitive analogues.

review

• This seems like the 2d matrix? Is this implemented as a module? (Gideon)
• Gideon's comment brings up an important point - in what form will the weights be presented? - Steven
• Could be more explicit about whether the experts are simulated agents or real people (Jon)

Evaluation for Recommendation and Incremental Improvement

• Owner: E J Kalafarski 14:56, 28 April 2009 (UTC)

Contributions: Using a limited set of interface evaluation modules for analysis, we demonstrate, in a narrow and controlled manner, the proposed efficiency and accuracy of a method of aggregating individual interface suggestions based on accepted CPM principles (e.g. Fitts' Law and Affordance) and applying them to the incremental improvement of the interface.

[David: I'm a little lost. Can you make this long sentence into a couple shorter simpler ones? I think I can imagine what you are getting at, but I'm not sure.]

Demonstration: A narrow but comprehensive study to demonstrate the efficacy and efficiency of automated aggregation of interface evaluation versus independent, analysis. Shows not total replacement, but a gain in speed of evaluation comparable to the loss of control and feature.

• Given a carefully-constrained interface, perhaps with as few as two buttons and a minimalist feature set, expert interface designer given the individual results of several basic evaluation modules makes recommendations and suggestions to the design of the interface.
• Aggregation meta-module conducts similar survey of module outputs, outputting recommendations and suggestions for improvement of given interface.
• Separate independent body of experts then implements the two sets of suggestions and performs user study on the resultant interfaces, analyzing usability change and comparing it to the time and resources committed by the evaluation expert and the aggregation module, respectively.

[David: nice demonstration!]

Dependency: Module for the analysis of Fitts' Law as it applies to the individual elements of a given interface.

Dependency: Module for the analysis of a Affordance as it applies to the individual elements of a given interface.

[David: I think this also has, as a dependency, some kind of framework for the modules. "narrow but comprehensive" sounds challenging. ]

review

• I see this as being the "black box" for our architecture? If so, good. Wouldn't the dependencies be any/all modules? (Gideon)
• [I think this is the same as my contribution, and that we should merge them together. You've covered some things that I didn't, such as a more detailed demonstration of how our framework is better than a human interface designer's evaluation. You also gave a demonstration of comparing recommendations, while I only covered evaluations. -Eric]
• I think this contribution and mine (the CPM-GOMS improvement) will need to be reconciled into a single unit. - Steven
• Interesting to note that this could describe the overarching architecture but it depends upon components of the architecture to work. Would it depend on any other modules? (Jon)

Evaluation Metrics

Owner: Gideon Goldin

• Architecture Outputs

• Time (time to complete task)

1. Performs as well or better than CPM-GOMS, demonstrated with user tasks [More an aspect of demonstration than design. - Steven]

• Dependencies

1. CPM-GOMS Module with cognitive load extension

• Cognitive Load

1. Predicts cognitive load during tasks, demonstrated with user tasks

• Dependencies

1. CPM-GOMS Module with cognitive load extension
2. Facial Gesture Recognition Module

• Frustration

1. Accurately predicts users' frustration levels, demonstrated with user tasks

• Dependencies

1. Facial Gesture Recognition Module
2. Galvanic Skin Response Module
3. Interface Efficiency Module

• Aesthetic Appeal

1. Analyzes if the interface is aesthetically unpleasing, demonstrated with user tasks

• Dependencies

1. Aesthetics Module

• Simplicity

1. Analyzes how simple the interface is, demonstrated with user tasks

• Dependencies

1. Interface Efficiency Module

[I agree that a metric (or several) is crucial to our framework, but what is your demonstration of the feasibility or usability of these particular values you've chosen. On an unrelated note, we talked a lot about how some of these might be "convertible" into others…do you see these metrics as different sides of the same coin? Can the units of one be "converted" into the units of another, like gallons to liters? E J Kalafarski 15:10, 29 April 2009 (UTC)]

[More detail on each demonstration would be helpful. -Eric]

review

• Words cannot express how amazing antiquated this contribution is. It The writer should be given as much money as requested drawn and quartered without question.
• How will the aesthetics module work? Would it have its own dependencies? (Jon)
• With regards to the aesthetics module, you might consider looking at the papers that have successfully correlated subjective evaluations of web page aesthetics with low-level image statistics. Siemens' research lab has been doing some great work in this area. (Andrew)

Parallel Framework for Evaluation Modules

• Owner: Adam Darlow, Eric Sodomka

This section will describe in more detail the inputs, outputs and architecture that were presented in the introduction.

Contribution: Create a framework that provides better interface evaluations than currently existing techniques, and a module weighting system that provides better evaluations than any of its modules taken in isolation.

Demonstration: Run a series of user studies and compare users' performance to expected performance, as given by the following interface evaluation methods:

1. Traditional, manual interface evaluation
   • As a baseline.
2. Using our system with a single module
   • "Are any of our individual modules better than currently existing methods of interface evaluation?".
3. Using our system with multiple modules, but have aggregator give a fixed, equal weighting to each module
   • As a baseline for our aggregator: want to show that the value of adding the dynamic weighting.
4. Using our system with multiple modules, and allow the aggregator to adjust weightings for each module, but have each module get a single weighting for all dimensions (time, fatigue, etc.)
   • For validating the use of a dynamic weighting system.
5. Using our system with multiple modules, and allow the aggregator to adjust weightings for each module, and allow the module to give different weightings to every dimension of the module (time, fatigue, etc.)
   • For validating the use of weighting across multiple utility dimensions.

Dependencies: Requires a good set of modules to plug into the framework.

review

• What exactly are the differences between this and EJ's earlier contribution? I think that if they are the same, this one is a bit more clear, IMHO. (Gideon)

[I have a similar question to Gideon's. My hunch is that this defines a sort of "namespace" for the modules themselves, while the other contribution (the one you guys assigned to me) is more explicitly the aggregator of these outputs. Correct me if I'm wrong. But if that's the case, the two contributions might need to be validated similarly/together. E J Kalafarski 15:17, 29 April 2009 (UTC)]

• I feel like this structure's clarity may come at expense of specificity, I'd like to know more than that it's parallel. - Steven
• Very thorough demonstration(s)! (Jon)
• I think that the contribution is much more clearly stated here -- I would argue for integrating them together. (Andrew)

Sample Modules

Fitts's Law

• Owner: Jon Ericson

Aims

• Provides an estimate of the required time to complete various tasks that have been decomposed into formalized sequences of interactions with interface elements, and will provide evaluations and recommendations for optimizing the time required to complete those tasks using the interface.
• Integrate this module into our novel cognitive framework for interface evaluation.

Contributions

• [Not sure here. Is this really novel?] [I like it. I think it's necessary. I think we demonstrated in class that this has not been formalized and standardized already. E J Kalafarski 15:22, 29 April 2009 (UTC)]

Demonstrations

• We will demonstrate the feasibility of this module by generating a formal description of an interface for scientific visualization, a formal description of a task to perform with the interface, and we will correlate the estimated time to complete the task based on Fitt's law with the actual time required based on several user traces. [Check. E J Kalafarski 15:22, 29 April 2009 (UTC)]

Dependencies

• Requires a formal description of the interface with graph nodes representing clickable interface elements and graph edges representing the physical (on-screen) distance between adjacent nodes. [Is this graph format you're suggesting part of the proposal, or is the "language" itself a dependency from literature? E J Kalafarski 15:22, 29 April 2009 (UTC)]

Module Description

• Inputs [In theory could the non-semantic inputs be automatically read from the interface? - Steven]
  • A formal description of the interface and its elements (e.g. buttons).
  • A formal description of a particular task and the possible paths through a subset of interface elements that permit the user to accomplish that task.
  • The physical distances between interface elements along those paths.
  • The width of those elements along the most likely axes of motion.
  • Device (e.g. mouse) characteristics including start/stop time and the inherent speed limitations of the device.

• Output
  • The module will then use the Shannon formulation of Fitt's Law to compute the average time needed to complete the task along those paths.

Affordances

• Owner: Jon Ericson

Aims

• To provide interface evaluations and recommendations based on a measure of the extent to which the user perceives the relevant affordances of the interface when performing specified tasks.

Contributions

• A quantitative measure of the extent to which an interface suggests to the user the actions that it is capable of performing.
• A quantitative, indirect measure of the extent to which an interface facilitates (or hinders) the use of fast perceptual mechanisms.

[Again, I'm a fan. I don't think this has been formalized already. E J Kalafarski 15:24, 29 April 2009 (UTC)]

Demonstrations

• We will demonstrate the feasibility of this module through the following experiment:
  • Specify a task for a user to perform with scientific visualization software.
  • There should be several different ways to complete the task (paths through the space of possible interface actions).
  • Some of these paths will be more direct than others.
  • We will then measure the number of task-relevant affordances that were perceived and acted upon by analyzing the user trace, and the time required to complete the task.
  • Use the formula: (affordances perceived) / [(relevant affordances present) * (time to complete task)].
  • Correlate the resulting scores with verbal reports on naturalness and ease-of-use for the interface.

Dependencies

• Requires a method for capturing user traces that include formalized records of interface elements used for a particular, pre-specified task or set of tasks.
• Providing suggestions/recommendations will require interaction with other modules that analyze the perceptual salience of interface elements.

[Some kind of formalized list or arbitrary classification of affordances might be necessary to limit the scope of this contribution. E J Kalafarski 15:24, 29 April 2009 (UTC)]

Description of the Module

• Module Inputs
  • Formalized descriptions of...
    • Interface elements
    • Their associated actions
    • The functions of those actions
    • A particular task
    • User traces for that task [Could this module benefit from eye-tracking? - Steven]

• Processing
  • Inputs (1-4) are then used to generate a "user-independent" space of possible functions that the interface is capable of performing with respect to a given task -- what the interface "affords" the user. From this set of possible interactions, our model will then determine the subset of optimal paths for performing a particular task. The user trace (5) is then used to determine what functions actually were performed in the course of a given task of interest and this information is then compared to the optimal path data to determine the extent to which affordances of the interface are present but not perceived.

• Output
  • The output of this module is a simple ratio of (affordances perceived) / [(relevant affordances present) * (time to complete task)] which provides a quantitative measure of the extent to which the interface is "natural" to use for a particular task.

Automaticity of Interaction

• Owner: Gideon Goldin

Measures how easily the interaction with the interface becomes automatic with experience and makes suggestions for improvements.

• Learning
  • Logan's instance theory of automatization [I feel like there's more research into general task-learning curves which would be very useful as we're trying to transition away from CPM's focus on expert users. - Steven]
• Fluency
  • As meta-cognitive information
  • As a cognitive heuristic

Inputs

• Interface
• User goals

Outputs

• Learning curve

Anti-Pattern Conflict Resolution

• Owner: Ian Spector

Interface design patterns are defined as reusable elements which provide solutions to common problems. For instance, we expect that an arrow in the top left corner of a window which points to the left, when clicked upon, will take the user to a previous screen. Furthermore, we expect that buttons in the top left corner of a window will in some way relate to navigation.

An anti-pattern is a design pattern which breaks standard design convention, creating more problems than it solves. An example of an anti-pattern would be if in place of the 'back' button on your web browser, there would be a 'view history' button.

The purpose of this module would be to analyze an interface to see if any anti-patterns exist, identify where they are in the interface, and then suggest alternatives.

Inputs

• Formal interface description
• Tasks which can be performed within the interface
• A library of standard design patterns
• Outputs from the 'Affordances' module
• Uncommon / Custom additional pattern library (optional)

Outputs

• Identification of interface elements whose placement or function are contrary to the pattern library
• Recommendations for alternative functionality or placement of such elements.

Integration into the Design Process

• Owner: Ian Spector

This section outlines the process of designing an interface and at what stages our proposal fits in and how.

The interface design process is critical to the creation of a quality end product. The process of creating an interface can also be used as a model for analyzing a finished one.

There are a number of different philosophies on how to best design software and in turn, interface. Currently, agile development using an incremental process such as Scrum has become a well known and generally practiced procedure.

The steps to create interfaces varies significantly from text to text, although the Common Front Group at Cornell has succinctly been able to reduce this variety into six simple steps:

• Requirement Sketching
• Conceptual Design
• Logical Design
• Physical Design
• Construction
• Usability Testing

This can be broken down further into just information architecture design followed by physical design and testing.

As far as this proposal is concerned in the context of interface design, the goal of our proposal is to improve what is involved with later end of the middle two portions: logical and physical design. Prior to feeding an interface to the system we are proposing, designers should have already created a baseline model for review that should exhibit the majority, if not all, of the functionality listed in the interface requirements. Once this initial interface has been created, our system will aid in rapidly iterating through the physical design process. The ultimate end products are then subject to human usability testing.

[This seems more like background work than a contribution. - Steven]