CS295J/Final contributions: Difference between revisions

From VrlWiki
Jump to navigation Jump to search
added temporal patterns sub-project
 
(3 intermediate revisions by 2 users not shown)
Line 212: Line 212:


* What exactly are the differences between this and EJ's earlier contribution? I think that if they are the same, this one is a bit more clear, IMHO. (Gideon)
* What exactly are the differences between this and EJ's earlier contribution? I think that if they are the same, this one is a bit more clear, IMHO. (Gideon)
<span style="color: gray;">[I have a similar question to Gideon's.  My hunch is that this defines a sort of "namespace" for the modules themselves, while the other contribution (the one you guys assigned to me) is more explicitly the aggregator of these outputs.  Correct me if I'm wrong.  But if that's the case, the two contributions might need to be validated similarly/together. [[User:E J Kalafarski|E J Kalafarski]] 15:17, 29 April 2009 (UTC)]</span>
<span style="color: gray;">[I have a similar question to Gideon's.  My hunch is that this defines a sort of "namespace" for the modules themselves, while the other contribution (the one you guys assigned to me) is more explicitly the aggregator of these outputs.  Correct me if I'm wrong.  But if that's the case, the two contributions might need to be validated similarly/together. [[User:E J Kalafarski|E J Kalafarski]] 15:17, 29 April 2009 (UTC)]</span> <span style="color: blue;">[I agree that this pretty close to E.J.'s contribution, and that maybe they should be merged. E.J. covered some things that I didn't, such as a more detailed demonstration of how our framework is better than a human interface designer's evaluation, and his gave a demonstration of comparing recommendations, while I only covered evaluations.]</span>
* I feel like this structure's clarity may come at expense of specificity, I'd like to know more than that it's parallel. - Steven
* I feel like this structure's clarity may come at expense of specificity, I'd like to know more than that it's parallel. - Steven
* Very thorough demonstration(s)! (Jon)
* Very thorough demonstration(s)! (Jon)
Line 269: Line 269:


'''Dependencies'''
'''Dependencies'''
*Requires a formal description of the interface with graph nodes representing clickable interface elements and graph edges representing the physical (on-screen) distance between adjacent nodes. <span style="color: gray;">[Is this graph format you're suggesting part of the proposal, or is the "language" itself a dependency from literature? [[User:E J Kalafarski|E J Kalafarski]] 15:22, 29 April 2009 (UTC)]
*Requires a formal description of the interface with graph nodes representing clickable interface elements and graph edges representing the physical (on-screen) distance between adjacent nodes. <span style="color: gray;">[Is this graph format you're suggesting part of the proposal, or is the "language" itself a dependency from literature? [[User:E J Kalafarski|E J Kalafarski]] 15:22, 29 April 2009 (UTC)]</span> <span style="color: blue;">I think the graph structure is just a good way of representing something that will be derived from an algorithm that simply measures the distances between interface elements -- it's not something novel that we're developing.</span>


'''Module Description'''
'''Module Description'''
Line 306: Line 306:
*Requires a method for capturing user traces that include formalized records of interface elements used for a particular, pre-specified task or set of tasks.
*Requires a method for capturing user traces that include formalized records of interface elements used for a particular, pre-specified task or set of tasks.
*Providing suggestions/recommendations will require interaction with other modules that analyze the perceptual salience of interface elements.
*Providing suggestions/recommendations will require interaction with other modules that analyze the perceptual salience of interface elements.
<span style="color: gray;">[Some kind of formalized list or arbitrary classification of affordances might be necessary to limit the scope of this contribution. [[User:E J Kalafarski|E J Kalafarski]] 15:24, 29 April 2009 (UTC)]</span>
<span style="color: gray;">[Some kind of formalized list or arbitrary classification of affordances might be necessary to limit the scope of this contribution. [[User:E J Kalafarski|E J Kalafarski]] 15:24, 29 April 2009 (UTC)]<span><span style="color: blue;">I'm not sure I understand the comment -- are you asking how affordances will be formalized?</span>


'''Description of the Module'''
'''Description of the Module'''
Line 316: Line 316:
***The functions of those actions
***The functions of those actions
***A particular task
***A particular task
***User traces for that task <span style="color: gray;">[Could this module benefit from eye-tracking? - Steven]</span>
***User traces for that task <span style="color: gray;">[Could this module benefit from eye-tracking? - Steven]</span> <span style="color: blue;">Response: Absolutely, and it would also benefit from some measure of endogenous attention as well because determining whether someone perceived an affordance requires knowing where they were looking, what they were attending to, and whether or not they actually performed the afforded action.</span>


*Processing
*Processing
Line 323: Line 323:
*Output
*Output
**The output of this module is a simple ratio of (affordances perceived) / [(relevant affordances present) * (time to complete task)] which provides a quantitative measure of the extent to which the interface is "natural" to use for a particular task.
**The output of this module is a simple ratio of (affordances perceived) / [(relevant affordances present) * (time to complete task)] which provides a quantitative measure of the extent to which the interface is "natural" to use for a particular task.
== Temporal Patterns in Interface Interpretation ==
* Owner: [[User:Adam Darlow | Adam Darlow]]
'''Aims'''
* Identify the interface factors that enable users to generate a causal theory of an application's behavior and whether that theory will be mechanistic or intentional.
* Apply cognitive theory of event perception to identify the cues by which users segment events under each of the overarching theories.
* Use the above knowledge to evaluate and improve the intuitiveness and discoverability of application interfaces.
'''Contributions'''
* Formulations of temporal interaction ''design patterns'' for use by interface designers. For example, a design pattern for rigid physical link would state that it is a mechanistic relation between two interface items whereby when one item is moved by the user, the other item makes exactly the same movements in synchrony. It would also state how much movement is sufficient to generate the impression of the rigid link.
* Implementation and evaluation of the addition of temporal patterns to an existing interface for more intuitive and dicoverable interactions.
* Implementation of an interface evaluation module that measures the presence and consistency of temporal patterns in an interface.
'''Demonstration'''
* We will demonstrate significant improvements to the memorability and discoverability of functions in existing interfaces by adding temporal patterns to the interface interactions. We will show that these patterns affect how users conceptualize the application and how they interact with it.
* We will demonstrate that the corresponding evaluation module can determine when an interface is lacking in temporal patterns or has temporal patterns that are inconsistent in terms of a mechanistic or intentional conceptualization of the application.
'''Dependencies'''
* Video capture of user interfaces synchronized with traces of the user interactions.
* The parallel evaluation framework proposed elsewhere in this proposal.
'''Background and Related Work'''
* Increased computational power allows us to go beyond metaphors from the real world and use realistic dynamic interactions to make interfaces more intuitive. The naive approach to this has been to simulate a complete physical environment <ref> [http://portal.acm.org/citation.cfm?id=1124772.1124965 Keepin’ It Real: Pushing the Desktop Metaphor with Physics, Piles and the Pen] </ref>. While effective at making functionality more intuitive and discoverable, this approach has only been applied to file organization and it isn't clear how it generalizes to applications in general.
* Research in cognitive science suggests a more sophisitcated approach. People automatically develop high level conceptualizations of a display based on certain low-level patterns of temporal dynamics <ref>[http://vrl.cs.brown.edu/wiki/Scholl-2000-PCA Perceptual causality and animacy] </ref>. Different patterns of interaction among elements suggest different causal or intentional interpretations. People use these interpretations when predicting how the system will behave. Therefore, having temporal dynamics based on a coherent conceptualization can make functions more intuitive, memorable and discoverable.
* System conceptualization also influences how events in that system are segregated <ref> [http://www.michaelwmorris.com/UsefulPapers/ZacksTversky2001.pdf Event Structure in Perception and Conception] </ref>. The user will make different generalizations from an interaction event depending on whether it is perceived as a single complex event or a series of simpler ones. This affects the user's ability to remember the interaction and to discover other possible interactions.
'''Significance'''
* To the best of my knowledge, no existing research has studied the general application of temporal dynamics from the real world to human-computer interface design beyond systems that simulate a physical environment. This approach aims to extend the advantages of simulated physical environments to other real world metaphors in interface design.
* One goal of this project is to open up a new field of research in the use of realistic temporal interactions to harness users' existing intuitions. Simulated physical environments <ref> [http://portal.acm.org/citation.cfm?id=1124772.1124965 Keepin’ It Real: Pushing the Desktop Metaphor with Physics, Piles and the Pen] </ref> use rich temporal interactions, but are limited to isolated projects because of the overhead in designing the system and the focus on the desktop metaphor. Other real world metaphors are ubiquitous in interface design <ref> [http://portal.acm.org/citation.cfm?doid=1188816.1188820 The reification of metaphor as a design tool] </ref> but don't take advantage of users' sensitivity to temporal dynamics. Our research aims to demonstrate that temporal dynamics can be applied to many interfaces with different metaphors.
* Another goal of the project is to better understand how different conceptualizations of an application lead users to interact with it in different ways. For example, when thinking of what a function would be called, a user with a mechanistic conceptualization of the interface might look for something denoting the process of the function, while a user with an intentional conceptualization might look for something denoting the end result of the function. If both interface designers and interface users think about interfaces in terms of these (and other) high level conceptualizations, this can lead to more consistent, intuitive and enjoyable interfaces. The idea of application conceptualization is wide open for research. As useful as it is, the distinction between mechanical and intentional is only the tip of the iceberg.

Latest revision as of 15:42, 13 May 2009

Specific Contributions

[I have removed almost everything that does not describe itself as a contribution. If your part is missing, a revision is in order and you can find what the former version on the page linked below (David)]

Thursday 2pm version of final contribution

Workflow, Multi-tasking and Interruptions

Aims

  • Develop a Theory of Multi-tasking Workflow and the Impact of Interruptions
  • Build a Meta-work Assistance Tool
  1. We will perform a series of ecologically-valid studies to compare user performance between a state of the art/popular task management system (control group) and our meta-work assistance tool (experimental group) [Didn't your study show that there is nothing close to a killer app in this department? - Steven] [By "state of the art" I mean a popular, well-recognized system (or systems), such as Outlook, that are used by large numbers of people.]
  • Evaluate the Module-based Model of HCI by using it to predict the outcome of the above-mentioned evaluation; then compare with empirical results

Contributions

  • Theory of Multi-tasking Workflow and the Impact of Interruptions
  1. We will undertake detailed studies to help understand the following questions:
  1. How does the size of a user's working set impact interruption resumption time?
  2. How does the size of a user's working set, when used for rapid multi-tasking, impact performance metrics?
  3. How does a user interface which supports multiple simultaneous working sets benefit interruption resumption time?
  • Internal dependencies: none
  • Empirical Evaluation of Meta-work Assistance Tool in an Ecologically-valid Context
  • Internal dependencies: Completion of a testable meta-work tool
  • Baseline Comparison Between Module-based Model of HCI and Core Multi-tasking Study
  1. We will compare the results of the above-mentioned study in multi-tasking against results predicted by the module-based model of HCI in this proposal; this will give us an important baseline comparison, particularly given that multi-tasking and interruption involve higher brain functioning and are therefore likely difficult to predict
  • Internal dependencies: Dependent on core study completion, as well as most of the rest of the proposal being completed to the point of being testable

Demonstration (we might consider calling these deliverables)

  • A predictive, quantiative model of controlled task performance and interruption resumption time for varying sized working sets
  • An ecologically valid evaluation which shows that the completed meta-work assistance tool outperforms a state of the art system such as Outlook
  • A comparison between the result predicted by the module-based model of HCI and the evaluation mentioned above; accompanied by a detailed analysis of factors contributing to the result

External Dependencies

  • Dependent on most of the rest of the proposal being completed to the point of being usable to predict the outcome of the meta-work tool evaluation

Background and Related Work

  • Salvucci et al's integrated theory of multi-tasking continuum integrates concurrent and sequential task performance
  • Gloria Mark's research group has conducted a number of empirical studies into multi-tasking and interruption

Preliminary Work

  • Results suggest there is not a single "ideal" meta-work tool today, motivating our proposal to build a new meta-work support tool based in part on a scientific theory of multi-tasking and interruptions.

Reviews

  • Not in the expected format -- split into appropriate parts and label them as in the other sections (David)

Recording User-Interaction Primitives

Aims

  • Develop system for logging low-level user-interactions within existing applications. By low-level interactions, we refer to those interactions for which a quantitative, predictive, GOMS-like model may be generated.[How do you propose to get around the problem Andrew brought up of accurately linking this low-level data to the interface/program's state at the moment it occurred? - Steven] [Low-level interactions will need to be logged along with a markup-language-style state description of the application itself. This would either require access to the source code, or a detailed interface description from the interface designer that would allow for complete simulation of the application.]
  • At the same scale of granularity, integrate explicit interface-interactions with multimodal-sensing data. i.e. pupil-tracking, muscle-activity monitoring, auditory recognition, EEG data, and facial and posture-focused video.

Contributions

  • Extensible, multi-modal, HCI framework for recording rich interaction-history data in existing applications.[Perhaps something should be said about sectioning this data off into modules - i.e. compiling all e-mail writing recordings to analyze separately from e-mail sorting recordings in order to facilitate analysis. - Steven] [I believe this is addressed in the "semantic-level chunking" section, given below.]

Demonstration

  • Techniques for pupil-tracking, auditory-recognition, and muscle-activity monitoring will be evaluated with respect to accuracy, sampling rate, and computational cost in a series of quantitative studies. [Is there an explicit, quantifiable demonstration you can propose? Such some kind of conclusive comparison between automated capture techniques and manual techniques? Or maybe a demonstration of some kind of methodical, but still low-level, procedure for clustering related interaction primitives? E J Kalafarski 15:02, 29 April 2009 (UTC)] [Things like pupil-tracking can be quantitatively evaluated through users tests. For instance, we could instruct users to shift the focus of their eyes from one on-screen object to another as quickly as possible some fixed number of times. We could then compare the results from our system to determine accuracy and speed, and by varying the size and distance of the target objects, we could obtain richer data for analysis. With respect to clustering low-level interactions, I believe that issue is addressed in the following section.][Should this take into account the need to match recordings to interface state? - Steven] [Possibly, if we go down the simulation route. If we're actually logging all of the events from within the source code itself, I believe the demonstration of syncing low-level interaction with application-state is self-evident.]

Dependencies

  • Hardware for pupil-tracking and muscle activity monitoring. Commercial software packages for such software may be required.

Background and Related Work

  • The utility of interaction histories with respect to assessing interface design has been demonstrated in [1]. Can you briefly summarize the utility with one sentence? Or is that what the next bullet does? (Jon) [In the instance of this work, utility was demonstrated with respect to automatically generating presentations from data explorations, and providing quantitative interface-usage analysis that aided future decisions for interface designers. i.e., this operations was performed a lot, and this one was not; therefore we should focus on making the first more efficient, perhaps at the expense of the second. The results of the work are fairly basic, but they do demonstrate that logging histories is a useful practice for a number of reasons.]
  • In addition, data management histories have been shown effective in the visualization community in [2] [3] [4], providing visualizations by analogy [5] and offering automated suggestions [6], which we expect to generalize to user interaction history.

'review'

  • This is recording everything possible during an interaction. It is necessary to do for our project. (Gideon)
  • Two thumbs up. - Steven

Semantic-level Interaction Chunking

Aims

  • Develop techniques for chunking low-level interaction primitives into semantic-level interactions, given an application's functionality and data-context. (And de-chunking? Invertible mapping needed?)
  • Perform user-study evaluation to validate chunking methods.

Contributions

  • Design and user-study evaluation of semantic-chunking techniques for interaction.

Demonstration

  • These techniques will be validated quantitatively with respect to efficiency and user error rate on a set of timed-tasks over a series of case studies. In addition, qualitative feedback on user satisfaction will be incorporated in the evaluation of our methods. [As above, how will this be validated quantitatively? Comparing two methodologies for manual chunking? What would these differing methodologies be? E J Kalafarski 15:06, 29 April 2009 (UTC)] [I don't intend for these techniques to be performed manually. Application-specific rules for chunking will be created manually, most likely by the interface designer, but those rules will be applied automatically to filter low-level primitives and application-states into semantic-level interactions. The success of such methods could be evaluated by having automatically-generated semantic-level histories compared to those manually created by a user. Of course, there's no guarantee that what a user manually creates is any good, so I think it would be more effective to have a quantitative study of user performance in which timed-tasks are completed with and without the ability to review and revisit semantic-level histories. The test without any history at all would serve as a baseline, and then performance with histories created using different chunking rules would speak to the effectiveness of different strategies for chunking.]


Dependencies

  • Software framework for collecting user interactions.[I'd like to see this fleshed out in a theoretical manner. - Steven] [I believe the previous section speaks to this question, but perhaps not with enough detail. It seems appropriate to have this discussion in class.]
  • Formal description of interface functionality.
  • Description of data objects that can be manipulated through interaction.

review

  • With what level of accuracy can this be performed. I like the idea, its worthy of a nice project in and of itself. (Gideon) [This is a great question. It really depends on what's meant by accuracy. If the only level of semantics we want is that defined by the interface-designer, then it's not really an interesting question. As we begin to talk about semantics with respect to what the user perceives or what the user's intent might be, it becomes more interesting, and the term accuracy becomes more ambiguous. I suppose accuracy could be tested by having users verbally speak their intent as they use an interface, and compare their descriptions with the output of the system, but I think that might be a messy study. Interesting to think about.]

Can you include something about the broader impact of this contribution? (Jon) [I think the broader impact here is offering interface designers tools to analyze interaction at a high enough level that they can begin to observe user's intent with an interface, and potentially infer overarching user-goals. If these tools prove effective, and could be implemented in real-time, there's a wealth of possibility in adaptive interfaces. In addition, the semantic-level interaction history construct has the potential to provide meaningful training data for machine-learning algorithms that could probabilistically model user-action and be used to offer suggested interactions.]

Reconciling Usability Heuristics with Cognitive Theory

Contribution: A framework for the unification of established heuristic usability guidelines and accepted cognitive principles. We propose a matrix pairing cognitive principles and heuristic guidelines into specific interface improvement recommendations.

[David: I'm having trouble parsing this contribution. How do you weight a framework? Also, I'd like to have a little more sense of how this might fit into the bigger picture. The assignment didn't ask for that, but is there some way to provide some of that context?] [I agree that the wording is is a little confusing. The framework is not weighted, but rather it weights the different guidelines/principles. It might also be worth explaining how this is a useful contribution, e.g. does it allow for more accurate interface evaluation? -Eric]

[Language cleaned up. While I still think it is important to give more "weight" to cognitive/heuristic principle pairs that agree more strongly than others and output more useful recommendations, I believe I've pitched it wrong and made it too prominent an attribute of this module. I've adjusted. E J Kalafarski 15:34, 1 May 2009 (UTC)]

Demonstration: Three groups of experts anecdotally apply cognitive principles, heuristic usability principles, and a combination of the two.

  • A human "cognition expert," given a constrained, limited-functionality interface, develops an independent evaluative value for each interface element based on accepted cognitive principles.
  • A human "usability expert" develops an independent evaluative value for each interface element based on accepted heuristic guidelines.
  • A third expert applies several recommendations from a matrix of cogntive/heuristic principle pairs.
  • User testing demonstrates the assumed efficacy and applicability of the principle pairs versus separate application of cognitive and heuristic guidelines.
  • Matrix can be improved incrementally by applying "weight" to each cell of the matrix, increasing its influence on final output recommendations, based on the measurable success of this study.

[There is still the question of how these weights are determined. Is this the job of the third expert? Or is the third expert given these weights, and he just determines how to apply them? -Eric] [Added more explanatory description of methodology as a bullet under Demo. E J Kalafarski 15:49, 1 May 2009 (UTC)]

Dependency: A set of established cognitive principles, selected with an eye toward heuristic analogues.

Dependency: A set of established heuristic design guidelines, selected with an eye toward cognitive analogues.

review

  • This seems like the 2d matrix? Is this implemented as a module? (Gideon) [Yes. E J Kalafarski 15:49, 1 May 2009 (UTC)]
  • Gideon's comment brings up an important point - in what form will the weights be presented? - Steven [Values which contribute to the "influence" of each recommendation. E J Kalafarski 15:49, 1 May 2009 (UTC)]
  • Could be more explicit about whether the experts are simulated agents or real people (Jon)

Evaluation for Recommendation and Incremental Improvement

Contributions: Using a set of interface evaluation modules for analysis, we demonstrate the proposed efficiency and accuracy of a method for aggregating these interface suggestions. In this preliminary demonstration, we limit the field to a strictly-defined interface and 2–3 CPM principles (e.g. Fitts' Law and Affordance). Unified recommendations are generated for application to the interface for incremental improvement in usability.

[David: I'm a little lost. Can you make this long sentence into a couple shorter simpler ones? I think I can imagine what you are getting at, but I'm not sure.]

Demonstration: A limited but strictly-controlled study to demonstrate the efficacy and efficiency of automated aggregation of interface evaluation versus independent, analysis. Shows not total replacement, but a gain in speed of evaluation comparable to the loss of control and feature.

  • Given a carefully-constrained interface, perhaps with as few as two buttons and a minimalist feature set, expert interface designer given the individual results of several basic evaluation modules makes recommendations and suggestions to the design of the interface.
  • Aggregation meta-module conducts similar survey of module outputs, outputting recommendations and suggestions for improvement of given interface.
  • Separate independent body of experts then implements the two sets of suggestions and performs user study on the resultant interfaces, analyzing usability change and comparing it to the time and resources committed by the evaluation expert and the aggregation module, respectively.

[David: nice demonstration!]

Dependency: Module for the analysis of Fitts' Law as it applies to the individual elements of a given interface.

Dependency: Module for the analysis of a Affordance as it applies to the individual elements of a given interface.

Dependency: Framework for the input of a given interface to individual modules, and standardization of the output of said modules.

[David: I think this also has, as a dependency, some kind of framework for the modules. "narrow but comprehensive" sounds challenging. ]

review

  • I see this as being the "black box" for our architecture? If so, good. Wouldn't the dependencies be any/all modules? (Gideon) [I limited the scope of this initial demonstration. E J Kalafarski 15:46, 1 May 2009 (UTC)]
  • [I think this is the same as my contribution, and that we should merge them together. You've covered some things that I didn't, such as a more detailed demonstration of how our framework is better than a human interface designer's evaluation. You also gave a demonstration of comparing recommendations, while I only covered evaluations. -Eric]
  • I think this contribution and mine (the CPM-GOMS improvement) will need to be reconciled into a single unit. - Steven

[Is it feasible to combine all three of these contributions? E J Kalafarski 15:47, 1 May 2009 (UTC)]

  • Interesting to note that this could describe the overarching architecture but it depends upon components of the architecture to work. Would it depend on any other modules? (Jon) [Yes, several modules are listed as dependencies. E J Kalafarski 15:46, 1 May 2009 (UTC)]

Evaluation Metrics

Owner: Gideon Goldin

  • Architecture Outputs
  • Time (time to complete task)
  1. In user studies, this module predicted time-to-completion more accurately than the standard CPM-GOMS model.
  • Dependencies
  1. CPM-GOMS Module with proposed cognitive load extension
  • Cognitive Load
  1. User studies were conducted during which participants brain's were scanned with EEG to measure activation levels.
  2. Self-report measures were obtained describing the amount of concentration/effort required on the part of the user.
  3. The literature base provided an accurate assessment of cognitive load in the tasks involved.
  4. Facial recognition allows us to approximate spans of high-load (e.g., furled brow)
  • Dependencies
  1. CPM-GOMS Module with cognitive load extension
  2. Facial Gesture Recognition Module
  3. Self-Report Measure
  4. Database of load per cognitive tasks
  • Frustration
  1. Frustration is measured by asking participants to respond on the self-report scale.
  2. Facial recognition allows us to approximate spans of frustration (e.g., frowning, sighing, furled brow)
  3. Galvanic Skin Response allows detection of emotional arousal
  4. Heart rate monitoring allows detection of emotional arousal
  • Dependencies
  1. Facial Gesture Recognition Module
  2. Galvanic Skin Response input
  3. Heart rate response input
  4. Interface Efficiency Module

review

Parallel Framework for Evaluation Modules

This section will describe in more detail the inputs, outputs and architecture that were presented in the introduction.

Contribution: Create a framework that provides better interface evaluations than currently existing techniques, and a module weighting system that provides better evaluations than any of its modules taken in isolation.

Demonstration: Run a series of user studies and compare users' performance to expected performance, as given by the following interface evaluation methods:

  1. Traditional, manual interface evaluation
    • As a baseline.
  2. Using our system with a single module
    • "Are any of our individual modules better than currently existing methods of interface evaluation?".
  3. Using our system with multiple modules, but have aggregator give a fixed, equal weighting to each module
    • As a baseline for our aggregator: want to show that the value of adding the dynamic weighting.
  4. Using our system with multiple modules, and allow the aggregator to adjust weightings for each module, but have each module get a single weighting for all dimensions (time, fatigue, etc.)
    • For validating the use of a dynamic weighting system.
  5. Using our system with multiple modules, and allow the aggregator to adjust weightings for each module, and allow the module to give different weightings to every dimension of the module (time, fatigue, etc.)
    • For validating the use of weighting across multiple utility dimensions.

Dependencies: Requires a good set of modules to plug into the framework.

review

  • What exactly are the differences between this and EJ's earlier contribution? I think that if they are the same, this one is a bit more clear, IMHO. (Gideon)

[I have a similar question to Gideon's. My hunch is that this defines a sort of "namespace" for the modules themselves, while the other contribution (the one you guys assigned to me) is more explicitly the aggregator of these outputs. Correct me if I'm wrong. But if that's the case, the two contributions might need to be validated similarly/together. E J Kalafarski 15:17, 29 April 2009 (UTC)] [I agree that this pretty close to E.J.'s contribution, and that maybe they should be merged. E.J. covered some things that I didn't, such as a more detailed demonstration of how our framework is better than a human interface designer's evaluation, and his gave a demonstration of comparing recommendations, while I only covered evaluations.]

  • I feel like this structure's clarity may come at expense of specificity, I'd like to know more than that it's parallel. - Steven
  • Very thorough demonstration(s)! (Jon)
  • I think that the contribution is much more clearly stated here -- I would argue for integrating them together. (Andrew)

Anti-Pattern Recognition & Resolution

Owner: Ian Spector

Interface design patterns are defined as reusable elements which provide solutions to common problems. For instance, we expect that an arrow in the top left corner of a window which points to the left, when clicked upon, will take the user to a previous screen. Furthermore, we expect that buttons in the top left corner of a window will in some way relate to navigation. An anti-pattern is a design pattern which breaks standard design convention, creating more problems than it solves. An example of an anti-pattern would be if in place of the 'back' button on your web browser, there would be a 'view history' button.

Aims:

  • Develop a framework for analyzing an interface and recognizing violations of established patterns of possible interaction
  • Develop an algorithm to quantitatively rate the amount of anti-pattern violations
  • Integrate cognitive theory in providing suitable alternatives

Contributions:

  • Creation of established patterns of interaction as determined by user studies
  • Creation of algorithm to detect match and library elements to interface elements

Demonstration:

  • The viability of this module will be tested through providing it with a poorly designed interface.
    • The design would be analyzed and tested by: a usability expert, normal users, the module whereby each would have to eventually provide a value as to how much the interface exhibits anti-patterns.
    • Based on the results provided, we should expect to see the expert and module reports to be similar

Dependencies:

  • Library of standard design patterns
  • Affordances module
  • Custom design pattern libraries (optional)

Inputs:

  • Formal interface description
  • Tasks which can be performed within the interface

Outputs:

  • Identification of interface elements whose placement or function are contrary to the pattern library
  • Recommendations for alternative functionality or placement of such elements.

Sample Modules

Fitts's Law

Aims

  • Provides an estimate of the required time to complete various tasks that have been decomposed into formalized sequences of interactions with interface elements, and will provide evaluations and recommendations for optimizing the time required to complete those tasks using the interface.
  • Integrate this module into our novel cognitive framework for interface evaluation.

Contributions

  • [Not sure here. Is this really novel?] [I like it. I think it's necessary. I think we demonstrated in class that this has not been formalized and standardized already. E J Kalafarski 15:22, 29 April 2009 (UTC)]

Demonstrations

  • We will demonstrate the feasibility of this module by generating a formal description of an interface for scientific visualization, a formal description of a task to perform with the interface, and we will correlate the estimated time to complete the task based on Fitt's law with the actual time required based on several user traces. [Check. E J Kalafarski 15:22, 29 April 2009 (UTC)]

Dependencies

  • Requires a formal description of the interface with graph nodes representing clickable interface elements and graph edges representing the physical (on-screen) distance between adjacent nodes. [Is this graph format you're suggesting part of the proposal, or is the "language" itself a dependency from literature? E J Kalafarski 15:22, 29 April 2009 (UTC)] I think the graph structure is just a good way of representing something that will be derived from an algorithm that simply measures the distances between interface elements -- it's not something novel that we're developing.

Module Description

  • Inputs [In theory could the non-semantic inputs be automatically read from the interface? - Steven]
    • A formal description of the interface and its elements (e.g. buttons).
    • A formal description of a particular task and the possible paths through a subset of interface elements that permit the user to accomplish that task.
    • The physical distances between interface elements along those paths.
    • The width of those elements along the most likely axes of motion.
    • Device (e.g. mouse) characteristics including start/stop time and the inherent speed limitations of the device.
  • Output
    • The module will then use the Shannon formulation of Fitt's Law to compute the average time needed to complete the task along those paths.

Affordances

Aims

  • To provide interface evaluations and recommendations based on a measure of the extent to which the user perceives the relevant affordances of the interface when performing specified tasks.

Contributions

  • A quantitative measure of the extent to which an interface suggests to the user the actions that it is capable of performing.
  • A quantitative, indirect measure of the extent to which an interface facilitates (or hinders) the use of fast perceptual mechanisms.

[Again, I'm a fan. I don't think this has been formalized already. E J Kalafarski 15:24, 29 April 2009 (UTC)]

Demonstrations

  • We will demonstrate the feasibility of this module through the following experiment:
    • Specify a task for a user to perform with scientific visualization software.
    • There should be several different ways to complete the task (paths through the space of possible interface actions).
    • Some of these paths will be more direct than others.
    • We will then measure the number of task-relevant affordances that were perceived and acted upon by analyzing the user trace, and the time required to complete the task.
    • Use the formula: (affordances perceived) / [(relevant affordances present) * (time to complete task)].
    • Correlate the resulting scores with verbal reports on naturalness and ease-of-use for the interface.

Dependencies

  • Requires a method for capturing user traces that include formalized records of interface elements used for a particular, pre-specified task or set of tasks.
  • Providing suggestions/recommendations will require interaction with other modules that analyze the perceptual salience of interface elements.

[Some kind of formalized list or arbitrary classification of affordances might be necessary to limit the scope of this contribution. E J Kalafarski 15:24, 29 April 2009 (UTC)]I'm not sure I understand the comment -- are you asking how affordances will be formalized?

Description of the Module

  • Module Inputs
    • Formalized descriptions of...
      • Interface elements
      • Their associated actions
      • The functions of those actions
      • A particular task
      • User traces for that task [Could this module benefit from eye-tracking? - Steven] Response: Absolutely, and it would also benefit from some measure of endogenous attention as well because determining whether someone perceived an affordance requires knowing where they were looking, what they were attending to, and whether or not they actually performed the afforded action.
  • Processing
    • Inputs (1-4) are then used to generate a "user-independent" space of possible functions that the interface is capable of performing with respect to a given task -- what the interface "affords" the user. From this set of possible interactions, our model will then determine the subset of optimal paths for performing a particular task. The user trace (5) is then used to determine what functions actually were performed in the course of a given task of interest and this information is then compared to the optimal path data to determine the extent to which affordances of the interface are present but not perceived.
  • Output
    • The output of this module is a simple ratio of (affordances perceived) / [(relevant affordances present) * (time to complete task)] which provides a quantitative measure of the extent to which the interface is "natural" to use for a particular task.

Temporal Patterns in Interface Interpretation

Aims

  • Identify the interface factors that enable users to generate a causal theory of an application's behavior and whether that theory will be mechanistic or intentional.
  • Apply cognitive theory of event perception to identify the cues by which users segment events under each of the overarching theories.
  • Use the above knowledge to evaluate and improve the intuitiveness and discoverability of application interfaces.


Contributions

  • Formulations of temporal interaction design patterns for use by interface designers. For example, a design pattern for rigid physical link would state that it is a mechanistic relation between two interface items whereby when one item is moved by the user, the other item makes exactly the same movements in synchrony. It would also state how much movement is sufficient to generate the impression of the rigid link.
  • Implementation and evaluation of the addition of temporal patterns to an existing interface for more intuitive and dicoverable interactions.
  • Implementation of an interface evaluation module that measures the presence and consistency of temporal patterns in an interface.

Demonstration

  • We will demonstrate significant improvements to the memorability and discoverability of functions in existing interfaces by adding temporal patterns to the interface interactions. We will show that these patterns affect how users conceptualize the application and how they interact with it.
  • We will demonstrate that the corresponding evaluation module can determine when an interface is lacking in temporal patterns or has temporal patterns that are inconsistent in terms of a mechanistic or intentional conceptualization of the application.

Dependencies

  • Video capture of user interfaces synchronized with traces of the user interactions.
  • The parallel evaluation framework proposed elsewhere in this proposal.

Background and Related Work

  • Increased computational power allows us to go beyond metaphors from the real world and use realistic dynamic interactions to make interfaces more intuitive. The naive approach to this has been to simulate a complete physical environment [7]. While effective at making functionality more intuitive and discoverable, this approach has only been applied to file organization and it isn't clear how it generalizes to applications in general.
  • Research in cognitive science suggests a more sophisitcated approach. People automatically develop high level conceptualizations of a display based on certain low-level patterns of temporal dynamics [8]. Different patterns of interaction among elements suggest different causal or intentional interpretations. People use these interpretations when predicting how the system will behave. Therefore, having temporal dynamics based on a coherent conceptualization can make functions more intuitive, memorable and discoverable.
  • System conceptualization also influences how events in that system are segregated [9]. The user will make different generalizations from an interaction event depending on whether it is perceived as a single complex event or a series of simpler ones. This affects the user's ability to remember the interaction and to discover other possible interactions.

Significance

  • To the best of my knowledge, no existing research has studied the general application of temporal dynamics from the real world to human-computer interface design beyond systems that simulate a physical environment. This approach aims to extend the advantages of simulated physical environments to other real world metaphors in interface design.
  • One goal of this project is to open up a new field of research in the use of realistic temporal interactions to harness users' existing intuitions. Simulated physical environments [10] use rich temporal interactions, but are limited to isolated projects because of the overhead in designing the system and the focus on the desktop metaphor. Other real world metaphors are ubiquitous in interface design [11] but don't take advantage of users' sensitivity to temporal dynamics. Our research aims to demonstrate that temporal dynamics can be applied to many interfaces with different metaphors.
  • Another goal of the project is to better understand how different conceptualizations of an application lead users to interact with it in different ways. For example, when thinking of what a function would be called, a user with a mechanistic conceptualization of the interface might look for something denoting the process of the function, while a user with an intentional conceptualization might look for something denoting the end result of the function. If both interface designers and interface users think about interfaces in terms of these (and other) high level conceptualizations, this can lead to more consistent, intuitive and enjoyable interfaces. The idea of application conceptualization is wide open for research. As useful as it is, the distinction between mechanical and intentional is only the tip of the iceberg.