CS295J/Review Response

Proposal Information

Proposal Number:

1047832

Proposal Title:

SI2-SSI: Collaborative Research: Cognition-aware Visual Analytics of Brain Circuits

Received by NSF:

06/14/10

Principal Investigator:

David Laidlaw

Co-PI(s):

David Badre

Steven Sloman

NSF Program Information

NSF Division:

Office of CyberInfrastructure

NSF Program:

Software Institutes

Program Officer:

Manish Parashar

PO Telephone:

(703) 292-8970

PO Email:

mparasha@nsf.gov

Panel Summary #1

Proposal Number: 1047832

Panel Summary: Panel Summary

INTELLECTUAL MERIT (INCLUDING POTENTIAL TRANSFORMATIVE ASPECTS):

POSITIVE ASPECTS OF THE PROPOSAL AND PROPOSED RESEARCH:

The panel noted that the proposal was clear and comprehensive, and addressed the criteria for SI2 proposals as well as the general NSF criteria. The proposed software would enhance both visualization of data on brain function and the knowledge discovery process of researchers in this area.

SHORTCOMINGS AND WEAKNESSES OF THE PROPOSAL AND PROPOSED RESEARCH:

The panel's discussion focused on the sustainability issue beyond the end of the project support timeframe. While a section of the proposal talks about the Outreach, Education, and Sustainability Plan, the community outreach and sustainability aspects are treated somewhat cursorily. These two issues are closely related: without community support, the software is unlikely to be sustainable in the long run. On the other hand, the proposers appear to be well known in their field, which may enhance community uptake.

The proposal was perhaps over-ambitious; aspects of the evaluation (for example, eye tracking) will create large amounts of data that will require correspondingly intensive data analysis. However, the panel felt that even if the project did not accomplish every detail of the proposal, it would still be highly worthwhile. Similarly, while some aspects of the project might be seen as risky, a certain amount of risk is acceptable in NSF proposals, or even expected. Moreover, any risk is mitigated by the qualifications of the PIs, as exemplified by their excellent track record.

==============

BROADER IMPACTS:

POSITIVE ASPECTS OF THE PROPOSAL AND PROPOSED RESEARCH:

The panel believes that the framework of this project would be portable to such other fields as gene regulation and the analysis of other complex networks.

SHORTCOMINGS AND WEAKNESSES OF THE PROPOSAL AND PROPOSED RESEARCH:

==============

ADDITIONAL REVIEW CRITERIA:

The proposed software would primarily be of use to brain researchers. Other fields would be impacted indirectly, in the sense that if this way of building software packages combining visualization with support for hypothesis testing and tracking of analyses succeeds, it might provide a pattern for those fields to follow in their own software development processes.

The proposers have laid out a detailed five year plan. One panelist questioned whether sufficient attention had been paid to issues of sustainability, in particular there was no mention made of plans for software support beyond the end of the five year plan.

==============

SYNTHESIS COMMENTS:

The panel agreed that this was a highly competitive proposal. In particular, the proposal attacks a large but tractable problem, and the team's wide and deep expertise gives the project a high probability of success.

==============

PANEL RECOMMENDATION (CHECK ONE): [X] Competitive (C) [ ] Not Competitive (NC)

This panel summary was read by panelists who participated in the discussion of this proposal, and they concurred that the summary accurately reflects the panel discussion.

Panel Recommendation: Competitive

Review #1

Proposal Number:

1047832

NSF Program:

Software Institutes

Principal Investigator:

Laidlaw, David H

Proposal Title:

SI2-SSI: Collaborative Research: Cognition-aware Visual Analytics of Brain Circuits

Rating:

Excellent

REVIEW:

What is the intellectual merit of the proposed activity?

SSI proposal.

A five year project to develop software for visualization and analysis of brain circuitry by working with brain researchers to analyze their cognitive processes that need to be supported. The software is to help link the visualization workflow to a "decisional" workflow, supporting "reasoning and analysis at a high level, rather than just displaying data."

The development phase will include studies of user interaction at both low level (eye tracking, mouse click logs) and high level (decision making).

What are the broader impacts of the proposed activity?

The proposers see this as sort of a prototype of the way software could be developed to support other scientific endeavors; this effect appears to constitute most of the Broader Impact (I wouldn't count benefit to "the entire brain science research community", mentioned in the BI statement, as broader impact).

Summary Statement

While this work is mostly outside my domain of expertise, it looks like a well chosen problem and an interdisciplinary effort. In particular, the team appears to have the broad range of people they would need to pull this off, from experts from the target audience (who suggested the project, and is listed as a co-PI), to cognitive scientists and computer scientists. In addition to the Intellectual and Broader Impact criteria, the specific "additional criteria" listed in the Program Solicitation have been explicitly and (to the extent that I can tell) well addressed. The required supplemental documents address the required points as well. One quibble I have in the Management and Coordination Plan is the statement that "The Stanford researchers will visit Brown if face-to-face interactions become necessary." While electronic communication allows collaboration in ways that would not have been possible before, I think it's not a question of whether face-to-face will be necessary, but how often. Fortunately, this appears to have been built into the travel budget.

Review #2

Proposal Number:

1047832

NSF Program:

Software Institutes

Principal Investigator:

Laidlaw, David H

Proposal Title:

SI2-SSI: Collaborative Research: Cognition-aware Visual Analytics of Brain Circuits

Rating:

Excellent

REVIEW:

What is the intellectual merit of the proposed activity?

The PI's of this project are proposing to develop, test, and deploy software tools for scientific study of brain circuits. The project will focus on building a cyber infrastructure software system that is intended to improve the speed at which those doing brain research are able to complete their data analysis and it will advance the understanding of human cognition. This project has potential to have impact in the way researchers in the field collect and analyze data by providing an a rich set of cyber infrastructure tools for use in studying and modeling brain circuits. The intellectual merit of the project is very high as it has the potential to greatly reduce the time required to collect and analyze data.

What are the broader impacts of the proposed activity?

This project will provide an enhanced set of software tools to researchers in several areas that are conducting research that is related to the human brain. Areas of research that the cyber infrastructure software can be used in included: gene regulation, protein signaling and even crime and terrorism analysis and all have the potential to benefit.

Summary Statement

This project is focused on an area of research that spans several disciplines that will be able to utilize the cyber infrastructure software that will be developed. The PI's have a proven record of accomplishment in prior research projects. The project has intellectual merit and will have a broad impact by providing an enhanced set of software tools that will facilitate the efforts of researchers doing work related to studying and modeling the brain.

Review #3

Proposal Number:

1047832

NSF Program:

Software Institutes

Principal Investigator:

Laidlaw, David H

Proposal Title:

SI2-SSI: Collaborative Research: Cognition-aware Visual Analytics of Brain Circuits

Rating:

Very Good

REVIEW:

What is the intellectual merit of the proposed activity?

The proposed activity will allow brain scientists to visualize brain functions more easily and in much more detail than in the past.Network models, coupled with sophisticated methods for dimensionality reduction, promise to offer unique insights into the workings of the human brain. Moreover, the proposed visualization tool will go through rigorous evaluation that will allow its constant improvement. The proposers form a very strong group of well-established researchers in brain science and data visualization, offering a unique collaboration.

What are the broader impacts of the proposed activity?

Researchers from other disciplines, e.g., who study gene regulation, protein signaling, or perform crime and terrorism analysis, etc., have the potential to be benefited by the proposed software.

Summary Statement

This unique collaboration between brain scientists and data visualization scientists promises to offer tremendous benefits to the scientific community. The software that will be developed, will allow researchers to understand the signal pathways in the human brain in more detail than ever before. The software will be analyzed through a rigorous process, using models of cognition. Review #4

Proposal Number:

1047832

NSF Program:

Software Institutes

Principal Investigator:

Laidlaw, David H

Proposal Title:

SI2-SSI: Collaborative Research: Cognition-aware Visual Analytics of Brain Circuits

Rating:

Good

REVIEW:

What is the intellectual merit of the proposed activity?

The goal of this proposal is to develop, test, and deploy interactive visualization tools for scientific study of brain circuits. The tools will help brain researchers view brain circuits at multiple scales and perform sophisticated analysis of research hypotheses. The team members have a decade of experience developing scientific visualization tools for scientific users and consist of experts in cognitive science, neuroscience, computer science, and visual design. They are well qualified to conduct the project.

What are the broader impacts of the proposed activity?

This is an interdisciplinary project and the target user community is brain scientists. The tools will be made available to the public and are expected to benefit the entire brain science research community as well as other disciplines studying linked types of data. The tools can also be used in classes to help students understand connectivity.

As the proposal is for SI2-SSI, I would like to know more on what the team plans to do to ensure the sustainability of the software and develop open-source community support. It is also unclear what the team will do to integrate diversity into the proposed activity.

Summary Statement

I can see the proposed work will be valuable for brain scientists to study connectivity and dynamics of neural circuits in intact brain as existing systems all have limitations and cannot satisfy the needs of brain scientists as discussed in the proposal. Other scientific domains may need similar tools.

This proposal focuses on an interesting problem for which it also provides a novel solution. Therefore, I think this is a quality proposal and worthy of support.