DTI-Circuit Requirements

Brain-Circuit Diagram Requirements

Some background: We made some progress in the fall on an interactive visualization that overlays a node-link diagram of functional brain connections above a anatomical map of these regions. For a starting point, this was in 2D (i.e., the spatial layout of the nodes was simply mapped over a single, static, sagittal view of the brain). In our talks with the Stanford group, it seems they are ultimately looking for a 3D "database" of these connections. In other words, they want to view connections in the context of the full brain volume.

A demo video of that tool is available here [ Media:Steve_2dcircuit_prototype.mp4 ]. Jeff Law, who was/is a post-doc in that group, hints at some reqs for the next design in his feedback to that video:

"The movie looks REALLY COOL. And a working version of this would be very helpful for us! But just to clarify what you've done:

So you basically loaded the database from the Allen atlas (the pdf file) and combined it with the connectivity information you got from BAMS. Am I right? One thing that would be really nice is instead of searching for specific area and show the connections (which is very useful), the software let the user to choose the area of interest by moving the cursor (and the tag would show the name of the area), and the connections to/from that area would be shown interactively. Do you get what I mean?"

Data

Multiple species support for collaborators

1. Rat/Mouse
2. Human
3. Macaque

Curated data

1. Brain atlases (Allen, Paxinos, etc.), spatial coordinate systems (e.g., MNI)
2. Histology/plates
3. Functional brain connectivity

Experimental data

1. fMRI time-series - Badre has data of this nature

For the last project, the Stanford group was using the BAMS database, and specifically looking at the connections in the rat brain. This data is serialized and available as XML; previously, we parsed the Swanson-98 XML to build the connection graph.

Visualization

Accessibility

1. Easy to learn and use
   1. One idea is to build off "canonical" brain diagrams that are familiar to neuroscientists, and use these as scaffolding for drawing other curated data (e.g., projections), like Radu's protein pathway viewer
   2. Conforms to good UI design rules
2. Easy to deploy to end-users (we should not preclude the CAVE if possible, though it is not the target display)
   1. Web deployment (e.g. Google Maps-style) if sufficiently powerful
3. Supports fast prototyping for new features, designs

Scale

1. Supports 3D brain data: coordinate system for areas/nuclei/regions, shape of fiber bundles (tractography), volume rendering of regions
2. Supports multi-scale brain understanding (single cell up to whole brain)
3. Supports viewing subregions/substructures, systems and global view
   1. Internal and external connections to a systems, with filtering by system/region

The Schnitzer group described their desired tool as a 3D visual database of functional (neuron projections) brain connections. Presumably, the "3D" part means they're interested in studying these connections w.r.t. anatomical coordinates of the neuron terminus.

Interaction

Here is the current interaction Jeff does when performing a "circuit query". The high-level goal should be to simplify this process and make new types of interactions/queries available.

"I'm interested in a few areas: namely the projection from visual/sensory areas to forelimb motor and prelimbic.
So the way I do my search is:
1. go on the bams website.
2. choose brain parts
3. put "primary motor", "Secondary motor" and "prelimbic" in "Full text serach of brain parts annotations" in 3 different searches
4. choose the rats database, Swanson 1998
5. Then in the search, look for primary motor again (this shows how inefficient the search on BAMS is, because I searched for primary motor and it returns a million irrelevant search results)
5. Click on "Primary motor"
6. click on ""Efferent projections of Primary Motor Area" -- those are the areas that primary motor area sends projections to
7. click on ""Afferent projections of Primary Motor Area" -- those are the areas that primary motor area receives projections from
8. That's how I find out the connectivity data from the database. It's not totally clear and not very intuitive to use.

Like I said, I also repeat the search with Secondary Motor and Prelimbic as keywords."

End-User Reqs/Use Cases [from Previous NSF Proposals]

Schnitzer Lab Brain Circuit Analysis

A research direction in the Schnitzer laboratory at Stanford is the study of neural circuits that underlie sensorimotor learning. A brain area that we are interested in is the prefrontal cortex, which is a polymodal area that receives sensory and reward signals from multiple sources and sends outputs to many cortical and subcortical motor structures to guide behavior. It is suggested that sensorimotor learning occurred as a result of changes in sensory-motor mapping that occurs within the local circuits in the prefrontal cortex.

To form hypothesis about the precise location at which learning occurs, it is important to consider simultaneously the external and internal connections. The visual analysis tools proposed would be particularly useful for this reasoning process because, as opposed to traditional brain visualization tools, it will allow simultaneous visualization of connectivity at multiple levels. The user can, as a result, visualize interactions between external and internal circuits in a common visual framework. The reasoning process can further be aided by selective filtering of irrelevant information (e.g. inputs from auditory cortex in a visual task), and post-hoc reevaluation of the visual reasoning process.

Badre Lab Brain Circuit Analysis

David Badre's lab studies cognitive control, which refers to our ability to plan and guide our behavior based on internally maintained goals. Human cognitive control function is classically associated with the prefrontal cortex (PFC), as damage to this region impairs goal-directed behavior. However, cognitive control function is modulatory rather than transmissive in that the route from sensation to action does not pass obligatorily through PFC. Rather, the distributed representation of goal information in PFC neurons modulates the mappings between inputs and outputs represented elsewhere in the brain. As a consequence, understanding cognitive control function requires studying how PFC operates dynamically within systems-level networks. At least two projects in the lab offer examples of this brain network-level approach to understanding cognitive control function.

(1) Memory requires cognitive control ... We are currently using a combined fMRI, effective connectivity, and DTI tractography approach to (a) locate anatomical evidence of dorsal and ventral PFC-MTL pathways in the human brain, (b) assess their differential functional connectivity and dynamics during a strategic retrieval task, and (c) study how certain neurotransmitter systems, like dopamine, may be critical signalers for this circuit.

(2) Ongoing behavior can be expressed at multiple levels of abstraction, from a general goal to a concrete sequence of motor responses ... We are currently using a combination of fMRI localization methods, functional connectivity, and DTI tractography in order to more fully characterize the dynamic interactions between basal ganglia and PFC during hierarchical cognitive control tasks.

The proposed tool will be valuable in both of these cases because it will help us mentally merge our circuit diagrams with the intrinsically 3D region and connectivity data that fMRI and diffusion MRI tractography provide. This is a particularly challenging cognitive task, and support for removing as many distractions and other obstacles from the analysis process will help us move forward morequickly and efficiently.

Some Requirements for Tools

These examples, combined with our understanding of visualization tools, of scientific support tools, and of cognition, suggest some specific requirements for the tools we are proposing. First, they need to be able to support reasoning about brain regions and their connections. This support needs to include access to brain region and connectivity knowledge that has already been published. These published data are typically available via several curated databases. Examples of brain connectivity databases are the Brain Architecture Management Systems (BAMS), Collations of Connectivity Data on the Macaque Brain (CoCoMac) and the Functional Anatomy of the Cerebro-Cerebellar System (FACCS). Second, this reasoning requires diagramming the regions and connectivity. Third, the diagrams need to handle multiple levels of scale and abstraction. Fourth, users need to be abel to interact with imaging data and time course experimental data to support the reasoning. Our examples of imagining data include functional MRI, diffusion MRI, and optical microscopy. Reasoning with these data will involve being able to interact with them in 3D and understand networks and brain regions both in their anatomical 3D space and in abstract representations. Similar, but simpler, support will be needed for time course data. Fifth, they need to be able to track and refine their scientific analyses over weeks to years.

SVL Project Requirements

Here's a short list of software deliverables from the SVL proposal:

1. A descriptive and predictive language for generating symbol-based visualizations of 3D tensor fields
   1. Theory and experiment needed to define symbol space and the "visual variables" that the language expresses and can perturb. (Is "symbol-based" a hard requirement? In practice, is that a simplification or something that will hold us back?)
2. Testbed tools for evaluating and optimizing visualizations
   1. Can we score a visualization automatically without humans using a model of perception/human information processing?
   2. Can we locally optimize visualizations using that model and a hill-climbing algorithm?
3. Application development for DTI/BioFlow

Previous Efforts

NSF Expeditions Pre-proposal (September 2010) -- Media:NSF_expedition_pre_circuits.pdf

NSF SI-S2 Proposal (June 2010) -- Media:SI2-SSI-submission_circuits.pdf

Reviews for both collected by DHL -- Media:NSF_circuit_prop_reviews.pdf