CS295J/Experiment results from class 6
David: interactively identifying brain pathways
Methods
Eni Halilaj, a CS PhD student, produced a 21 minute recording of herself running an interactive application named brainapp. This application displays hundreds to thousands of curbed cylindrical tubes that are calculated from medical imaging data and that represent coherent white matter. Her task was to select a subset of the tubes corresponding to two specific tracts: the corpus callosum and the corticospinal tract. To select a subset, she creates axis-aligned 3D boxes through which the tubes pass together with logical rules for which tubes to retain. She demonstrates one case where she selects the tubes that pass through one box but that miss another as well as a case where the tubes pass through two different boxes. Eni has used this program quite a bit, so is quick at it.
The video is online at CS, but is 4+Gb, so I did not put it in the wiki. It was taken on tripod and showed the computer display, keyboard, mouse, back of Eni's head, and her right arm and hand. It was recorded onto a miniDV videotape, and transferred to computer using Windows MovieMaker software. I played it back in PowerDVD, pausing and rewinding as necessary, while entering her actions. The time display was in seconds, and the detail is at that granularity.
In the detail below the second column shows the time, the third column the activity following that time. The first column clusters those activities into higher-level types of actions according to the following key:
k - keyboarding
m - mousing
x - refering to external info
w - waiting
2 - 2d viewing and interaction (e.g., windows)
3 - 3d viewing and interaction
T - talking out loud in ways that are likely delaying the task
k 00:09 sits down k 00:12 text window pops up; moving text window around k 00:22 starts typing into text window k 00:44 still typing k 00:51 starts changing the subject name/location; is doing this by editing a text file, backspacing over something and then typing something new kx 01:07 looking over at some paper looking back at the screen still editing the subject info k 01:21 starting to start brainapp; typing at a command line prompt describing the options to the command k 01:57 still typing the options k 02:07 still typing the options w 02:17 an empty graphics window pops up m2 02:23 brings text window to the front, moves it to the corner w 02:29 done with that w 02:33 waiting w 02:37 waiting w 02:40 waiting w 02:50 waiting k 02:58 types in the text window in response to output w 03:08 additional bunch of text output came out m 03:14 a brain shows up in the graphics window m2 03:16 pops the graphics window in front of the text one m3 03:18 resizes the brain m3 03:25 rotating brain and bringing up dialog boxes; everything being done with the mouse; uses slider to adjust threshold, increasing/decreasing tube density m3 03:42 moves toward T 03:50 pointing at the corpus callosum; explaining what she'll do m3 04:05 adjusting box parameters in the menus slider bars m3 04:22 moved head forward to look very closely at the display m3x 04:28 still looking very closely, looking off to the left at paper m3 04:30 looking back at the display; adjusting some parameters. mT 04:44 rotating the brain around to different positions to bring in up them between the various menu windows; describing how to pick boxes so that they don't have too many incorrect tubes, and finally rotating around to verify that the box has good stuff in it m3 05:20 add ROI many more tubes come up trying to highlight the box, by clicking on it having difficulty because it's surrounded by some of the other things m3 05:49 box is now selected; all of the unselected tubes go away; rotating this around mT 05:58 pointing out some unwanted tubes mT 06:07 describing putting a new box to get rid of unwanted tubes dragging m3 06:15 dragging box faces around with sliders m3 06:24 moving, zooming in m3 06:28 adjusting box parameters m3 06:37 done with second box; looking to use it as a tool for removing unwanted tubes m3 06:47 rotating around to check for accuracy m3 06:52 rotating around to check for accuracy m3 06:57 identifies more tubes that are incorrect m3 06:76 working to place another box adjusting the edges at m3 07:14 carefully adjusting the faces m3 07:19 rotating around to check m3 07:22 rotating m3 07:26 more rotating m3 07:32 very intense study of the display while rotating things adjusting the box size left right with the mouse m3 07:50 saving the boxes for future use, closing the window k 07:56 the windows closed k 08:08 cortical spinal tract in the same subject starting up brainapp again w 08:10 waiting w 08:22 waiting w 08:30 waiting kw 08:37 something pops up in the text window types a response m2 08:49 window shows up with brainapp and she starts rotating the brain brings up menus m3 09:03 adjusting brain orientation m3 09:13 still adjusting the brain m3 09:22 still adjusting the brain m3 09:25 rotating around m3T 09:29 at the cortico-spinal tract and describing it m3 09:47 adjusting parameters of the box m3 09:54 adjust another parameter, adjusting to the sliders m3 09:59 another one m3 10:04 now rotating the brain around to look at the box that's been selected m3 10:12 continuing to rotate, continue to rotate and translate m3T 10:25 describing that it has some extra tracks and what should be done m3 10:35 selecting just the tracts going to the box m3T 10:45 pointing at the things that we want to get rid of m3 10:52 creating another box m3 10:58 moving it around and adjusting it m3 11:04 continuing to move it m3 11:10 now rotating the brain, duplicating that box and moving it over to the other side so that you can catch the pieces that are wrong on that side m3 11:31 and adjusting the third box little bit location wise m3 11:39 zooming in m3T 11:45 explaining how to combine boxes (tracts through 2) m2 11:51 select 2 boxes m2 12:00 bring up combinations dialog box m2 12:02 select 'and' m2 12:06 select other two m2 12:10 select 'and' T 12:12 explain m3 12:16 move head in and carefully study, zoom, rotate m3 12:21 explain extra fiber m3 12:31 make another box m3 12:42 adjusting parms m3 12:47 visually verifying k2 12:51 saving out w 12:56 saved and closed