Anticipatory, multimodal interfaces: General aviation weather interface agent

Existing methods for human-computer interaction have many disadvantages in mobile environments. For example, a pointing device may be difficult to use while moving, menu structures may require too much visual attention, and direct manipulation to extract data of interest may require too much cognitive attention. Our research is concerned with reducing the workload of users in hands-busy, eyes-busy mobile environments by designing anticipatory, multimodal interfaces (AMIs) that better complement human capabilities by offering multiple modalities for interaction and by providing a task and domain knowledgeable, context-aware, interface agent for assistance with tasks suitable to a computer. Further, AMIs can be personalized to provide only help desired by the user, and they can automatically adapt by learning the user's habits, decreasing the need for explicit personalization.; The domain of our experiments is general aviation (GA). We describe an anticipatory, multimodal interface to help GA pilots develop (pre-flight) and maintain (in-flight) weather situational awareness. Our system, Aviation Weather Environment (AWE), provides information graphically and through speech to decrease the time spent interpreting data and looking at information. We applied information visualization techniques to develop new graphical representations of airport-specific current and forecast weather conditions and forecast winds aloft, developed a speech grammar inspired by typical pilot-controller communication to aurally extract desired information, and developed an interface agent to provide assistance in retrieving and monitoring weather conditions. The weather agent tracks the aircraft's position along a pilot-selected route of flight; uses domain and task knowledge and heuristics to interpret weather data and provide the pilot information relevant to the current phase of flight; and uses pilot knowledge to provide the information in a format preferred by the pilot. Knowledge about the pilot's preferences comes both directly from the pilot and indirectly through learning her habits using an enhanced reflex learning algorithm we designed.; AWE was evaluated by pilots through questionnaires, interviews, and part-task simulations. Pilots preferred AWE's graphical representations over seven representations of four state-of-the-art weather briefing systems, and their workload was reduced over conventional weather briefing methods by 2.5 times for pre-flight briefings and 5.5 times for in-flight briefings.