RESUMO
How do we gauge understanding? Tests of understanding, such as Turing's imitation game, are numerous; yet, attempts to achieve a state of understanding are not satisfactory assessments. Intelligent agents designed to pass one test of understanding often fall short of others. Rather than approaching understanding as a system state, in this paper, we argue that understanding is a process that changes over time and experience. The only window into the process is through the lens of natural language. Usefully, failures of understanding reveal breakdowns in the process. We propose a set of natural language-based probes that can be used to map the degree of understanding a human or intelligent system has achieved through combinations of successes and failures.
RESUMO
BACKGROUND: General Aviation (GA) pilots who encounter hazardous weather inflight have a high probability of incurring fatal accidents. To mitigate this problem, previous research investigated pilot decision making and the effects of new technology. Limited investigations have examined usability and interpretability of observation and forecast weather products available to pilots. Therefore, this study examined the interpretability of weather observation and forecast reports that GA pilots use for preflight weather planning and the impact of pilot certification level on the interpretability of these displays.METHOD: There were 204 GA pilots (Mean age = 22.50 yr; Median flight hours = 131.0) who completed a 90-item multiple choice Aviation Weather Product Test. The questions portrayed static weather displays available on the NOAA/National Weather Service Aviation Weather Center website. The questions were designed to have high cognitive fidelity in comparison with preflight weather planning tasks.RESULTS: The results revealed overall low mean interpretability scores (Mean percent correct= 59.29%, SD = 16.01%). The scores for observation products and product attributes were lower for student pilots than experienced pilots. Forecast product scores for student and private pilots did not differ, however, student pilot scores were significantly lower than instrument rated private and commercial pilots.DISCUSSION: The low interpretability scores indicate that GA pilots misinterpret weather information provided by most weather observation and forecast products. Possible contributing factors to the low product interpretation scores include poor usability and a lack of training. Future research should measure the usability of weather displays designed for pilots.Blickensderfer BL, Guinn TA, Lanicci JM, Ortiz Y, King JM, Thomas RL, DeFilippis N. Interpretability of aviation weather information displays for general aviation. Aerosp Med Hum Perform. 2020; 91(4):318-325.