Research On The Impact Of System Transparency And Reliability On Human-agent Collaborative Decision-making Performance

With the development of intelligent technology,human-agent collaborative decision-making has become a common working mode in the field of complex systems.Research shows that low system transparency can lead to doubts or misunderstandings about the accuracy and correctness of its decision recommendations,thereby affecting the efficiency of collaborative decision-making.There are two main ways to define system transparency in existing research: one is to represent transparency simply by the visibility or amount of information of the system,and the other is to define system transparency as the comprehensibility and predictability of system behavior and other characteristics in the human-agent interaction process.System transparency is directly related to the information perceived by the operator,and the quality of information will directly affect the system transparency.Existing research generally uses a single dimension such as information quantity to represent transparency for theoretical or experimental research,and lacks inspection of the information quality to improve system transparency.In addition,research on the interaction between system transparency and other system attributes such as reliability is also relatively scarce in the literature.In response to the aforementioned shortcomings,this paper first explores the theoretical connotation of system transparency and,based on the theoretical framework of information quality,points out that system transparency consists of four parts: essential transparency,presentation transparency,situational transparency,and other dimensions such as information comprehensibility and information completeness.Different levels of transparency were designed in the context of decision-making tasks such as post-disaster investigation and military reconnaissance.Three human-machine collaborative task experimental platforms were developed based on Unity3 D,and three experiments were designed to investigate the impact of information comprehensibility and information completeness on collaborative decision-making performance,as well as the interaction of information completeness and dynamic reliability on decision-making performance.Study 1 explored the impact of information comprehensibility on decision-making performance and considered the moderating effect of individual cognitive style.A single-factor 2-level experiment was designed,and 54 science and engineering university students were recruited to complete a path decision-making task in post-disaster investigation.The dependent variables,such as adoption intention and adoption impact time,were collected,and the experimental data were analyzed using a general linear model.The results showed that the improvement of information comprehensibility significantly increased the adoption impact time,and there was no significant interaction between individual cognitive style and information comprehensibility on decision-making performance.This suggests that making system information more comprehensible leads participants to spend more time in the judgment and analysis stage,which may help them make the correct decision.Study 2 explored the impact of information adequacy on decision-making performance and considered the moderating effect of individual cognitive style.A single-factor 3-level experiment was designed,and the same participants as in study one completed a photo decision-making task in post-disaster investigation.The dependent variables,such as decision time and photo score,were collected,and the experimental data were analyzed using a general linear model.The results showed that increasing information adequacy significantly increased decision time and photo score.In addition,compared to holistic participants,sequential cognitive style significantly enhanced the effect of transparency on photo decision-making time.This suggests that providing more effective information allows participants to have more space for judgment and analysis,and is beneficial to task performance improvement.Study 3 explored the impact of transparency and reliability on collaborative decision-making performance and their interaction.This study used a new method of dynamic reliability operation,with military reconnaissance as the task background,and designed a 2(information adequacy)× 2(variable reliability)mixed experiment.Sixty university students were recruited to complete 60 path decision-making tasks,and decision time,correct decision times,and confidence were collected.The experimental data were analyzed using repeated measures ANOVA or generalized estimating equations.The results showed that reliability had a significant main effect on decision time and information,and transparency had a significant main effect on correct decision times and correct acceptance times.Additionally,transparency and reliability had significant interaction effects on confidence,correct decision times,and correct acceptance times.This suggests that improving transparency allows participants to understand the reasons for system errors,better understand the system’s capabilities,and more easily form optimal collaborative strategies with the system.When task rounds were included in the analysis,it was found that under low transparency,there were significant differences in correct decision times and correct acceptance times in the last 10 rounds of the task under two reliability conditions,while under high transparency,there were no significant differences in correct decision times and correct acceptance times.This indicates that improving system transparency weakens the negative impact of system errors on subsequent tasks.This study constructs a new theoretical framework for transparency and studies the impact of information comprehensibility and information completeness on collaborative decision-making performance based on path decision-making and photographic survey tasks.The interaction of information completeness and reliability on collaborative decision-making performance is also studied by combining path decision-making tasks.The results enrich the theory of system transparency and provide reference for the information design of decision support systems.