Detecting Faking On Self-reported Personality Measures

Conceptually,faking refers to motivation-driven or goal-driven interactive behavior involved in individual traits and surrounding situations,which results in inaccurate or elevated self-image.Since self-report scale is the most common type of personality tests,the research interests concerning about detecting faking have been lasted for all the time.Modeling faking by advanced model-based approach obtained methodological and technological support with principle improvement and technology progress of test theory and mathematical model.At the same time,considering the development of computer software programs,it is possible to bring mathematical calculations to success for integrating complex measurement models and statistical models.Therefore,investigating on faking detection possesses the base of various model-based methods.Correspondingly,advanced statistical analysis techniques could be well applied in the related field of psychometrics and personality assessment for solving concrete issues.The thesis has theoretical significance to faking research as preliminarily conducting principal system and procedure for detecting faking on self-reported personality measures,and providing theoretical basis for following investigations along with faking research system;Methodologically,on the basis of modern test theory models and mathematical model approach,the thesis constructs a new research mode for detecting faking to solve psychology issues by applying various statistical methods and techniques;Practically,the thesis provide guidelines for appropriately using self-report scales and treating measure outcomes,as well as benefit the development and improvement of personality testing tools.The thesis has its own innovations on detecting faking.In the context of lacking research frameworks and operational procedures systematically based on model-based methods,the thesis focused on the perspective of interdiscipline that integrated by psychometrics and statistics to lead into faking detection via mathematical models and measurement models.In this way,not only the analysis level of faking detection research can be promoted,but also the application of statistical theory and approach can be extended into other subject fields such as psychology;Furthermore,the thesis developed a research framework in which both quantitative and qualitative dimension of faking were considered,modern test theory was taken as principle basis and model-based methods were regarded as research ideas.This innovation makes a breakthrough in the single and base research mode of faking detection so that to overcome the limitation of both classical test theory and detection scale methods.Specifically,we focused on the faking detection on self-reported personality scales,and explored how to model and apply personality response data under distinct responding conditions based on model-based methods such as item response theory(IRT)models,factor analysis(FA),structural equation modeling(SEM)and mixture models,as well as discussed the performance of each model for detecting faking.Solved key problems including: how the effectiveness of detection scale methods is when detecting faking;how to use unfolding IRT models for detecting faking on item level;how to collect the evidence of faking by item level analyses under the framework of latent variable modeling;the effects of faking on personality measures;how to explain faking scores and its relationship with traits distortion;whether the faking-related heterogeneity of population exists,and what the differences are about the distinct response patterns.There are five parts of research work that contained nine studies in total.Part One is the research reason and the raise of issues.As a session of this part,study 1employed systematic review to investigate Chinese faking literature in psychology.On the whole,fundamental research was mainly focused on the conceptualization of faking and the summary of methods and techniques.Among faking detection empirical studies,only a small amount of them were based on model-based methods,but rarely referring to advanced statistical models;the main popular IRT models were one-parameter logistic model or binary IRT models,very seldom mentioned polytomous unfolding IRT models.Then the interested issues were raised from the findings of study 1,and the logic framework of entire thesis has been conducted.Part Two is study 2,which uses traditional identifying scales to detect faking and assesses the effectiveness of detection scales.Each participant of undergraduates in the sample was instructed to both fake and honestly respond to all tests.The MCSD was used as identifying scale,and Conscientiousness Scale and Neuroticism Scale based on Big Five dimensions were employed as personality instruments.The differences between within-subjects honest and faking scores represent the amount of faking.The results suggest statistically significant correlations between MCSD and the amount of faking on conscientiousness and neuroticism.However,the classifications of suspected fakers identified by the amount of faking and identifiers flagged by MCSD scores are independent.In addition,the diagnostic accuracy of faking detection tools is out of expectation,as well as the results that the amount of faking regressed on MCSD scores are not reasonable.In summary,the use of SD scales for detecting faking is not adequate.Methodology,analyses that only relay on scale scores level are not recommended.Instead,more precise assessment that based on item level is a better consideration.Part Three is study 3,which applies unfolding IRT models(ideal point models)for detecting faking on personality measures.By modeling data sets under honest and faking response conditions via generalized graded unfolding model(GGUM),the shifts of the item location parameter estimates occurred on all items across conditions.Moreover,the changes of options on the function of response-trait shown that,across honest and faking response conditions,the peak values of endorsement probability transferred on the continuum of conscientiousness and neuroticism.In conclusion,GGUM is capable of detecting faking and capturing faking-related change on item level.Moreover,when GGUM is applied in practice,the balance between “benefit” and “cost” should be under consideration in terms of the mathematical complexity of model on parameter estimation.Part Four consists of four studies,which focuses on factor analysis and structural equation modeling for detecting faking on personality measures.In this part,measurement errors can be handled when latent variables modeling approach is employed.Study 4 employed one-factor analysis model to fit responses of conscientiousness and neuroticism from both honest response condition group and faking response condition group.Consider the magnitude of factor loadings,model-data fit indicators and the variation of measurement errors,it can be concluded that,on the one hand,the effects of faking on psychometric characteristics are limited accompanied with trait difference;on the other hand,faking could shape the quality of personality measures to a certain degree to increase the measurement precision.Study 5 examined measurement invariance of responses between honest and faking conditions using multi-group confirmatory factor analysis(MGCFA).The findings revealed that the basic measurement structures of both Conscientiousness Scale and Neuroticism Scale were constant between honest condition and faking conditions and weak measurement invariance was also confirmed across conditions.That is,factor loadings are equal for both honest condition and faking condition.However,the next more restrictive model has been rejected,which means strong measurement invariance cannot be confirmed.It indicated that there might be systematic response bias in answering personality items.The differences of item interceptions hence can be interpreted as systematic tendency on measured contents between honest group and faking group.Since the item interceptions are not invariant,the differences of raw scores between conditions should not be attribute to the mean differences of latent factors.Consequently,further examinations are needed for mean differences of latent factors.Recall the findings of study 5,weak measurement invariance was hold between honest group and faking group.On the basis of that,study 6 obtained a prerequisite for further discussing the means structure of personality traits and differential item function(DIF)across groups.Multiple Indicator Multiple Cause model(MIMIC)was used to model conscientiousness and neuroticism data from honest and faking response groups,respectively.The results displayed that group differences were statistically significant on factor means of latent traits.Similar significant effects of mean differences on latent factors appeared in different levels of age,but did not appear in gender.In addition,with regard to biased items,MIMIC model identified several conscientiousness biased items between faking group and honest group,but less than that found in study 3.Unfortunately,no biased item was found in Neuroticism Scale between groups.Therefore,IRT models are more recommended for DIF compared to SEM methods.Based on hybrid structural equation model,study 7 explained the SD scores and assessed the relationship between social desirability and conscientiousness or neuroticism.Recall the findings of study 2,the raw scores of MCSD are not adequate to be indicators of the effect of faking.Still,the results under latent variables modeling framework indicted that social desirability can not significantly predict faking effects on both Conscientiousness Scale and Neuroticism Scale,when considering social desirability as substantive personality characteristics.However,different results occurred when modeling social desirability as a covariate to reflect response style.Significant differences of estimated factor means on conscientiousness and neuroticism were found between high desirability group and low desirability group.People with strong response style would predict greater conscientiousness factor means and lower neuroticism factor means.Synthesis of the above,social desirability can be prediction for faking effects as bias on response style,but not as substantive personality traits.Part Five consists of study 8 and study 9,which identifies faking on personality measures based on mixture models,aiming at examining the qualitative distinctive nature of faking except for its quantitative difference.Study 5 has found that the invariance of structural parameters between different response groups was not confirmed.It means that there may be sample heterogeneity among corresponding parameters.Thus,study 8 conducted latent class analysis(LCA)to explore the different ways in which individuals responded to personality tests.The results demonstrated that target population can be assigned into three latent classes which represented three different ways to answer personality tests: honestly responding,slight faking and extreme faking.Correspondingly,estimated factor means existed significant difference across different classes.The findings evidenced that individuals did not respond to personality items with the exactly same manners,but in qualitative distinct responding ways.However,study 8 could not answer how the distinctions were.Study 9 integrated Rasch model and LCA to be a so called mixture IRT model to discriminate heterogeneity in population who faked personality tests.Mixture model identified two latent classes in faked respondents,which means there are different faking types on personality measures: slight faking and extreme faking.Response pattern of individuals was quite similar within each faking type,but distinct different between faking types,as response patterns were impacted by latent class to which individuals belonged.The differences of response patterns were reflected on the change of item location parameter estimates and the function of response-trait on each item between latent classes.On the basis of above,mixture Rasch model is appropriate for distinguishing the different types of faking so that testing the exploration results of responding manners in study 8.Compared to GGUM,mixed Rasch model can provide much more information about classification.However,some parts of information would also be lost due to using dichotomous IRT model for fitting polytomous response data.In conclusion,we summarized the whole research work and listed as follows:(1)With respect to the faking-related quantitative differences,the scale-based method that using social desirability scales to detect faking on the basis of total scale-scores analysis is not powerful enough.Therefore,we recommend model-based methods that are conducted in the SEM framework to estimate mean differences between different response conditions with measurement errors under consideration;(2)Regarding to the qualitative distinction of faking,LCA can be used to distinguish unknown response manners of target population.A better choice should be mixture IRT model,because not only the types of faking can be identified,but also people fake to which extent can be specified;(3)Speaking of the faking-related change on item level,ideal IRT model(e.g.,GGUM)performances well in detecting shifts of item location parameters to identify DIF.Therefore,unfolding IRT models are appropriate for polytomous response Likert scales.If dichotomous response data are collected,mixture Rasch model can be employed for detecting faking;(4)Focus on the impact of faking on personality measures,the assessments of CFA and MGCFA are consistent in considering faking has limited negative effects on psychometric characteristics.However,the conclusion is lack of evidence derived from structure model invariance.Besides,comparing to honest response condition,the model-data fit is much better under faking condition.It demonstrates that faking may artificially improve personality tests.Also,non-invariance on item inceptions indicates that latent mean structure changes due to faking;(5)In terms of the nature of social desirability responding,hybrid SEM model supports “situation” opinion rather than “trait” opinion.Faked scores mainly reflect temporary change related to responses instructed by context demands,whereas the inflated scores are independent with true personality characteristics.In addition,response style can significantly predict the distortion on personality traits.The findings can be concluded as:(1)Detecting faking is an important issue in personality measures,however,empirical research account for a small proportion;(2)Faking detection comes to be successful by using IRT models,SEM models and mixed models;(3)Item level analysis is prior to scale level analysis.Unfolding model and mixed IRT model are the most effective methods.The more complex the model is,the more informative it is.However,it also means it is harder to estimate.If item location estimates are less interested,SEM models could be good choice;(4)Self-report scales are critical tools for personality assessment.It is necessary to detect faking by internal procedure for following correcting,controlling and against with faking.