The Identification Research Of "Bie Shuo" For Chinese Information Processing

In the Chinese part-of-speech tagging,it is always difficult to label Chinese multi-category words."Bie Shuo" is a linguistic unit which is a typical Chinese multi-category word.It has three attributes: verb phrase,conjunction and discourse marker.After sorting,summarizing and comparing the fixed sentence patterns,fixed collocation words and contextual characteristics of "Bie Shuo",this paper summarizes the identifying rules that can help the computer to recognize the specific attributes of "Bie Shuo",so as to help improve the tagging accuracy of the word.First,this paper introduces the specific attributes of "Bie Shuo"."Bie Shuo" has three attributes.One is the verb phrase "Bie Shuo 1",one is the conjunction "Bie Shuo 2",and another is the discourse mark "Bie Shuo 3".The three attributes differ from each other in terms of usage and function."Bie Shuo 1" mainly has three functions: ending current topic,preventing future topic and subjective evaluation.There are three functions of "Bie Shuo 2" : contrastive statement function,emotional attitudes function and semantic emphasis function.The function of "Bie Shuo 3" mainly reflects the interpersonal interaction function of the meta-pragmatic function.Secondly,we sort out and quantify the syntactic position and structural rules of "Bie Shuo".When "Bie Shuo" is at the beginning of a sentence,the probability of "Bie Shuo" being used as a conjunction is 70.99% and being used as a verb phrase is 28.97%,and is less likely to be used as discourse markers.When "Bie Shuo" is in a sentence,the probability of "Bie Shuo" being used as a verb phrase is 52.23%,and the probability of "Bie Shuo" being used as a conjunction is 47.50%.The probability is not much different,and the probability of using "Bie Shuo" as a discourse marker is still small.When "Bie Shuo" is at the end of a sentence,the probability of using it as a discourse marker is 81.94%,and only 17.61% is used as a verb phrase.The probability of using it as a conjunction is very small.In the case of separate sentences,"Bie Shuo" was used as a discourse marker 91.30% of the time,and only 8.70% of the time as a verb phrase.Never used as a conjunction.Contrast in the rules of structure,we sort out the "Bie Shuo" three attributes of the fixed sentence and fixed collocation.There are 10 kinds of fixed sentence and collocation of "Bie Shuo 1",and 6 kinds of "Bie Shuo 2".We sort out all kinds of "Bie Shuo 3",which are used as discourse markers.There are 23 types,and nine classes of the fixed word collocation.We also sorted out the context of "Bie Shuo" and made a quantitative analysis based on the syntactic position of "Bie Shuo".According to the analysis,we sorted out the context contents corresponding to high probability when "Bie Shuo" is used in the beginning,in the sentence,at the end of the sentence or alone.Finally,we integrated all the rules,established the corresponding rule set,formulated the identification process,and verified the identification rules.In this chapter,we set up a set of fixed sentence patterns.The codes are "JS1","JS2" and "JS3",which correspond to "Bie Shuo 1","Bie Shuo 2" and "Bie Shuo 3" respectively.The collocation word set is established too.The codes are "DPC1","DPC2" and "DPC3",which correspond to the three attributes of"Bie Shuo".Combining the syntactic position,we established the context content set.The code is "SW2","SX1","SX2","ZX1" and “ZX2".According to the determination of fixed sentence pattern,we will make the first-level rule(R1).The second-level rule(R2)is according to the collocation word set.The third-level rule(R3)is according to the determination of the above content.And the fourth-level rule(R4)is according to the determination of the following content.We will screen and mark the "Bie Shuo" step by step and finally output the result.After that,we use the original corpus and the third-party corpus to verify the effect of the identifying rules.Finally,the tagging accuracy of the original corpus is 94.59%,and that of the third-party corpus is 97.49%.Through the feature extraction and induction of "Bie Shuo",we sorted out the unique rules of "Bie Shuo" for different attributes.The verification of identifying rules also prove that it is effective to distinguish the three attributes of "Bie Shuo" by this method.