Anti Monopoly Regulation Of Internet Companies Abusing Crawler Agreements In Bilateral Markets

In today’s information age,the Internet industry is booming,and many customary rules have been recognized and abided by the participants in the long-term business dealings.Crawler protocol,as one of the important conventions in the Internet industry,is often used by website owners to tell web crawlers which content of their website can be crawled or not.It was originally designed to improve the efficiency of information sharing in the Internet age,to protect the normal operation of the server of the website owner,and to prevent the disclosure of the private information of network users.However,as can be seen from the case of "Qihoo v.Baidu" in 2013,some Internet enterprises in the dominant position in the market abuse crawler agreement,making it a tool to exclude and restrict competition.This practice obviously violates the original wish of the founders of crawler agreement and hinders free and effective competition.If it is allowed to develop,it will certainly harm the interests of consumers and hinder the sustainable and healthy development of the Internet industry.This behavior should be regulated by law.However,the Internet industry belongs to an emerging industry,and the market it is in reflects the obvious two-sided market attribute,which is very different from the one-sided market served by the traditional industry.China’s current Anti-Monopoly Law has come into effect since 2008,and its regulatory objects are mainly traditional industries with unilateral market attributes.Facing emerging industries in some aspects,it shows obvious weakness and helplessness.Therefore,starting from the case of "Qihoo v.Baidu",this paper analyzes whether China’s current Anti-Monopoly Law can be used to regulate the crawler agreement from two aspects of feasibility and necessity,by understanding the background of crawler agreement and combining with the current abuse behavior of crawler agreement.And according to the two-sided market attribute of Internet enterprise service market,the paper concludes that the characteristics of Internet enterprise’s network cross externality and natural monopoly pose a great challenge to the existing Anti-monopoly Law in defining relevant market,identifying dominant market share and lagging law enforcement team.Finally,the author puts forward some measures to define the relevant market,such as bilateral comprehensive consideration of the platform,perfecting the putative monopolist test method and following the principle of case analysis.It is suggested to take the number of users as the standard to measure the market share,to pay attention to the natural monopoly of the market,and to consider the dominant market by integrating various factors.