Reddit CEO Criticizes the Hassle of Blocking Search Engines like Bing; Microsoft and Others Accused of Freeloading Data

Reddit, the well-known online community, has already banned all search engines except Google from indexing its content. This exclusive allowance is because Google pays Reddit $60 million annually for content licensing, enabling Google to use this content for artificial intelligence training.

Other search engine developers who refuse to pay the fee are naturally blocked. Microsoft's Bing search head had previously stated that, as of September 2023, Bing has been providing website crawling controls that can manage how Bing crawls their content.

However, it was later revealed by Microsoft that Reddit had indeed blocked Bing's crawlers and other data, affecting not only Bing's access to Reddit's content but also other search engines that rely on Bing, such as DuckDuckGo.

As a result, users can no longer find Reddit content through Bing or DuckDuckGo and must switch to Google to find useful or latest posts and comments from the Reddit forums.

In a recent interview, Reddit's CEO, Steve Huffman, disclosed the reasons behind the blockade:

Microsoft has been using Reddit's data for training its artificial intelligence models, incorporating Reddit's content into Bing searches "without knowledge". Additionally, two AI developers, Anthropic (with its main product Claude) and Perplexity (with its AI search engine of the same name), have also been training their systems using data from Reddit.

Microsoft, Anthropic, and Perplexity seem to act as if all content on the internet is freely available for their use, which Huffman pointed out as their true stance.

Huffman also mentioned the inconvenience of blocking these companies, indicating a shift from the traditional scenario where search engines index website content and bring traffic that could be converted into revenue. Now, search engines scrape data to train models, making the exchange no longer equivalent in value.

Moreover, it has been reported that companies like Microsoft have outright refused to negotiate content licensing with Reddit. Even with the search engine ban in place, they have no intentions of paying the forum any fees for access or for purchasing content data.