Meta Updates Crawl Clause In Commercial Terms
Meta has revised its crawl clause to remove training, affecting businesses using its products. The change takes effect October 30.
Original reporting · searchenginejournal.com
Original: “Meta Drops Training From Its Crawl Clause And Keeps The Crawl via @sejournal, @MattGSouthern”
Read “Meta Drops Training From Its Crawl Clause And Keeps The Crawl via @sejournal,…” at Search Engine Journal →
How this was written. Adtomix wrote this article with a language model from Search Engine Journal’s reporting (model: @cf/meta/llama-3.3-70b-instruct-fp8-fast); it describes that reporting rather than adding to it. The original, linked above, is the full story.
Key points from Search Engine Journal’s report
- Meta has revised its crawl clause to remove training, effective October 30
- The revised clause connects crawling to advertising and business products, as well as maintaining the integrity and security of Meta's products
- Legal experts have raised concerns that the revised wording may still be too broad
- Businesses can use robots.txt to control which areas of their website are crawled
- The revised terms may still allow Meta to scrape entire sites, depending on the interpretation of the language
Meta has made a significant change to its Commercial Terms, which will go into effect on October 30, according to Search Engine Journal. The company has removed training from its crawl clause, a move that affects businesses that use Meta's products. This revision comes after concerns were raised about the original wording, which some believed could be used to justify scraping news sites for AI training.
The original clause had sparked controversy, with the European Publishers Council and the UK's News Media Association expressing objections. The revised clause now connects crawling to advertising and business products, as well as maintaining the integrity and security of Meta's products. This change aims to clarify the purpose of crawling, which is still permitted under the agreement.
The revised clause allows Meta to crawl websites associated with a business's products or services, but only for specific purposes. These purposes include providing, developing, and improving advertising and business products, as well as maintaining the integrity and security of Meta's products. The company has stated that this revision does not change how it uses web crawlers.
Meta spokesperson Daniel Roberts explained that the update was made to clarify the terms, which only applied to ads and business products. He also stated that the change does not affect how Meta uses web crawlers and is not about scraping news sites. However, some legal experts have raised concerns that the revised wording may still be too broad.
The legal experts have pointed out that the term 'develop' could be interpreted to include 'train', which could potentially allow Meta to use crawled content for AI training. Additionally, the language regarding 'business products' is not clearly defined, which could lead to confusion. The experts have also noted that the revision does not restrict the language regarding websites 'associated with your products or services', which could allow Meta to scrape entire sites.
Meta has cited robots.txt as a method for businesses to control which areas of their website are crawled. The company has also listed separate crawlers for AI training and Meta AI search, which can be controlled using robots.txt. However, it is unclear whether these methods limit what a business grants when they accept the terms.
The revised terms will take effect on October 30, and businesses are advised to review their robots.txt file to ensure it is configured correctly. Meta has updated the preview of its Commercial Terms, and businesses should check the clause again before the effective date.
Questions and answers
What is the purpose of Meta's crawl clause?
The purpose of Meta's crawl clause is to allow the company to crawl websites associated with a business's products or services for specific purposes, including providing, developing, and improving advertising and business products.
Can businesses control which areas of their website are crawled?
Yes, businesses can use robots.txt to control which areas of their website are crawled.
What are the concerns about the revised wording of the crawl clause?
Some legal experts have raised concerns that the revised wording may still be too broad, potentially allowing Meta to use crawled content for AI training or scrape entire sites.
