As we reported over the weekend Agence France-Presse is suing Google for $17.5 million. We reported that Agence France-Presse doesn't know how to write a robots.txt file.

We were wrong on that. Carl Malamud (no picture, sorry -- he's shy) found a reference to a robots.txt file on the Agence France-Presse site at http://www.afp.com/robots.txt

While AFP stories are not directly linked to Google News as of March 21, affiliates' publishing of those stories are.

The Agence suit, filed in the U.S. District Court in Washington, D.C., alleges Google News "stole" its content by linking to it, with headlines and inserting thumbnails of photos. No claim is made that Google cached whole copies of the news agency's stories.

A U.S. court ruled in 2000 that it's perfectly legal to link deep into another site. But it is also legal to write a program that prevents robots from linking to any page. This is the robots.txt file mentioned earlier.

Some folks have written to say a court loss by Newsbooster, a Danish site, re-opens the question of linking. But Newsbooster's business model was to sell e-mails, not publish a database.

Here is the full AFP robots.txt file found by Malumud:

User-Agent: *
Disallow: /beta
Disallow: /francais/news
Disallow: /english/news

Malamud also found this file at Archive.org

The impact this has on a suit for damages depends on further discovery:

• When did the robots.txt file go in?
  
• Did Google News ignore the robots.txt file after it was put in?
  
• Is Agence France-Presse really alleging that links into its affiliates should be prohibited?
  
• Does Agence France-Presse really want to disappear from the Web or does it just want to be paid for all incoming links?
  

I'll add this personal note. If you really need written permission to link to a Web address, then the Web technology itself is against the law. If the Web technology itself is against the law, then the law needs to be changed, and will be.