Google keeps a rapidly extending list of copyright-infringing URLs which they haven’t listed yet. This blacklist makes sure that these links are never put forward to the search engine. Appreciatively, a new update in the transparency report allows us to know how many non-registered links every takedown notice includes, which is astonishingly high in some cases.
Over the years, Google has had to deal with a continuous rise in takedown requests which target pirate sites in search results.
The total number of ‘discarded’ URLs just touched 3.5 billion, and millions more are added daily.
Although that is not new, the thing that is new is Google sharing some further insight into the nature of these requests.
Fact is, millions, if not hundreds of millions, of the links copyright holders target, have never shown up in Google’s search index.
Earlier in the year Caleb Donaldson, Google copyright counsel, disclosed that the company had started to block non-listed links ‘prophylactically.’ Meaning, Google blocks URLs prior to them appearing in the search results, as some sort of piracy vaccine.
Donaldson stated “Google has critically expanded notice and takedown in another important way: We accept notices for URLs that are not even in our index in the first place. That way, we can collect information even about pages and domains we have not yet crawled.”
Additionally, he said “We process these URLs as we do the others. Once one of these not-in-index URLs is approved for takedown, we prophylactically block it from appearing in our Search results.”
Regrettably, Google gave no easy way to see how many links in a request were not listed, but that has now been rectified.
The previous week or so Google added a new signal to its DMCA transparency report showing how many of the submitted URLs in a notice are not listed yet. In some cases, it is most of them.
For example, Mexican branch on the anti-piracy group APDIF is one of the most active DMCA reporters and has requested Google to eliminate over a million URLs last week alone.
Given below are links where the majority of them appear to be non-indexed links.
Google now reporting non-listed takedown requests
These URLs are not removed well because they weren’t listed. Stated earlier by Google, they are kept on a separate blocklist instead, which denies them from being added hereafter.
Apart from APDIF, Rivendell is also an active sender with a high rate of non-listed links, often well over 50%.
While not all reporting agencies have such high percentages as APDIF, it is clear that millions of non-listed pirate URLs are added to the preventive blocklist every month.
Technically, the DMCA takedown procedure is meant for links and content which really exist on a server, but maybe Google wants to take it a step further themselves.