copyright reference-request
The legislative limitations on web pages mirroring
<p>Are there legislative limitations on web pages mirroring?</p>

<p>Let me explain the task. If to publish some article with HTTP links in any web site, the source of information that has been used could be checked immediately by clicking on the links, (Wikipedia do such as to provide the changeability of objectivity of the information). However, in time the source web page that has been used as reference could be deleted, as result it becomes impossible to check the source of information. The way out is to create the mirror of source web page and include this mirror in own web site. So even if source web page will be deleted, it’s still possible to refer on the mirror.</p>

<p>Let me give an example of the conflict situation. Mr. A in a fit of temper published on his home page the unfavorable information in respect to himself, then later comes around and delete this information. However Mr. B, who has the interest to Mr. A, already quoted the deleted information with http-link and also just in case created the mirror of the home page of Mr. A. Although Mr. A already delated the information, Mr. B still can use the mirror to refer on source of information. All that left for Mr. A to resque his requtation is to hire the high-payed lawyer and prove the false fact that mirrored information has been edited. </p>

<p>OK, it was the private example, however to exhaust this theme, it’s required to consider also the conflict situations with organizations. Here is another instance. Do you remember the Snowden's exposure about global surveillance system? E. g. Mr. C going to publish the essay about these systems based on Snowden’s information on his home page. He will be refer on newspaper articles that has been published also in the internet, e. g. “The Guardian” or “The New York Times”. But if CIA threats “The New Yourk times” and demand to delete the publications about surveillance systems, of course the newspaper administration will do it. However Mr. C created the mirror of “The New York times” news page and it will be available to check if from web page of Mr. C. What will be then, I can just suppose, but let us come back to original question and discuss the law related with web page mirroring.</p>

<p>I think you can look at how archive.org does the capture and mirror of the internet.  They have a fairly long and detailed <a href="https://archive.org/about/faqs.php" rel="nofollow">FAQ</a> discussing lots of theses issues.</p>

<p>Crawling websites is generally not a problem, and it is the basis for how the internet works -- you would never be able to use google if it was not for the fact that they had crawled all the information already.</p>

<p>However material is still subject to copyright -- so the usage of the information is key.  Just like with google, nobody will object if you pull the site content assuming that you are not just trying to republish the content in breach of their copyright.  They will however object if you are trying to pull a copy every minute as that would considered an attack on their servers.</p>

<p>There is however information which website owners explicitly state they don't want automated crawlers to access.  This is by convention declared in a file on the website called <a href="http://www.robotstxt.org/" rel="nofollow">robots.txt</a>.  You will see in the archive.org FAQ that they do not crawl those pages.</p>

<p>Likewise, archive.org is removing content from website owners who give legal notice of not hosting their content - They will have to in order to comply with the DMCA.</p>
