copyright data-ownership data-storage
Is it legal to scrape a website and create my own database?
<p>I'm working on a website that needs a database which is available publically on another site (let's just says it is IMDB). It'd be great if I could create my database by scraping IMDB. It'll be a one-time thing. Once I get my database, my website can work on its own (without scraping or anything). So, Is it legal?</p>

<p>Usually the answer to "is it legal?" to wholesale and methodically copying another website (or any other published work, be it a book, movie, music, article, internet resource, etc.) is clear: <em>of course not.</em></p>

<p>It doesn't matter if it is a one time scrape or a continuous crawl and copy; it's still copying and there are copyright laws around the world, albeit different in many countries. But still, copyright exists, everywhere.</p>

<p>Read more: <a href="https://law.meta.stackexchange.com/questions/541/i-have-a-question-about-copyright-what-should-i-read-before-i-ask-it">I have a question about copyright. What should I read before I ask it?</a> and <a href="https://law.stackexchange.com/questions/tagged/copyright">https://law.stackexchange.com/questions/tagged/copyright</a></p>

<p>Copying something in its entirety is not Fair Use, either; Fair Use is the use of a small amount of material, not a complete work. See <a href="http://fairuse.stanford.edu/overview/fair-use/what-is-fair-use/" rel="nofollow noreferrer">http://fairuse.stanford.edu/overview/fair-use/what-is-fair-use/</a> and <a href="https://law.stackexchange.com/search?q=fair+use">https://law.stackexchange.com/search?q=fair+use</a></p>

<p>To be sure, read the TOS for the site you want to scrape. If there is no TOS, or the TOS says you can scrape and copy, go for it. If the TOS says scraping is not allowed, you may still be able to license the content; ask them.</p>

<p>If the TOS is like IMDB's, scraping and copying is illegal, but licensing is possible. Read IMDB's TOS <a href="http://www.imdb.com/conditions" rel="nofollow noreferrer">http://www.imdb.com/conditions</a> :</p>

<p><em>(my <strong>emphasis</strong>)</em></p>

<blockquote>
  <p>Copyright</p>
  
  <p><strong>All content included on this site in or made available through any
  IMDb Service, such as text, graphics, logos, button icons, images,
  audio clips, video clips, digital downloads, data compilations, and
  software, is the property of IMDb or its content suppliers and
  protected by United States and international copyright laws.</strong> The
  compilation of all content included in or made available through any
  IMDb Service is the exclusive property of IMDb and protected by U.S.
  and international copyright laws. All software used in any IMDb
  Service is the property of IMDb or its software suppliers and
  protected by United States and international copyright laws.</p>
  
  <p>License and Site Access</p>
  
  <p>Subject to your compliance with these Conditions of Use and your
  payment of any applicable fees, IMDb or its content providers grants
  you a limited, non-exclusive, non-transferable, non-sublicenseable
  license to access and make personal and non-commercial use of the IMDb
  Services, including digital content available through the IMDb
  Services, and not to download (other than page caching) or modify this
  site, or any portion of it, except with express written consent of
  IMDb. Additional license terms may be found in the Terms. <strong>The IMDb
  Services or any portion of such services may not be reproduced,
  duplicated, copied, sold, resold, visited, or otherwise exploited for
  any commercial purpose without express written consent of IMDb.</strong> This
  license does not include any resale or commercial use of any IMDb
  Service or its contents or any derivative use of this site or its
  contents. All licenses are non-exclusive and all rights not expressly
  granted to you in these Conditions of Use or any applicable Terms are
  reserved and retained by IMDb or its licensors, suppliers, publishers,
  rightsholders, or other content providers. You will use all IMDb
  Services in compliance with all applicable laws.</p>
  
  <p>Robots and Screen Scraping: <strong>You may not use data mining, robots,
  screen scraping, or similar data gathering and extraction tools on
  this site, except with our express written consent as noted below.</strong></p>
  
  <p>Framing: <strong>You may not frame or utilize framing techniques to enclose
  any trademark, logo, or other proprietary information (including
  images, text, page layout, or form) of IMDb without express written
  consent.</strong></p>
</blockquote>

<p><strong>You could license IMDB's content:</strong></p>

<blockquote>
  <p>Licensing IMDb Content; Consent to Use Robots and Crawlers: If you are
  interested in receiving our express written permission to use IMDb
  content for your non-personal (including commercial) use, please visit
  our Content Licensing section or contact our Licensing Department. <strong>We
  do allow the limited use of robots and crawlers, such as those from
  certain search engines, with our express written consent. If you are
  interested in receiving our express written permission to use robots
  or crawlers on our site, please contact our Licensing Department.</strong></p>
</blockquote>
