internet software
Is it legal to change user agent?
<p>Assume (hypothetically), I'm using jsoup to download certain news website in my public android app. When I don't change user-agent in <code>jsoup</code> it redirects me to another website.</p>
<p>Is it legal to change the user agent of my public app, so the user of the app will get the correct news article? Does it matter that the website is scraped a little after downloading it?</p>

<h2>Read the ToS of the website</h2>
<p>News websites implement such measures because unscrupulous people infringe on the copyright of their articles. Typically, such websites <strong>also</strong> have terms os service that explicitly makes usage of the website contingent on not scraping the website. For example, <a href="https://help.nytimes.com/hc/en-us/articles/115014893428-Terms-of-Service" rel="nofollow noreferrer">the New York Times</a> say:</p>
<blockquote>
<ol start="4">
<li>PROHIBITED USE OF THE SERVICES</li>
</ol>
<p>You may not access or use, or attempt to access or use, the Services to take any action that could harm us or a third party. You may not use the Services in violation of applicable laws or in violation of our or any third party’s intellectual property or other proprietary or legal rights. You further agree that you shall not attempt (or encourage or support anyone else's attempt) to circumvent, reverse engineer, decrypt, or otherwise alter or interfere with the Services, or any content thereof, or make any unauthorized use thereof. <strong>Without NYT’s prior written consent, you shall not:</strong></p>
<p>(i) access any part of the Services, Content, data or information you do not have permission or authorization to access or for which NYT has revoked your access;</p>
<p>(ii) use robots, spiders, scripts, service, <strong>software or any manual or automatic device, tool, or process designed to data mine or scrape the Content, data or information from the Services, or otherwise access or collect the Content, data or information from the Services using automated means;</strong></p>
</blockquote>
<p>Violating the Terms of Service means, you are in breach of contract and got no license to access any of the data. This allows claims starting with breach of contract and going up to intruding into the servers and violating various computer crime laws. However, do note that having a possible charge does not mean that it will result in a  trial or even conviction.</p>

<p>It is hard to prove a negative (there is no comprehensive list of things that are legal). However a specific user agent name is not mandated by any law, so there can hardly be a law against changing it [*] (think of it this way, instead of using jsoup you could write your own crawler/parser - there is no government agency that would knock at your door and ask you to use a specific name for your software).</p>
<p>Here in <a href="/questions/tagged/germany" class="post-tag" title="show questions tagged &#39;germany&#39;" aria-label="show questions tagged &#39;germany&#39;" rel="tag" aria-labelledby="tag-germany-tooltip-container">germany</a> there is a law against circumventing effective protective measures to get access to a computer system, but it is doubtful that a website would be considered a computer system in the meaning of the law, and relying on the user agent would not be effective protective measures.</p>
<p>A site might try to ban scraping via their terms of service, but that would not always be enforceable, and would not depend on your user agent name. I still would not recommend to rely on data that a company does not want you to scrape, because even without legal measures they have ways to make this infeasible as a business model (and there might be legal issue such as copyright that still apply).</p>
<p>But as far as UA names go, knock yourself out.</p>
<p>[*] If it is a configurable property of your software. The software itself might have terms of service that prevent you from changing its properties, but I take it this is not what you are asking about.</p>
