Can I scrape data from the Internet?

The Internet contains a wealth of useful commercially reusable information. When is it allowed and when is it not allowed to collect it through scraping or otherwise exploit it?

Re-use of data on the Internet

Information and data on the Internet offers interesting opportunities for its (re)use. Think of sites with:

• travel and airline data,
• research results, and
• data on motor vehicles.

It is very tempting to use such data for one's own product or service. However, the fact that something can be found 'freely' on the Internet does not give you a licence to use it just like that.

Depending on the type of information and any applicable agreements, various rules and restrictions may apply that must be taken into account. Where the boundaries lie is not always easy to determine. This is mainly due to the fact that several legal areas can apply at the same time.

The following is a list of possible points to consider if you want to exploit data obtained via the Internet.

Copyright

First of all, the data may be protected by copyright.

Copyright is the exclusive right of the creator of a 'work' to reproduce or publish that work. A work can be a photograph or a text, for example. The work must be the intellectual creation of the author in order to be protected. In short, this means that it must contain some creativity.

In each case, it will have to be assessed whether the data are protected by copyright. Then the question is what reuse of that data is permitted?

In principle, permission from the copyright holder is required for reproduction and publication. But copyright law also has exceptions that you may be able to invoke, such as the right to quote. The method of use is also relevant (is it a copy, or only a hyperlink or embedded link?).

Copyright example

A party operates a radio portal, a website on which hyperlinks can be found to various streams of radio stations (to be found on the websites of those radio stations).

Here, any changes in the URLs of the radio stations could even be adjusted automatically by scraping, so that the links of the portal are always up to date.

Does the radio portal thereby infringe copyright by publishing links to these streams? In a case in which such a situation was at issue, it was ruled that this was not the case. The court referred to the Svensson judgment of the European Court of Justice.

The Court of Justice ruled in this regard:

"However, it is clear from settled case-law that a communication such as that at issue in the main proceedings, which concerns the same works as the initial communication and which, like the initial communication, was effected via the internet and therefore by the same technical means, is covered by the concept of 'communication to the public' within the meaning of Article 3(1) of Directive 2001/29 only if it is directed at a new public, that is to say, a public which was not taken into account by the copyright holders when they authorised the initial communication to the public ....

In the present case, it must be held that making the works in question available via a clickable link such as that at issue in the main proceedings does not lead to their being communicated to a new public.”

On this basis, the court ruled that the requirement of a "new audience" had not been met in this case, so that the radio portal's actions did not constitute copyright infringement:

"The ECJ has already decided in the Svensson judgment that in the circumstances at issue in the main proceedings of that case there was no question of a new public and therefore no relevant communication to the public within the meaning of Article 3 (1) of the Arl. Since the factual circumstances in the present case are similar, it also applies to Nederland.FM that it does not communicate to the public within the meaning of Article 8 (2) of the Vrl with its embedded hyperlinks to broadcasts of radio stations. After all, also in the case of Nederland.FM the 'new public' requirement has not been met.

See the judgment of the District Court of The Hague of 11 January 2017:

ECLI:NL:RBDHA:2017:218

Database law

A collection of purely factual data is not likely to be copyrighted, however it is possible.

Database law is another source of rights that can apply to this type of data. This is based on the Databases Act. A database is:

"a collection of works, data or other independent elements which are systematically or methodically arranged and accessible separately by electronic means or otherwise and the acquisition, verification or presentation of which gives rise to a substantial qualitative or quantitative investment".

So not every data collection is a protected database. In short, there must be one:

Substantial investment has been made in:

• collecting,
• check whether
• presenting the contents of the database.

In practice, this is not always easy to prove.

The holder of a database right cannot prohibit every use of that database. The assessment of what is still allowed, and what requires the authorisation of the database right holder, requires careful examination.

Example database law

You want to build a meta search engine that can search for advertisements in different advertising sites. However, the collections of advertisements may be protected databases. Specifically for 'dedicated' meta search engines, case law has discussed when there is 're-utilisation' of the whole or a substantial part of the contents of a protected database. That is one of the acts that the database right holder may prohibit. Re-utilisation is at issue if this meta search engine:

"- provides the end user with a search form that offers essentially the same functionalities as the database form,

- translates the end-users' queries into the search engine of the database in real time, so that all the data of this database is searched, and

- displays to the end user the results found in the layout of its website, combining the duplicates into one item but in an order based on criteria similar to those used by the search engine of the database concerned to display the results. "

Judgment of the Gerechtshof Den Haag of 6 May 2014, following a reference for a preliminary ruling to the Court of Justice of the European Union:

ECLI:NL:GHDHA:2014:1558

Privacy and data protection

If the data can be traced back directly or indirectly to individuals, the right to respect for privacy and the right to protection of personal data must also be taken into account.

This may also include data which appears to be anonymous. Truly anonymous data is rare.
The fact that this information is freely available on the Internet does not automatically justify its use for any purpose. Not even if the data has been made public by the person concerned.

The EU General Data Protection Regulation sets rules on the processing (including collection) of personal data. Among other things, personal data must be processed in a way that does not harm the data subject.

• lawful,
• properly; and
• is transparent.

Also, personal data may only be processed for

1. well-defined,
2. explicitly defined and
3. justified purposes.

The collection of personal data on a larger scale is particularly risky, especially when the parties involved are not aware of it. The Dutch Data Protection Authority (AP) is critical of the use of personal data for purely commercial interests. According to the AP, this cannot be a legitimate interest on which the processing can be based. There are, incidentally, a number of objections to this. Other European supervisory authorities seem to offer more room for this. But vigilance is still required.

In an administrative law case, however, it was assumed that the municipality of Amsterdam was allowed to use scraping on websites of providers of tourist rental accommodations such as AirBnB. This was done to monitor possible violations of the municipal regulations for tourist rentals. In that case, it was ruled that this method of working did not constitute an unlawful breach of privacy.

See the judgment of the Amsterdam District Court of 27 June 2018:

ECLI:NL:RBAMS:2018:4442

"The general management is allowed to make use of 'scraping' in order to gather and analyse data on, in this case, the website airbnb.nl, by means of software. From the overviews submitted by the general board it appears that these are reviews, with data entry, of guests about an accommodation offered for rent to tourists on the website airbnb.nl. (...)
Insofar as the collection of these data would lead to any infringement of privacy, this is so limited that no special legal basis is required. The supervisors may do this (or have this done) by virtue of their general investigative power based on Article 3:2 of the General Administrative Law Act (Awb) and Title 2 of Chapter 5 of the Awb".

Privacy and data protection example

You want to collect a large amount of messages from users on social media such as Twitter and Facebook to make behavioural analyses for direct marketing purposes.

This is a rather risky activity, as those involved will often not be aware of it. The consequences for those involved are also not always clear. In any case, the fact that the messages have been made public by those involved is certainly not a licence to use them for one's own purposes just like that.

Contract law

Parties can make agreements as to whether or not they are allowed to use certain information. It is therefore possible that certain (managers of) websites or other Internet sources from which you wish to collect data have agreed with you that this is not permitted. This may, for example, be prohibited in general terms and conditions. This is possible even if no rights, such as copyright or database right rest on them. After all, the conditions, if they have actually been agreed, are a binding agreement which you can be bound by. However, it is necessary that the general conditions have been agreed in some way. The mere presence of the general conditions on a website, without the visitor having to accept them, is in principle not sufficient to bind the visitor.

Example of contract law

Here, as with the other examples mentioned above, the question may arise as to which law is actually applicable. After all, the website can be managed by a party in another country than the party that wants to scrape the content of the website. And different countries have different rules about when a contract comes into being. This is a question in the field of international private law (the rules about which law is applicable and which judge is competent in international legal relations).

Such an issue was addressed in the judgment of the Court of Appeal of The Hague of 23 January 2018 in the case of Ryanair/PR Aviation:

ECLI:NL:GHDHA:2018:61

In this case, Ryanair had placed restrictions on the use of flight schedule data in the conditions of its website. But the visitor did not have to actively accept these before the website could be used. PR Aviation offered a website on which consumers could search for flights, using the collection of flight data from the Ryanair website.

First, the court ruled, partly on the basis of the international and Union law agreements in this area, that Irish law was applicable to the question of whether the conditions had become applicable. The Dutch court therefore had to apply Irish law on this point and concluded that the conditions had not been agreed. PR Aviation was therefore not bound by the usage restrictions in the conditions:

"73. Applying these general principles, the Court found that under Irish contract law the formation of a contract requires four constitutive elements, namely: offer, acceptance, consideration and intention to create legal obligations.

(...)

80. In such a case acceptance cannot be assumed. The circumstances that PR Aviation knew or should have known about these terms and conditions of use, that it is a professional party, that it uses(s) terms and conditions of use on its own website via browse-wrapping (see paragraph 3.9 above), and that PR Aviation was subsequently explicitly reminded of Ryanair's terms and conditions of use, for instance in a letter dated 11 January 2008 (cf. paragraphs 3.10 and 98), do not alter this. After all, these circumstances - each in isolation and also taken together - cannot entail that, objectively, a reasonable person would think that PR Aviation intended to accept (the choice of law in) the conditions of use.

(...)
82. In conclusion, there is no question of acceptance by PR Aviation, and under Irish law there is therefore no question of a (choice of law) agreement between the parties. The question of whether there is consideration and/or intention to create legal obligations can therefore be left open."

Under Dutch law, the outcome would probably not have been much different, because also under Dutch law an offer and an acceptance are necessary for the formation of a contract.

Conclusion

Do you wish to scrape or otherwise collect data from the Internet for your own use? Please note that different jurisdictions may impose restrictions or even prohibitions on this. By seeking expert advice before you start, you can avoid the risk of claims for damages, and enforcement by the privacy regulator.

Advice

Does your company or organisation have a privacy statement and/or other privacy-related documentation? Have privacy-related documentation drawn up or checked.

Do you want to use our knowledge and call us with some regularity? Subscribe to the Virtual Legal Department. 

Interested? Call or email us:

T. +31(0)70-3819281
E. info@wisemen.nl

For the latest information, subscribe to the newsletter now

WiseMen Newsletter!