How GDPR affects OSINT

The introduction of GDPR was a shock to many. While there are limitations, it doesn’t prohibit OSINT work completely. Find out what you can or cannot do when conducting investigations.

activity-board-game-connection-613508.jpg

In almost all OSINT activities, we process (e.g. collect, store, analyse, reproduce) personal data including names, addresses, user names, phone numbers, IP addresses and much more. However, the new data protection legislation introduced in the European Union in May 2018, the General Data Protection Regulation (GDPR), restricts the processing of personal data. Therefore, OSINT researchers need to have good understanding of how the GDPR applies to their situations, if only to stay on the legal side with their work.

In this blog post, we will discuss GDPR basics for OSINT researchers. We will not look at the exceptions. Processing personal data for household use or for journalistic purposes are for the most part exempted under the GDPR. Of course, the devil is in the details with respect to when these exemptions actually apply and this blog post will not go into those cases. Also, we will not look at OSINT for law enforcement use, as that has a different legal framework for dealing with personal data.

Hence, we will aim at OSINT in a commercial setting, where the researcher is dealing with matters such as background investigations, third party assessments and pre-employment screenings. Furthermore, this article will only discuss those GDPR aspects which are most relevant for OSINT work as we see it now. The GDPR is an extensive regulation with numerous aspects, which we cannot fully discuss in a single blog post.

Also please keep in mind: We are not lawyers and the implementation of the GDPR can differ between EU countries in numerous details. If you are reading this to seek legal advice, you should consult your friendly lawyer instead.

Summarising the GDPR, the aspects relevant for OSINT work are:

  1. You need a legal basis for processing personal data;
  2. You need to apply certain principles in the processing of personal data;
  3. The data subject of whom you process personal data has specific rights you need to understand, anticipate and honour.
  4. Understand if you are the data controller or the data processor.

Legal basis

Data protection regulation was never meant to render the processing of personal data impossible. Instead it is meant to balance the need for data exchange in our society on the one hand, and the fundamental right of privacy for citizens on the other. It is  therefore important to note that privacy is not an absolute right. The GDPR balances this right with other rights by restricting the processing of personal data to instances where there is a legal ground (Article 6). It lists six different grounds, of which three are potentially relevant for OSINT work: consent (Article 6a), legal obligation (Article 6c) and legitimate interest (Article 6f). We will discuss these three in detail.

Article 7 of the GDPR sums up the conditions for consent. Consent is tricky because the GDPR states that the data subject should have free choice when giving consent. However, for example, free choice in an employer/employee relationship does not really exist. Consent can – according to the GDPR – also be withdrawn at any time. So what happens when a data subject withdraws his or her consent halfway into an investigation? Or when it turns out afterwards, that the consent cannot be regarded as given freely?

Due to this ambiguous legal nature of consent under the GDPR, we believe that for OSINT investigations the use of consent should be avoided whenever possible. The risk that the data subject could afterwards argue that he or she had no choice than to give consent because of the consequences at risk, is simply too large. Moreover, often it is practically not possible to obtain consent especially if you, for example, are examining social media of the circles around your subject.

The second potential ground for processing personal data, that may be relevant for OSINT work, is legal obligation. This could be the case when your client has the obligation to identify their customers and the source of their funds under Anti Money Laundering (AML) regulations. Especially if you are instructed by a financial institution, this may likely be the overall legal basis for your OSINT work.

The third ground for processing personal data that may be relevant for OSINT work is a legitimate interest of your client.

What is legitimate interest? This is a tricky question, as it does not directly relate to any other law or regulation. Imagine the following scenario: A large stock-listed fashion company is in the process of hiring a new CEO. A final candidate is presented and the company puts him through a pre-employment screening. The company is known for their strong stance against animal cruelty and has supported many awareness campaigns in this regard. It has become their corporate identity. During the pre-employment screening pictures are found on social media, showing the CEO-candidate participating in annual fox hunts. If this information would leak when the CEO is already in position, it would surely cause a major scandal, possibly decreasing the stock value of the company and causing job layoffs.

The scenario described above could be a legitimate interest, as the financial situation of the company and thus the prosperity of many employees could be affected be the actions of one person. Nonetheless, pre-employment screenings are frowned upon in certain EU-countries.

Another example can be a simple fraud investigation, where you are tasked with identifying possible assets belonging to a person suspected of fraudulent actions. Your client claims to be defrauded and would like to initiate legal action. As such, the client has a legitimate interest to instruct you identifying possible assets of the perpetrator.

In sum, as an OSINT investigator, you should always understand and document the legal basis to process personal data before conducting your research. In a commercial setting, that will most often be either be a legal obligation or a legitimate interest of your client. We advise to always properly document the legal basis, for example, by explicitly detailing the situation in an engagement letter or contract for the work. More in general, and from an ethical point of view, we believe that you should always want to understand what the purpose of your work is and the interest of the client.

Principles

Regardless the exact legal basis, Article 5 of the GDPR imposes a number of principles for the actual processing of personal data. Each of these are relevant for OSINT work and therefore we will discuss them all.

  1. Lawfulness, fairness and transparency

Lawfulness is self-explanatory: You need a legal basis to process the personal data. This should be one of the six legal bases as provided in Article 6. Furthermore, you should not hack, steal or lie to get the data and to prove this, you need to document the sources – which a professional OSINT researcher would do anyway. Not only are these actions unfair and unethical, they may as well be illegal in many cases. Fairness also relates to proportionality. Is the amount of personal data you collect on the data subject proportional to the task at hand? Transparency is touched upon in more detail in Article 14 of the GDPR (situations where the data is not collected from the data subject), we will discuss this further along.

  1. Purpose limitation

Again, pretty easy. What you collect for one purpose, shouldn’t be used for another incompatible purpose.

  1. Data minimization

You should minimise the amount of personal data to what you really need. As much as necessary, as little as possible. So when you scrape gigabytes of data, only the information relevant to your task should be retained.

  1. Accuracy

You have the obligation to make sure that data is of good quality, thus do not use outdated information or data of which you know it is incorrect. Especially when working with people search engines, you will stumble upon a lot of old or false info (false flags) on individuals. You are responsible for verifying the data where possible before further processing or reporting.

  1. Storage limitation

How long do you retain your records? The GDPR states that it should not be longer than needed. In an ongoing litigation that could be years, but often not more that general data retention obligations such as in civil or tax law should be followed. Again, this can differ per country.

  1. Integrity and confidentiality

You are responsible for keeping the personal data you process secure. The fact that you may have collected it from open sources is completely irrelevant, it shouldn’t be publically disclosed from your side. The adagio here is: If you cannot protect it, don’t collect it.

  1. Accountability

The basically means that when processing data you should not only adhere to the previous principles, you should also be able to demonstrate afterwards that you did. The GDPR has shifted the burden of proof to those processing personal data, so you need to document!

A most efficient solution to document on how you comply with the GDPR is to draft an investigation protocol in which you describe how you process personal data and how you apply these principles to your work. In any assignment or when you get questions from the Data Protection Authority (DPA), you can refer to the protocol.

Data subject rights

The third category of GDPR provisions relevant for OSINT, is the data subject rights. According to GDPR, a data subject has rights of:

  • Notification

Every data subject shouldbe notified which personal data is processed by whom, why and where. GDPR Article 13 and 14 state that no matter if collected from the subject itself or without knowledge of the data subject, they must be informed in order to be able to contest this data collection and processing. Again, this proves to be tricky, especially when conducting investigations against a certain subject.

In some cases, the notification might be disproportional in regards to effort that has to be made in order to inform the data subject.

  • Every data subject has the right of access, right to rectification, right to erasure, right to restriction

If informed, a data subject has the right of access to all data stored on him- or herself, the right to restrict further dissemination (if not necessary by law), and last but not least the right to have all data deleted. Of course, this must be weighed against the legal basis or legitimate interests upon which the investigations took place.

There is also an exemption on these rights possible under article 23 of the GDPR which states that Member States can limit the obligations and rights under the GDPR in certain instances. This has to be done by law and every Member State may have implemented this provision differently. The Netherlands have Article 41 of their national implementing law which fully integrates Article 23 GDPR.  Does your country have the same?

Of course, if you would use an exemption, you need to document the circumstances and considerations on why you think that this exemption is justified. Be careful with this and understand the local implementation of the (conditions for) exemptions before you apply them.

Are you the controller?

A final important point relevant for OSINT work is whether you are the data ‘controller’ (the one who determines the purposes and means of the processing of the personal data) or the data ‘processor’ (the one who processes personal data on behalf of a controller).

The easiest situation is where you are the data processor and you process data under responsibility of the data controller. In those instances, most of the legal obligations are  mainly the responsibility of the data controller, which usually is your client.

However, the determination on whether you are the processer depends on the level of freedom you have in choosing the purpose and methods of processing the data. If you determine the purpose, types of data and methods applied, you cannot argue that you are just the processor, as you in fact ‘control’ the data processing.

Having a data processing agreement with a client – or adding a section on data processing to your existing agreement – is an important prerequisite to be regarded as the processor. To be regarded as the processor, the agreement should clearly show that you are instructed for a specific purpose, looking at specific types of data and applying specific methods (and excluding anything else) and reporting in a specific way. Once more, consult your friendly lawyer for more details.

There is limited jurisprudence on GDPR issues and this means, that in many instances it is not exactly sure how the GDPR will be interpreted exactly. The principles should give some guidance, nonetheless, in high profile cases make sure you discuss these matters with your client and (their) inhouse legal counsel.

We have given a general overview of the most relevant aspects of the GDPR for OSINT work. However, we realise that we are not complete in that matter. The GDPR has a number of other relevant articles, for example on processing of special categories of personal data, but there are limits to what we can cover in one blog post.

So, get your copy of the GDPR today as well as a copy of the implementing law in your country, read it, seek advice, understand it and most important: Comply with it!

Ludo Block & Matthias Wilson (I just chipped in a little)/ 11.06.2019

Why Primary Sources Matter

Hurray! German company data is now available in OpenCorporates! Does this mean I don’t have to pay for the official company register access anymore?

This morning I confronted my boss Christian with a fact that I had found on the internet yesterday evening. Although he claimed to be the director of his company, I could not find him on OpenCorporates. For those of you who do not know what this platform does: OpenCorporates is the largest open database of companies and company data in the world. The site claims to have over 160 million companies indexed. As of yesterday, they added 5 million German companies to their database. Should I believe Christian or OpenCorporates in this matter?

When I conduct due diligence and background checks, OpenCorporates is among one of the first platforms I use. As good as it is, OpenCorporates is still a secondary source and when it comes to reliable and present-day information, I rather choose to trust primary sources.

Don’t get me wrong, secondary sources such as the aforementioned or compliance tools like LexisNexis are amazing and are really helpful to get an overview of what you are dealing with, but they all have little flaws. In some cases, the data is not as up-to-date as it should be, in other cases they are lacking essential information, such as the company shareholders. The worst-case scenario is when data is falsely aggregated during the import-process, linking the wrong entities to each other. Throughout my investigations, I have stumbled upon these issues more than once when using secondary sources.

Based on yesterday’s import of the German company data into OpenCorporates, I decided to check my own employer: Corporate Trust, Business Risk & Crisis Management GmbH. This is what OpenCorporates provided:

sources

There are some flaws in this dataset, because I am sure Christain would love to see his name in here as well. After all he founded the company and has been the director of Corporate Trust ever since. This is not just a problem within OpenCorporates, I have seen similar issues quite often in expensive commercial compliance databases as well. As you can see, the dataset is also missing information on the company’s shareholders. Even when this information is contained in compliance databases, it is sometimes outdated.

These are the reasons I always try to use primary sources, such as official government company registers, whenever possible. OpenCorporates is a great starting point to tell me where to look for more detailed information, especially since it offers the possibility to search for individuals (something that many government company registers lack), but the official company registers provides the real intelligence. This is where things can get challenging. Let us have a look at the company register in Germany, our Handelsregister. It requires a formal registration, which is only available in German. No credit card payments are possible, only direct debit. For many countries, this alone may prove to be an obstacle. On the bright side, once you have access to this database, you will gain access to the original company documents, including a list of shareholders for private limited companies.

In other countries, you can only gain access to the national company registers if you are a resident of that country and in most cases against payment. Unfortunately, nothing in life is free (except the amazing British Companies House). So when it comes to obtaining all relevant and up-to-date data, a bit more is required than just the access to (free) secondary sources.

Just to be sure about Christian, I checked our company in the official German company register. Turns out he is listed as director in the Handelsregister after all.

Matthias Wilson / 06.02.2019

百度地图 – On China’s Streets with Baidu Maps

Different countries, different customs. It doesn’t always have to be Google. Today I’ll present a possibility to look at addresses in China.

Google Street View is a must-have from OSINT investigators nowadays. Especially when conducting Geolocation Verifications, this tool is a valuable asset. The overall coverage is getting better day by day and in larger cities, such as Paris and London, the Google Street View car has passed multiple times, allowing us to see changes over the years. Even in third world and emerging countries there might be a solid Street View coverage. This effortlessly enables us to have a look at a remote village in Slovakia in order to check an address which supposedly belongs to a large company.

Unfortunately, there are still many blind spots on the Google Street View map. This isn’t Google’s fault and mostly results from regulatory reasons and/or security policies in various countries.

In Germany, the main reason is the complicated relation between Germans and data privacy. Only a few major cities have Street View coverage from 2009 and lots of locations are pixeled. Germany is a digital developing country.

PNG 1Google Street View coverage for Germany compared with neighboring countries

China also does not have a Street View coverage (except Hong Kong). This has regulatory reasons. However, China wouldn’t be China, if they didn’t have a copy of Street Maps. The Chinese search tool Baidu also incorporates a map tool that has something similar to Street View called Total View. There is no complete coverage in this tool, only in the larger cities and economic centers. Investigators conducting a due diligence of new business partner in China can use Baidu Maps to verify addresses. If the address which is supposed to house a large business only shows a small newspaper kiosk, something might not be right.

PNG 2Baidu Total View coverage (blue shaded area) in and around Shanghai

The big challenge here is the language barrier. Baidu is in Chinese and the automatic translation of this site sometimes does not work properly, so we’ll have to copy and paste sections of the page to get proper translations.

You can acces the Baidu Maps by clicking on 地图 (this translates to ‘map’) at the top right of the Baidu landing page.

PNG 3

In general, this tool is built like Google Maps. On the top left you’ll see the search field (red box). On the bottom right you can choose between the different view types: Street Map (green box), Satellite View (yellow box) and Total View (purple box).

PNG 4

It is best to search with Chinese search terms when using Baido. So, if we want to search for the address of a Chinese company, we should look up the address in Chinese on the website of the company.

Let us take Volkswagen (China) Investment Co. Ltd. (大众汽车(中国)投资有限公司), for example. This company is a subsidiary of the German automotive group. On the company’s website www.vw.com.cn we’ll find the company name and address in Chinese, of course we have to use Google Translate to get this far.

After copying the Chinese address into the Baidu Maps search, we’ll receive a result. Now we can switch to the Total View mode and place the camera icon right in front of the address.

PNG 7

Just like Google Street View, we now have to possibility to pitch and turn the camera, as well as zoomin in and out and ‘driving’ along the street. In our case, we can clearly see the Volkswagen building with its logo.

PNG 8

It isn’t always this easy, sometimes you have to look around a bit on Baidu Total View to actually find what you’re looking for.

I hope this short and simple blog post can help you when using Baidu Total View. Just play around with the tool a bit to learn more. If you have any questions or remarks, feel free to use the comment section underneath this blog post.

Ingmar Heinrich / 03.12.2018

Interdisciplinary Intelligence Preparation of Operations – (I2PO)

Whether you are

  • a HUMINT case officer in military intelligence,
  • a detective in the police force,
  • a SIGINT analyst in an intelligence service,
  • an investigator supporting or conducting due diligence businesses cases,
  • or a journalist researching for a new article,

you should have extensive knowledge of OSINT techniques.

Now why should these roles, especially the HUMINTer or SIGINTer, be proficient at OSINT? The following article will explain a concept of work that I call ‘Interdisciplinary Intelligence Preparation of Operations’, I2PO in short. The basic idea is that every element working within an intelligence cycle requires OSINT knowledge to either prepare, enable, conduct or support operations. In the future, I will also make a point on how this concept easily transfers to business cases, such as due diligence checks, and journalism as well.

First, let us define what OSINT actually is. Open Source Intelligence is acquiring information from generally  accessible sources. This includes data found on the internet as well as within traditional print media, TV- and radio broadcasts. I tend to use the term ‘generally accessible’ as opposed to ‘publicly available’ or ‘openly accessible’, as the data is accessible, however, sometimes in closed networks, behind paywalls or not traceable without extensive knowledge of OSINT. This, in my opinion, rules out the use of ‘publicly’ or ‘openly’, which implies that everyone could access the data easily.

Another important aspect is the term ‘intelligence’ within OSINT. Merely collecting data is not OSINT. Connecting the dots, looking for missing links, assessing the data and producing customer driven reporting is what makes intelligence out of it. This requires knowledge, experience and instinct; a combination which is very hard to replicate using fully automated OSINT tools. Thus, the most important element of OSINT is the analyst, no matter how many software-based tools and gadgets he or she uses.

Before considering how OSINT should be used in combination with other intelligence collection types (ICT), I want to point out some of the advantages when working with OSINT. OSINT data is usually available the moment you start working on a case and often published in near- or real-time, especially when following events on social media. Cases in which you work in a real-time environment, with changes occurring momentarily, bring us to the most important OSINT rule:

If you see it, save it!

You will never know if the data will still be there the next time you intend to look for it.

Depending on the case, you will also be dealing with mass data (or big data). This is where a certain degree of automation might be helpful, keeping in mind that the final assessment shouldn’t be performed solely by an AI. When speaking of quantity, you must consider the quality of the collected data as well. Especially in times like these, verifying information and filtering out disinformation is vital!

After years of work within government intelligence structures and working on business cases, I have therefore developed the concept of I2PO to define my work. This is also something I use as a theoretical basis in the OSINT and INTEL classes I teach. As mentioned before, the general idea is that many different jobs require OSINT skills in order to successfully achieve their goals. Therefore, I highly recommend an interdisciplinary approach. This means not only relying on one ICT, but also having an understanding on how OSINT can support HUMINT and SIGINT operations, police investigations and business cases and vice versa, just as well as OSINT provides information for decision makers as a standalone ICT.

In the following weeks, I will post examples of I2PO in different lines of work (e.g. SIGINT, HUMINT, police investigations, due diligence, journalism and more) to emphasize and further explain this concept.

To start out, I’ll describe I2PO when applied in a military intelligence environment supporting military operations.

I2PO to Support Military Operations

Military operations, such as the ongoing coalition missions in Afghanistan and Iraq, have heavily relied on intelligence collection through SIGINT and HUMINT in the past. These two ICTs demand a large amount of preparatory work and in times in which our adversaries are more cautious and OPSEC-aware, these two ICTs are hitting boundaries. HUMINT sources are having a harder time receiving information from core target networks and communications encryption is on the rise, creating new challenges for SIGINT. At the same time, the amount of information available through the extensive use of social media, even in the aforementioned crisis areas, is vastly growing on a daily basis. In Syria for example, information on troop movements or combat actions find its way across Twitter in near real-time.

In order for decision makers on the battlefield to react to situational changes in a timely manner, it is essential to have forward deployed intelligence elements able to conduct OSINT as it happens. In many cases, this work is done by special OSINT teams, many of them not even being in the actual combat zone. This will always lead to a time delay when disseminating information to the final intelligence customer and decision maker. As with tactical SIGINT or HUMINT, which are close to or in some cases organic to their intelligence customers, tactical OSINT is the answer. Sending a dedicated OSINT analyst forward to support operations is one solution. However, training existing intelligence personnel, enabling them to independently conduct OSINT on a case-by-case basis is another option. On these terms, the training would enable personnel to answer requests for information as they come in, rather than relaying these requests to another element, thus again resulting in a time delay.

This is what I understand as I2PO. Having an all-source analyst who is able to conduct OSINT research and to immediately verify the collected information when needed in time critical situations to support before, during and after military operations. In this example, two different skill sets (one being the all-source analytical expertise) being used in an interdisciplinary approach is the core factor of I2PO.

Matthias Wilson / 16.08.2018