Managing Knowledge: best practices and methodology for OSINT

There are no defined standards or metrics that can be applied to OSINT that are definite. There are a number of models and working situations in use but each Government or Company will have a unique requirement that will focus its specific needs. However, from the work that Hawk has done with the EUROSINT Forum we can begin to understand the problems and requirements and propose methods that will be in the best interests of a particular organisation. This work will be highlighted in the following section.

A EUROSINT Forum Working Group on Best Practices in Open Source Intelligence was established back in September 2006. Its purpose is as follows:

1. To explore best practices in the collection, processing, storage, manipulation, dissemination and use of open source information/intelligence (OSINF/T).

2. In collaboration with the European security community, to develop a best practice training syllabus that meets the needs and requirements of those agencies and individuals working in, or with, OSINF/T.

3. To identify and encourage new thinking on the use and application of open source intelligence.

One meeting in January 2009 goes some way to addressing best practices and their development with regards to analysts.

What are the Challenges Facing Intelligence Analysts?

• Access to the right information – no one can have “perfect intelligence”. However, some agencies do not have access to a) important proprietary sources and/or b) are unaware of valuable free sources. Source awareness must be improved across the board

• Problems of interpretation – analysts are not always clear how to interpret or understand new security challenges

• The job of the analysts is evolving. Increasingly, their task is not just to inform but also to explain

• Quality and quantity of staff

• Time pressures – operational demands are rising

• Skills shortages – e.g. in extrapolating economic and strategic trends, dealing with disinformation, etc.

• A desire to please customers rather than communicate uncomfortable truths

• Analysts working with open sources have to give up the “Google crutch”

• Poor knowledge of the theoretical aspects of intelligence

• Many agencies are subject to political changes. As a result, they are unable to develop consistent working processes, nor can they communicate a consistent message regarding the value of their work

• Limited access to valuable tools and technologies

• Analysts must be better trained and better skilled at finding solutions to the problems they encounter on an individual, not just an organisational, basis.

The Process of Effective Intelligence Analysis

Examining the analytical process is a means by which areas for improvement can be identified. A critique follows of the six step approach to effective intelligence analysis (based on global best practices):

1. Defining the problem

2. Gathering, evaluating and organizing data

3. Dealing with cognitive biases and potential mental traps

4. Generating and testing hypotheses

5. Constructing a sound argument

6. Giving an effective presentation

• Step 1 is often skipped by analysts looking to save time. This is unfortunate as it creates inefficiencies elsewhere in the intelligence chain. It is critical that the analyst understand the task they have been given and the scope of the problem the decision maker has to deal with. Indeed, before moving to step two, the analyst would do well to “agree the task” with their superior.

• Step 2 should allow for data validation and feedback loops. It was noted that some agencies do not employ consistent schemas with regard to the source of information, its reliability, analytic confidence, etc. Others have no such schemas at all. Step 2 should also acknowledge the different approaches needed for strategic and operational intelligence.

• Having gathered their data, analysts should address issues of denial and deception. This should follow immediately after Step 2. If necessary, data collected should be verified against other sources. Human intelligence is an effective means of countering denial and deception.

• The importance of Step 3 should not be overlooked. Most analytical errors begin here. Greater effort is needed to train intelligence staff on the effects of cognitive biases on their work.

• The process does not account for standard project management issues such as resource planning, mapping dependencies, risk management, etc. These must also be taken into account for larger intelligence projects.

Effective working processes are both a cause and consequence of effective management. The role of management is therefore very important. Managers can help the analytical process by:

• Developing a healthy working environment, one that is open to new ideas and new approaches

• Being open-minded, subjective, honest, transparent, diplomatic and focused

• Trusting their subordinates

• Managing analytical or organisational problems effectively

• Removing bad analysts from their positions

• Avoiding micromanagement

• Allowing sufficient time to develop and apply new operational disciplines; by comparison, bad management is abandoning effective disciplines and assigning tasks without explaining their objective

• Knowing their end-user. Further, the manager should have the capacity to change the end-user's point of view when necessary through a convincing chain of reasoning

• Allowing end-user to integrate their data into the analytical process

• Understanding that facts change and so must analytical conclusions

• Wearing their knowledge lightly; managers must be just as willing to learn as the staff they

employ

Advanced Analytic Concepts and Techniques

New threats require new ways of performing intelligence tasks. They also require a new generation of intelligence analysts.

This generation should better understand the technological culture they will be operating in. They must also be better trained in different analytical and collection methodologies. Every analyst should have a complete toolbox of resources at their disposal.

Work practices and analytical competencies must evolve to meet the challenges of the 21st century. More work should be done to benchmark best practices within and between different analytical agencies and to compare and contrast the experiences of analytical teams.

Further, greater emphasis should also be placed on improving team working skills and understanding human psychology (both of the analyst and the people they are studying – e.g. criminals, terrorists, etc.).

Organisational Recommendations

With regard to the effective functioning of analytical units / organisations:

• At present, organisational structures are slow to respond to the rapid evolution of tools and technologies; procurement cycles are too long. In some instances, software tools are obsolete by the time they have been installed.

• The same structures limit sincere efforts to address doctrinal challenges as quickly as they would like. Organisational doctrine should empower rather than constrain the work of the analyst.

• Analytical teams should demonstrate greater transparency with regard to their analytical results; in other words, they should be able to demonstrate how they came to their conclusions.

• Organisations are limited by their own biases, not least of which is the argument that “things have always been done like that”, so there's no need for change.

• Very few organisations are willing to invest in information and knowledge management issues.

• Analytical units have to contend with the fact that there is no methodological watchdog on the Governmental level for intelligence work. There is no one to help them do their job in a consistent and scientifically sound way.

• In some instances, decision makers are wary of strong, well functioning analytical units. Why? Because the better the analysis the greater the expectation on the part of the decision maker to act.

• In some organisations, passion and competence can limit an analyst's career. Organisations do not know how to deal with, or get the best from, their “mavericks”.

• Analytical units are obliged to demonstrate their value added. But there is little agreement on how this is to be measured. Naturally, good intelligence is one way of measuring success. But other benchmarks are also needed to measure efficiency and effectiveness.

• Analytical units should improve their information sharing abilities. This is critical to their effectiveness. Even if an agency is barred from sharing open source information with external actors, it can still find ways of improving information sharing among internal staff.

• There are more and more people doing intelligence-related work (i.e. information gathering and analysis). More resources are needed across the board, and these should be distributed fairly.

• Analytical organisations should consider tapping the knowledge, skills and expertise of academia as necessary.

Whilst action is needed on the above points, they form the basis, if addressed with specific reference to a Government or Business, of best practices in the OSINT analytical process.