Clue Analytics Connector – For Data Analysis

Clue Analytics Connector

Clue has become the go-to Case Management tool for investigation professionals. We always liked that it’s dedicated to managing intelligence and investigations. It has a thought-out workflow that isn’t the ‘one size fits all’ of a standard Case Management system.

S-branch has always focussed on the data analysis piece of the puzzle. So we were delighted to see the addition of the Clue Analytics Connector.


What is it?

A ready-made connector that allows quick exports from Clue to Power Query for advanced reporting and data analysis. Power Query is built in both Excel and Power BI, both of which are available to any business with a Microsoft 365 account.

The connector allows easy access to pull data via the API from Clue into Power Query, without the need for expertise in APIs, JSON, or programming.


What can it do for me?

A lot! If we’re talking Excel, it can be used to pull out a Registers data, or a Register’s linked data for external analysis. It can be great for performing additional analysis, calculations or even duplicate detection.

When used with PowerBI, the data within Clue can be used to produce powerful dashboards. Using Clue data in this way allows Analysts to visualise and share insights from the Clue with other interested parties. It also allows drill-down analysis of a Register and its linked Registers.

What’s more, Clue provides ready-made templates that utilise Power Query to export data for use in Power BI and Excel.

The connector allows easy access to pull data via the API from Clue into Power Query, without the need for expertise in APIs, JSON, or programming.


Tell me more about the Excel Templates!

Well, there is a standard empty template that comes with all the Power Query functions you need to pull data out of a desired Register. There is also a Duplicate Checker that will work with any Clue implementation. This will pull data out from POLE-style registers (People, Objects, Locations and Events) and look for duplicates between the data using fuzzy matching.

This will pulls out date from POLE style registers and looks for duplicates.

Finally, there is the DVLA checker, which goes through the Vehicle Register picking out Vehicle Registration Numbers and returning the DVLA information about that vehicle. We see this as an excellent example of how the Clue Analytics Connector can be used to collaborate Clue data with other sources (Companies House comes to mind).

New templates are also being added all the time, which is great to see.


Okay, what about the Power BI Templates?

For S-branch, this is where it gets exciting. We’ve always been fans of Power BI. Again, there is a standard empty template, which allows data from multiple Registers to be extracted and then linked together using the Power BI model view. As for pre-configured templates, there is a great user management template, which allows administrators to easily see when a user last logged in, what teams they belong to and even the last record they added/updated. This isn’t a full audit log but it’s great for giving management an idea of how users are using the system.

Allows administrators to easily see when a user last logged in, what teams they belong to and even the last record they added/updated

There are also some great Register templates. The Incident template not only gives great insight into the Incident itself, but it also reports on how many days pass until a Task, Event or Investigation is created from the Incident. Great for management reporting.


In Conclusion

The Clue Analytics Connector is a great addition to Clue and a real asset for teams who want to get the most out of their Clue Implementation. There is a growing community using the Connector and a great knowledge base that is constantly evolving. The flexible nature of using Power Query means that feedback can be considered quickly. It also gives customers a great opportunity to be part of the design piece, as feedback on these templates can be channeled to the product eventually.

If you think this would be a useful addition to your Clue implementation or you are looking at purchasing Clue, please contact the Clue team who will be able to help you. Likewise, if you are looking into Data Analysis tools in general, or would like Consultancy on either Clue or PowerBI, please contact S-branch here.

FIXED – IBM i2 iBase/Analyst’s Notebook ‘hangs’ when using the Find function

Alot of our clients use dual monitors. It’s great for comparing data and especially good when comparing rows and columns to visualisations in i2.

A few of our clients have reported that Analyst’s Notebook/iBase seemed to ‘hang’ and not respond when a user clicks on the ‘Find’ function, either through the Data Sources pane or through the right-click menu. This usually coincided with switching from dual screens to a single screen.

Analyst’s Notebook/iBase seems to assume there is another screen when there isn’t! It sometimes persists even when you go back to two screens.

There is a quick registry change to fix this. We recommend that you ask your IT to do the following tasks or back up your registry settings before attempting the below.

  • Close iBase and Analyst’s Notebook
  • Go to the start windows start menu and type “Regedit”. Say okay to any of the following screens until you are presented with the registry editor. You may need IT to give you access/to do this.
  • Use the folder tree on the left to navigate to: Computer\HKEY_CURRENT_USER\Software\i2\iBase\8\
  • Delete the folder DialogSettings
  • Reopen Analyst’s Notebook and try again.

S-branch

We’ve been part of the i2 journey since 2006, either as trainers, technical consultants or marketers. Through these roles, we’ve worked with several police forces both in the UK and abroad, charities, insurance firms, law firms, government agencies, and commercial clients.

If you’re an i2 user and require technical expertise, please feel free to contact us

VA Insight: Filling the hole left by Intellishare

hole left by Intellishare

Intellishare was a lightweight browser based application. It provided searching and basic visualisations across data held in an i2 data repository; in this case i2 iBase.

I can remember when Intellishare was first released. It was the brainchild of some very clever people in the i2 consultancy team. ‘IBase Web’ as it was called then was trialled in Hertfordshire Police here in the UK, before being made available as the product known as Intellishare.

Intellishare became incredibly popular with the i2 community, with its easy to use web-based interface and straightforward deployment model. Its find and explore module gave access to valuable data sitting within an iBase database. While its data entry module, meant that real time information could be captured by front line staff.

Software Withdrawal

In September 2019 Intellishare was withdrawn and support was discontinued for Intellishare. While this was understandable, as a lot of the supporting frameworks were now themselves out of date, this left Intellishare users with some limited options.

Introducing VA Insight Portal

The VA Insight Portal ™ is a modern, 3-tier architecture, browser-based solution allowing users to streamline data collection and to view tactical visualisations while on the go. It offers a simple, cost-effective deployment model that compliments an existing i2 environment.

S-branch have had the opportunity to thoroughly test the latest release of the VA Insight Portal and to share our user experience directly with the Insight development team. Our client base has felt the hole left by Intellishare being discontinued. We feel that VA Insight fills that hole nicely!

We deployed the VA Insight Portal™ on a range of our test SQL source databases. In this blog, we will share our user experience with the platform:

Dashboards

The team at S-branch were very excited to see dashboards in VA Insight. It’s too often that product demos of i2 start with a single entity, a target person or phone for example. The real problem is often identifying that target, which then gives a start point to begin an investigation.

The dashboards are basic, don’t expect a full business intelligence suite like PowerBI or Cognos. However, for getting a “lie of the land” it works well.

Entities and Links

Entities and links can be searched, either by a basic or advanced search. The record view shows a basic visualisation and all the record details in one easy to read screen. One feature that we loved was the export functionality. It allowed a pdf of the record to be created with a visualisation attached; a really nice touch.

As of today, there is no expand function (that you would find in iBase). It is possible to drill down into an entity by clicking on it, this will open up that entity and show its visualisation. We think this is fine for the use cases that VA Insight is targeted at. Hardcore analyst’s will be still using the back office i2 suite, while front line staff will be accessing the data through the easier to use VA Insight.

Queries

We were impressed that VA Insight came with its own visual query builder. We were even more impressed that it supports parameterised queries, such as @#NOWDATE. While this is actually an improvement to the functionality within Intellishare, it should be noted that queries can be made diretly within the source SQL  database and then made available to VA Insight.

Data Entry

VA Insight supports data entry through its entities and links, its visualisation screen and through custom forms. We like the flexibility this brings, and we found the data entry to be intuitive.

Summary

In summary we found the VA Insight Portal™ to be an excellent alternative for IBM i2 Intellishare users. While not a direct replacement, it offers intuitive workflows, increased functionality and it is complimentary to most clients existing infrastructure.

There are other options available to address the hole that was left behind by Intellishare. However, we feel that this is the most appropriate in regard to technology used, the deployment time and importantly the price.

If you’re looking for an Intellishare replacement, looking into i2 or you’re looking into data analysis tools in general; please feel free to contact us.

Analysing the Paris Attacks OSINT data – using SIREN

Paris Attacks

Paris attacks kill at least 128. That was the first news headline I read on my phone on the morning of November 14th 2015. We had our flat in London at the time, it was a sunny day and we were just heading out for brunch at a local café.

BBC News – November 14th 2015

I remember that day well, mainly because we had a friend who was visiting Paris. For the purpose of this blog let’s call this friend ‘Steve’. I quickly checked Steve’s Social Media accounts to see if he was safe; he had no posts since November 10th 2015.

The news reports mentioned the Bataclan and ‘restaurants and bars at five other sites in Paris’. I had no idea where Steve was staying in Paris or indeed his plans for the evening before.

This event occurred when I was working with a commercial Social Media Monitoring platform. This meant I was able to monitor in real-time posts on social media. I decided to monitor keywords such as Paris, Attack, Bomb, Shooting, Shot etc. We then left the flat and I left the monitor running.

While we were at brunch, I checked my personal facebook and saw the following post from Steve:

Although safe, Steve and his friends were confined to their apartment for a while.

Although relieved that Steve was safe 130 people lost their lives during the attack and 413 people were left injured.

After brunch we returned to the flat and I stopped the monitor. I’ve always kept the dataset I generated from that day, it’s a memory of my relief but also of how lucky Steve was.

SIREN – Investigative Intelligence Platform

One of the great things about S-branch, is being software agnostic. We’re not tied to a particular software vendor or piece of software. This means 2 things: the client always gets the best tool for their requirement and we get to play with lots of cool software!

The Siren Investigative Intelligence Platform is something we’ve been playing with for a while now. We’ve been impressed with the demonstrations and tutorials but we really wanted to try it with some real data; like the Paris Attacks data.

We’re not tied to a particular software vendor or piece of software. This means 2 things: the client always gets the best tool for their requirement and we get to play with lots of cool software!

Accessing the data was quick. Siren can analyse data from REST Services, JDBC Data Sources or flat files such as CSV. The ability to use JDBC means that with the correct driver you can connect to pretty much any external database.

The Paris Attacks data was in CSV format. The loading process allowed us to easily perform transformations to the data. We only did some minor formatting transformations, such as splitting a field based on a comma and formatting dates.

SIREN – Loading the data

Autoselect Most Relevant and Generate Dashboard

Once loaded it’s incredibly quick to start gleaming insights from the data. There were two features that we loved: Autoselect Most Relevant and Generate Dashboard.

These two processes took less than a minute to run and automates something which can take a long time to design and get right.

Autoselect Most Relevant analyses the fields from the data source and selects which ones contain the most relevant data for analysis. Generate Dashboard then takes these fields and generates a dashboard from them. These two processes took less than a minute to run and automates something which can take a long time to design and get right.

SIREN – Generate Dashboard

The finished dashboard gives a good idea of what was being mentioned along with where, when and who. This shows just how versatile and quick Siren can be with any sort of data.

Graph Explorer

Dashboards give a great overview of data. It allows analysts to quickly understand and drill down into large sets of data. Once areas of interest have been identified, analyst’s usually want to ‘look into the weeds’ of the data. One way to do this is to look at the rows of data, this can be time consuming and tedious. Another way is to use Graph Visualisations.

Siren has a relations auto-discovery wizard which is in a Beta state at the moment. As data modelling is something S-branch does on a daily basis we chose to do this manually.

Graph Visualisations require data modelling. This is the process where you model entities or nodes and relations or edges. Siren has a relations auto-discovery wizard which is in a Beta state at the moment. As data modelling is something S-branch does on a daily basis we chose to do this manually.

SIREN – Graph Explorer

Once the data had been modelled it was then possible to visualise the results of the social media dashboard on the graph explorer. It is also possible to overlay this information alongside other data from other dashboards. In the example above the Paris Attacks data was narrowed down to posts geotagged within the Paris area. We can clearly see users retweeting the same message.

Summary

This exercise demonstrated to us how versatile Siren can be and how quickly you can glean insights from a set of data. Siren has recently announced the addition of NLP (Natural Language Processing) and Anomaly Detection, meaning we could glean even more information from the data. We look forward to trying this out.

For more information on Siren visit their website or contact S-branch. If you’d like to find out more about visual analysis software, then feel free to call.

S-branch offers independent advice, meaning if Siren is not for you at this time, we can help you find the solution that is.

Why Rosoka is the ‘no-brainer’ plug-in for IBM i2 Analyst’s Notebook

Why Rosoka is the ‘no brainer’ plug-in for IBM i2 Analyst’s Notebook

It’s no secret that i2® is a popular choice for visualising and analysing data. That’s why in 2011 IBM paid a rumoured $500 million to acquire the small British software company. Its flagship product: Analyst’s Notebook has now become the standard for charting data and is used in nearly all of the UK police forces. If you have a sporadic flow of data in formats that vary greatly, IBM i2 is still one of the most compelling tools on the market.

IBM i2® products have always worked well with structured data. This has given its users the ability to ‘bring to life’ data hidden in columns and rows. There are many other places that data can hide though and one of those places is in unstructured text.

Unstructured Data

Unstructured text can pose a challenge to analyse and understand, and we keep generating more of it every day. This text can be anywhere from documents, emails or social media posts. In March 2018 we wrote a blog detailing how data hidden in unstructured text can be utilised. It focussed on the use of Natural Language Processing and particularly Entity Extraction. You can read it here. That blog post concentrated on what could be done using freely available tools.

At the time we stated that there are many commercial offerings that offer that functionality but in a much slicker and easy to access interface. One of those tools is Rosoka Text Analytics. Rosoka is an industry leader of text analytics solutions. Their enterprise solution Rosoka Server is great for large scale analysis of unstructured data. This blog is about their standalone solution which is a plug-in for i2.

Rosoka

Rosoka Text Analytics

So, why is it a ‘no-brainer’?

Well, to start off with its fully integrated with IBM i2 Analyst’s Notebook. It’s a plug-in that has been created in close collaboration with IBM and you can tell. We tested the plug-in against several datasets, including the case study we used in the previous blog post. This data can be found here.

Here are the reasons why we think it’s a ‘no-brainer’ plug-in:

  • It extracts entities and links from unstructured files such as PDF’s, Word documents and txt files. It then allows users to generate Analyst’s Notebook charts based on the content
  • Processing time is quick compared to reading the documents manually and quicker than the free extraction engines used in our previous blog
  • During testing we found the entity extraction to be very accurate. What was more impressive was the relationship extraction. In our previous blog we inferred links based on how many times two entities are mentioned in the same document. What Rosoka do is far more sophisticated and is based on the language around the two entities
  • Entity extraction is good but it can create a lot of noise. Rosoka’ s use of Salience (relevance) means we could always find the most relevant extracted entities and ignore the others
  • Rosoka applies entity resolution to its extraction. This meant that if Theresa May is mentioned as ‘Theresa’, ‘Mrs May’ or even ‘She’ in a document, Rosoka will be able to resolve these different texts as one entity.
  • It’s easy to reclassify an extracted entity from say a person to a company (for example Robert Dyas the UK hardware store). Rosoka can also learn from your decisions so it doesn’t make the same mistake again
  • Rosoka is truly multilingual. We tested this by running several news articles about Islamic State through Rosoka in different languages. The organisation Islamic State was mentioned in both the Russian and the Arabic news articles. On the chart they were resolved into one entity, despite them originating from two different languages. That meant that when we expanded Islamic state we got linked items from both the Russian and the Arabic articles.
  • The most surprising reason Rosoka is a no-brainer though is the price. For a fraction of the cost of IBM i2 Analyst’s Notebook the plug-in delivers accurate, multilingual, unstructured data analysis. We’ve looked at costs for this type of technology for clients before and often the requirements were dropped, due to budget restrictions; well not with Rosoka!

In conclusion, we believe Rosoka is a ‘no-brainer’ plug-in for i2 Analyst’s Notebook. The functionality, ease of use and price will just make any analysts life easier!

For more information on Rosoka visit their website or contact S-branch. If you’d like to find out more about unstructured data analysis, then feel free to call.

S-branch offers independent advice, meaning if Rosoka is not for you at this time, we can help you find the solution that is.