Cognitive Technology

In its simplest form, cognitive technology is software mimicking the human brain. A human brain can read text, listen to sound and spot objects in videos. A human brain understands context, it can learn from previous experience and it can adapt as new information comes its way. In short: the human brain is brilliant. Why would we use anything else?

Well, we are in an age of data generation. On average users of the Internet generate 2.5 quintillion bytes of data each day. It’s not just reserved for the Internet either, companies are generating more internal data than ever and as storage becomes cheaper the data only grows.

The key to a company’s success may be hidden in their own data. The meaning of life may well be held on the Internet; you never know.

With this newly acquired data comes great opportunity. The key to a company’s success may be hidden in their own data. The meaning of life may well be held on the Internet; you never know. The problem of course is the sheer volume. The task of understanding the data and obtaining anything useful from it is impossible to do manually; although I’ve seen people try!

This is where cognitive technology comes in. Forms of artificial Intelligence, computer vision, machine learning, natural language processing and speech recognition; that can be used to understand the data for us. It is only then that we can then gain intelligence from the huge amount of data. All that is needed is an analytics system with cognitive technology…

All that is needed is an analytics system with cognitive technology…

Build your own

One option is to build a system from scratch.  By cherry picking the best technology companies (as well as open source resources) that provide cognitive analytics. You could feed data to and from these technologies into the system. You could pick a high-performance graph database to deal with the data and build a cutting edge browser-based UI to give statistical and visual analytics to the users.

This option is a viable but costly one. It will require a good solutions architect who understands the problem, some high-end developers, a data scientist and a project manager. It’s also a time-consuming proposition. You aren’t going to be getting results tomorrow.

Off-the-shelf

Why build something when somebody else has already developed it? Another option is to buy an ‘off the shelf’ solution. Here the deployment will be fast, you will get support when you get stuck and the UI and back end database have already been thought through.

This option is appealing and may be cheaper. It is common though that you are then tied in to any technology that the software vendor has partnered with. This may mean: if you want improved voice recognition or natural language processing in Arabic, you may not find the solution you want.

Visallo – Plug ang Play Cognitive Technology

Visallo is one of the ‘off the shelf’ solutions described above. It is based on a high-performance graph database and it allows users to perform: statistical, visual (graph), temporal and geo-spatial analysis all from one intuitive browser-based UI.

Visallo

While that is all appealing, the interesting part to Visallo is their attitude to extension and customization. They actively encourage it. Jeff Kunkle, the President of Visallo actually wrote a blog entitled ‘We love when customers replace our software’

Visallo was designed with extension and customization in mind. From the underlying data store to data processing algorithms and UI plugins, you’re not stuck with a one-size-fits-all solution.

This means if the default OpenNLP plug-in that analyses text and extracts entities (People, Organisations, Locations and Vehicles) from unstructured data doesn’t fit your needs; choose another one. If you want to integrate with you existing investment in voice recognition technology; no problem.

Plug and Play

This attitude to ‘plug and play’ technology allows the fast deployment and support of an off-the-shelf product, with the flexibility of a build-your-own solution.

For more information on Visallo visit their website or contact S-branch to find out more. If you’d like to find out more about cognitive technologies and the application into crime and fraud; feel free to call.

S-branch offers independent advice, so even if Visallo is not for you, we can help you find the solution that is.

Introduction

Large amounts of text can pose a challenge to analyse and understand. We keep generating more of it every day. This ‘unstructured text’ can be in documents, emails or social media posts. Indexing this text can help when searching for keywords, but what if you don’t know the keywords? What if you don’t know where to start?

This post describes some of the technology and techniques that are available. It focusses on free to use (but not to distribute) software. It will not focus on particular commercial software; although some products are mentioned.

The Data

Case Study 21

The data I’ve used for this is real. A case study from a public inquiry into Satyananda Yoga Ashram at Mangrove Mounting (Australia) for allegations of child sexual abuse. The allegations are made against a spiritual leader Akhandananda in the 1970’s and 1980’s with submissions from survivors, held in multiple documents.

https://www.childabuseroyalcommission.gov.au/case-studies/case-study-21-satyananda-yoga-ashram

WARNING: The content of these documents are disturbing. Reader discretion is advised. 

Natural Language Processing
To help understand the data within these documents, I’ve first used Natural Language Processing (NLP). NLP allows machines to ‘read’ unstructured text. One of the ways NLP evaluates a document is: Named Entity Recognition (NER). NER gives an understanding of what entities (for example: people, organisations or locations) are present in the text.

Natural Language Processing (NLP)

To demonstrate NLP I’ve used free software created by Stanford University. There are many other good commercial NLP products on the market offering various language/analytic capabilities.

Statistical Analysis

To give meaning to the results of the NLP I’ve summarised the documents and their contents dependant on the entity type. In this example, I’ve stuck with people, organisations and locations. Below is a statistical visualisation.

Statistical – PowerBI

For each entity type, a count of occurrences is shown as well as document coverage. ‘Akhandananda’ is the person that has appeared the most times and the document ‘Transcript – Day 108’ contains the most people.

This basic visualisation was built very quickly using the free version of PowerBI. The free version allows you to create visualisations but limits the sharing options available. There are other good tools available for this kind of analysis, most offer free trials which is great if you want to compare functionality.

Visual Analysis

To take the analysis further I really needed to load the results into a database. I chose a graph database, simply because graph databases perform well with entity data such as people, organisations and locations.

For this example, I’ve loaded my data into Neo4j Community Edition which is free to use (under the GPL v3 license). If I was building something commercial which needed to scale then Neo4J offer commercial licenses as well.

Once modelled, I could easily generate networks uncovering interconnecting entities within the data. The resulting visualisation is based on a short query. People are represented in green, organisations in blue and locations in pink.

Inquiry Network

Here we can see that Akhandananda and Shishy are key entities, mentioned alongside a lot of other entities. We can also see that Commission, Satyananda, Tim Clark, DWYER etc are linked to both Akhandananda and Shishy. These now become entities of interest.

This gives a place to start. I first concentrated on Satyananda. Satyananda is a complicated entity as it’s the name of an individual and also the Yoga organisation. In this case, NLP has pulled out Satyananda the person. I used Neo4J to drill down into these entities, showing three documents that all entities were linked to. This time the numbers on the links refer to the occurrences of that entity in that document.

Document Network

A quick look through those three documents soon shows Satyananda, Akhandananda and Shishy being mentioned together multiple times. The subject matter is disturbing, so I’ve chosen the text carefully. Below is a screenshot where Shishy is mentioning both Satyananda (as a person) and Akhandananda when being questioned.

Highlighted Text

Summary
Using a mixture of NLP, statistical and visual analysis tools allowed me to very quickly narrow down which documents and entities are of interest.

In truth, with a small dataset like this (under 40 documents) reading each document manually is another valid option. NLP is also not completely accurate and cannot fully replace the human touch. If faced with 1000 + documents then techniques like above can really help direct analysts to the key information.

This is a demonstration only and focusses on freely available tools. There are many commercial offerings that offer the functionality described above (and more) in one seamless application. If you are looking for fraud or crime in unstructured text then S-branch can help guide you to the appropriate products available.

Following an initial period of successful freelancing, I’m delighted to announce that the S-branch brand has now been set up.

I’d like to thank all of our existing customers for their support so far and will ensure that the transition is a painless one.

S-branch will continue to assist it’s clients in identifying crime and fraud in large amounts of data. While experienced with individual pieces of software, S-branch will remain software agnostic ensuring our clients get the best solution for their needs.