ChatGPT and Bard analyse: Dickens – A Christmas Carol

A Christmas Carol

S-branch traditionally shares a festive message with its followers during the holiday season. Last year, we treated you to a bloopers video from our self-paced online training course, “i2 Analyst’s Notebook – Creating Charts.” This year, we aim to infuse a bit more holiday spirit while highlighting the impactful work undertaken at S-branch.

This year I set myself the task of analysing: Charles Dickens – A Christmas Carol, using nothing more than a browser, a text editor and freely available online tools.

Back in 2018, I wrote a blog post about analysing and understanding unstructured text. It focussed on using a freely available Natural Language Processing tool to process and analyse some online documents. I then used PowerBI and Neo4J to explore this data further. Things have moved on since then, including the biggest and most hyped technology of the last 12 months: AI Chatbots. ChatGPT was released in November 2022, if you’re anything like me, you will have been using it ever since. This year I set myself the task of analysing: Charles Dickens – A Christmas Carol, using nothing more than a browser, a text editor and freely available online tools.

Desired Result

The question I want to ask is: “Who are the characters and how are they linked together?” I wanted to answer the question using Network Analysis Visualisation. Remember, I can only use freely available online tools. I chose Gephi which is a visualisation and exploration software for all kinds of graphs and networks, it’s also open-source and free. Gephi also provides a lightweight browser version called Gephi Lite, which I chose to use.

Gephi Lite comes with a few example graph files. One example is a character analysis of Les Miserables. I took this as inspiration. I wanted to achieve something similar without a) reading the Dickens book myself and b) writing a line of code.

Bard

To gather the necessary data, I turned to Google’s experimental tool, Bard. Despite its experimental status, Bard’s ability to reat and analyse documents proved invaluable. I pointed Bard at this text and started asking questions.

Bard reading the document

Bard was able to tell me which characters appeared in which chapters, I then asked it to put this information into a CSV format. Admittedly Bard initially summarised this information but after a quick clarification, I was able to get the data I wanted. The other great thing about Bard, is it helpfully puts the data into a Google Sheet

Generated CSV
Generating a CSV
Simplified CSV
Simplifying the CSV
Generated Google Sheet

Lastly, I asked Bard to add a column, counting how many times a character appears in a chapter. As before, it generated a Google Sheet for me too.

Adding a count to the CSV.

At this stage, I had the data I needed. I now needed to convert this data into a Gephi format (gexf), that could be opened in Gephi Lite. The gexf format is essentially an XML format, so I’d need to get Bard to generate code. I tried at length to get Bard to generate valid gexf, with no luck. Again, Bard is in an experimental stage and things are improving all the time.

Chat GPT

I switched over to ChatGPT and copied the data from the CSV/Google Sheet. I then asked it to generate the data in a gexf format. It’s worth considering your wording when asking an AI Chatbot to generate code. The resulting code was copied into a text editor (I used NotePad++) and saved as a gexf file. ChatGPT managed to generate a valid gexf file straight away.

ChatGPT generating a gexf file.

The resulting file in Gephi Lite looked very much like the image below. The only addition I made was adding colour to the nodes, Green for Characters and Grey for Chapters. I asked ChatGPT to do this, but on the first attempt it failed (although it did generate a valid gexf). I had to look at the Les Miserables example and “trained” ChatGPT on how to generate colour. It learnt quickly and was able to generate a valid gexf file with colour.

An AI generated gexf file

Gephi allowed me to run a Force Directed layout on the data. It also has a useful feature where highlighting a chapter highlights the related characters.

Focusing on the characters

I wanted to take the analysis a step further and focus on the characters. For this, I returned to Bard. Bard was able to bring back the original CSV, I then asked it to bring the data back in a different way.

Characters mentioned togeather.

I then took this data, copied it into ChatGPT and asked it to generate another Gephi file.

ChatGPT generating a character relationship Gephi chart

The resulting Gephi chart looked very much like the one below. It’s worth noting that I did ask ChatGPT to size each node depending on how many connections it had. While it generated the data well, I had to “train” it to use the correct format. It’s worth mentioning that it managed to generate a valid gexf file every time I asked it!

The final generated Gephi file
“Training” ChatGPT

Summary

Reflecting on this journey, it’s evident how technology has advanced in five years. When I wrote the original post on NLP I had to write code and install Java to get similar results. The example above is silly and perhaps a little oversimplified, but I was surprised how quickly I could get results.

While this exercise explored the potential of using Bard for document analysis, it also revealed concerns about consistency. Repeatedly asking the same question yielded varying responses, raising doubts about its suitability for real-world data analysis. Additionally, sharing sensitive data with any AI chatbot is inherently risky. Therefore, it might be more prudent to utilize ChatGPT’s script generation capabilities on a prepared data format (rather than giving it data) instead. This way, visualization wouldn’t require repeatedly exposing actual data to Bard.

S-branch is a consultancy company focusing on data analytics software and technology, specifically for looking for Fraud and Crime. We offer consultancy on products in this space and have helped a number of our clients get the most out of their data. If you need help with your data or would like to get out of your existing software investment, please contact us.

Realtime Environmental Monitoring with Sentinel Online

Since our establishment in 2018, S-branch has consistently maintained a strong affinity for conservation. A substantial number of our clients operate within this domain, spanning environmental and species preservation efforts.

We also, by the nature of our work with these clients, spend a lot of time looking at their data. This has been ‘after the fact’ data or data gleaned from using open-source intelligence. We have not been able to access real-time data in this domain, that is until now.

Sentinel-Online

Sentinel-online is a solution that allows real-time analysis of environmental pollution. The hardware kit we saw was focused on environmental, gas and particulate sensing. However, the team at Sentinel-Online tell me that almost any sensor type imaginable could be integrated. These devices can be bolted onto anything from a solid building to a moving train. They could even be placed to sense pollution in water.

The data from these devices is then delivered to a modern browser-based interface, meaning you can access the data from any device. Sentinel-Online informed us that It’s very much designed with ‘one-finger’ use in mind, meaning it’s tailored to tablets or phones. It does, however, work fine on a laptop which is predominately where we tested it.

Who would use this?

in an everchanging world of ULEZ zones and electric car mandates, where we’re being asked to justify the impact of our actions on the planet. The only way this can be done fairly is with access to accurate data in real-time.

Technology like this is timely. Especially in an everchanging world of ULEZ zones and electric car mandates, where we’re being asked to justify the impact of our actions on the planet. The only way this can be done fairly is with access to accurate data in real-time. Maybe as a fleet manager, you are being asked to reduce the carbon emissions on your fleet. Maybe as council-manager, you are tasked with reducing emissions across your city. Maybe at a waste processing plant, you are mandated to survey your sites for emissions. This technology can help.

Real-Time Sensing

Traditional point sampling methods, which are usually used in this space are obsolete. It would be easy to ‘prove’ that activities at a specific location are not detrimental to the environment by taking samples at certain times of the day. Sentinel-Online supports real-time, predictive and post-event analysis and investigation. Giving an accurate and fair analysis of the data.

The Analysis

The team at Sentinel-Online gave us access to one of their test sites. Here multiple remote solar-powered devices were deployed with calibrated sensors for CO2, H2S, NO2 and Volatile Organic Compounds (VOCs) sensors. These were dotted around a particular location, to give a clear picture of that environment.

The impact of analysing the data from the sensors over time is very powerful. It’s very easy to read the values of each sensor on each device in real-time. It’s also possible to look at a period of time. This, the team at Sentinel-Online tells me, is particularly useful for complaints of smells from a particular site. If the complaint comes in at 14:00 it is easy to review the data from all the devices and narrow down where emissions may have come from.

We also noted how impressively smooth the visualisation was with the real-time data, which is not always an easy thing to do.

Want to know more?

You can find more about Sentinal Online here. For more information on crime and fraud prevention analysis tools in general, please contact S-branch here

Clue Analytics Connector – For Data Analysis

Clue Analytics Connector

Clue has become the go-to Case Management tool for investigation professionals. We always liked that it’s dedicated to managing intelligence and investigations. It has a thought-out workflow that isn’t the ‘one size fits all’ of a standard Case Management system.

S-branch has always focussed on the data analysis piece of the puzzle. So we were delighted to see the addition of the Clue Analytics Connector.


What is it?

A ready-made connector that allows quick exports from Clue to Power Query for advanced reporting and data analysis. Power Query is built in both Excel and Power BI, both of which are available to any business with a Microsoft 365 account.

The connector allows easy access to pull data via the API from Clue into Power Query, without the need for expertise in APIs, JSON, or programming.


What can it do for me?

A lot! If we’re talking Excel, it can be used to pull out a Registers data, or a Register’s linked data for external analysis. It can be great for performing additional analysis, calculations or even duplicate detection.

When used with PowerBI, the data within Clue can be used to produce powerful dashboards. Using Clue data in this way allows Analysts to visualise and share insights from the Clue with other interested parties. It also allows drill-down analysis of a Register and its linked Registers.

What’s more, Clue provides ready-made templates that utilise Power Query to export data for use in Power BI and Excel.

The connector allows easy access to pull data via the API from Clue into Power Query, without the need for expertise in APIs, JSON, or programming.


Tell me more about the Excel Templates!

Well, there is a standard empty template that comes with all the Power Query functions you need to pull data out of a desired Register. There is also a Duplicate Checker that will work with any Clue implementation. This will pull data out from POLE-style registers (People, Objects, Locations and Events) and look for duplicates between the data using fuzzy matching.

This will pulls out date from POLE style registers and looks for duplicates.

Finally, there is the DVLA checker, which goes through the Vehicle Register picking out Vehicle Registration Numbers and returning the DVLA information about that vehicle. We see this as an excellent example of how the Clue Analytics Connector can be used to collaborate Clue data with other sources (Companies House comes to mind).

New templates are also being added all the time, which is great to see.


Okay, what about the Power BI Templates?

For S-branch, this is where it gets exciting. We’ve always been fans of Power BI. Again, there is a standard empty template, which allows data from multiple Registers to be extracted and then linked together using the Power BI model view. As for pre-configured templates, there is a great user management template, which allows administrators to easily see when a user last logged in, what teams they belong to and even the last record they added/updated. This isn’t a full audit log but it’s great for giving management an idea of how users are using the system.

Allows administrators to easily see when a user last logged in, what teams they belong to and even the last record they added/updated

There are also some great Register templates. The Incident template not only gives great insight into the Incident itself, but it also reports on how many days pass until a Task, Event or Investigation is created from the Incident. Great for management reporting.


In Conclusion

The Clue Analytics Connector is a great addition to Clue and a real asset for teams who want to get the most out of their Clue Implementation. There is a growing community using the Connector and a great knowledge base that is constantly evolving. The flexible nature of using Power Query means that feedback can be considered quickly. It also gives customers a great opportunity to be part of the design piece, as feedback on these templates can be channeled to the product eventually.

If you think this would be a useful addition to your Clue implementation or you are looking at purchasing Clue, please contact the Clue team who will be able to help you. Likewise, if you are looking into Data Analysis tools in general, or would like Consultancy on either Clue or PowerBI, please contact S-branch here.

Using i2 with a Mac – what are your options?

Although not prolific in Law Enforcement or Government Agencies, we are seeing an increase in the use of Macs in fraud analysis space. The sheer firepower of the M1/M2 chipset is hard to ignore and more Analysts are being given the choice of Mac or PC. We are not going to open the can of worms of discussing which is best, this blog will address the issue of how to get i2 to run on a Mac. This blog will concentrate on Mac’s with Apple Silicon (M1/M2)

i2 and Macs

Natively i2 is not compatible with a Mac operating system, in fact, it’s only compatible with a Windows desktop machine. I2’s visualisation product, Analyst’s Notebook, is considered the standard for charting data. For this reason, Mac users must find a way of getting it to run on their Mac. I2’s widely used analysis datastore, iBase, is also Windows-based and is designed to be run on a Windows server with SQL Server installed.

UTM

UTM is virtualisation tool that allows virtual machines (VMs) to be run on the Mac operating system. It’s open source (free), lightweight and quick to set up. You will need to purchase an appropriate Windows license.

it is a separate environment with little interaction with the Mac OS. Setting up a cloud drive (OneDrive or Google Drive for example) helped but it always felt removed. It’s also a pain remembering where supporting tools (like Excel) are installed.

https://www.uubyte.com/blog/how-to-install-windows-11-on-mac-with-utm/

Parallels

Parallels is like UTM but with more interaction between Windows and the Operating System. It’s possible to add Windows applications like i2 onto the Mac dock, so it opens as if it is installed on the Mac. We also found that dealing with files is far more intuitive.

There is a cost to Parallels, on top of the Windows operating system cost. However, if you use applications like i2 or PowerBI daily then the additional cost may be worth it.

S-branch Analyst’s Workspaces

We are fortunate enough to be able to install tools like the above on our machines and trial them. Some IT departments will not want the additional cost or time of supporting additional tools like the above.

If this sounds like your organisation there is another option. S-branch has partnered with Bytes and Amazon Web Services to offer pre-configured cloud environments for users of i2. These hosted environments come loaded with the latest versions of the i2 software suite and the necessary tools for an analyst to do their magic.

Each environment is dedicated and licensed to one client, meaning your data is secure, encrypted and only accessed by you and your team. Being cloud-based, these environments can be available all the time, be leased on a monthly basis or even ‘rested’ when not required.

The main benefit is that i2 can be accessed through your Mac, on a Windows machine, on a tablet, even through a browser.

S-branch

We’ve been part of the i2 journey since 2006, either as trainers, technical consultants or marketers. Through these roles, we’ve worked with several police forces both in the UK and abroad, charities, insurance firms, law firms, government agencies, and commercial clients.

If you’re an i2 user and require technical expertise, please feel free to contact us

Learn how to create i2 Association and Timeline charts

Do you prefer learning at your own pace?

Shortest Path Training Ltd and S-branch are proud to announce a whole new way for you to gain your i2 skills.

Online Self-paced Learning

With our first course focused on creating charts in i2 Analyst’s Notebook. These online self-paced courses are perfect for motivated individuals who are either unable to attend Shortest Path Training’s tutor-led classroom or online training, or who learn better on their own and at their own pace.
Each course is structured using tried and tested training methodologies, and delivered to Shortest Path Training’s usual high standards, utilising many years of proven experience training organisations around the globe at both a local and national level.

How does it work?

The learning environment is accessed through a web browser, and on starting the course you will be guided by defined learning objectives, video explanations, and examples; and will then have the opportunity to practice each lesson by downloading course materials and using your own copy of i2 Analyst’s Notebook to complete exercises. To reinforce and check your learning we have also included a series of multiple-choice questions for you to complete as you progress.

What skills will you gain?

“I want to be able to create Association and Timeline charts for visualisation and to learn how to structure charts correctly to enable effective analysis.”

  • Learn how to build and structure Association and Timeline charts to enable understanding of the key concepts of Entities and Links
  • Learn how to create charts that enable accurate analysis and effective interpretation of networks and sequences
  • Learn how to annotate briefing and presentation charts, and how to convert your charts into other file types for dissemination

Objectives

By the end of the course, delegates will be able to:

  • Add data to create both an Association and a Sequence of Events chart: using Types, Identities, Labels, Pictures, Attributes, Links, Date & Time, Direction and Line Strengths
  • Annotate charts using Labels and a Legend, and save them in both PDF and Picture format

Is this course right for you?

This course is aimed at:

  • New, self-taught, or informally taught users; and those who want to refresh their knowledge Those for whom tutor-led training is not suitable due to time or other constraints
  • Those who are self-motivated and prefer to learn on their own

Testimonials

The ability to repeat the training videos is very practical, as the trainee can refer to the course materials at any time.


The trainer presented the materials in a professional way; bringing appropriate examples on the subjects, like how to use tools, or different ways to reach some steps.


The training sessions were distributed in a very orderly way, as information is provided gradually, which helps in understanding the subject easier and faster.

Excellent! How do I book on?

For further information and to book yourself onto the course, please email Shortest Path Training at Info@shortestpathtraining.co.uk. Further information on the course can be found here: Download Course Information