The Story of Paliscope Link Analytics

This is a blog post by Christian Berg, Deputy CEO and co-founder of Paliscope. Originally posted on LinkedIn.

At the beginning of YOSE’s history, I spoke to a friend within the Swedish police force who was working on a murder case. He told me about how he had to manually compare telephone lists in Excel from multiple operators with their own lists of known “telephone numbers.”

Hearing about the drudgery of this time-consuming task set off a thought, which would later become YOSE: What if we could make everything searchable?

2022

Fast forward. With help from a talented team and great partners, ambassadors, investigators, analysts, ideas—and a lot of AI—we’ve uncovered even more problems faced by investigators. Here are a few use case examples:

• We met an investigator who had a huge Excel file full of chats, and wanted to compare that to a PDF with IP numbers to find out if there were any cross matches.
• Another investigator, from the US, told us he’d received tons of data from a return of legal process—thousands of CSV files combined with files. How could he make sense of it all?
• We’re working with online investigators who have seized lists of friends from Facebook and now want to find the shared friends.
• To put YOSE to the test, we indexed the Pandora papers from the International Consortium of Investigative Journalists with millions of data points about companies and persons who have avoided taxes to understand how it is all connected.
• A friend in the private sector is now trying out YOSE to understand the flow of traffic using netflow.

What do all these use cases have in common?

• There is always a lot of data. ALWAYS.
• The data comes in various formats: Excel. CSV. PDF. DOCX. HTML. IMAGES. VIDEOS…
• They all contain intelligence which may be interconnected.

From the beginning, YOSE was able to extract intelligence, using NLP, from structured and unstructured files to help analysts and investigators connect the dots.

In 2021, we upgraded the link chart viewer  to make it hardware accelerated. This made it possible to easily have thousands of entities on the screen all at once.

However, there was one problem left to solve: the ingestion of data. Having to manually classify content in Excel or CSV files is time-consuming. Sure, it works. But it is boring and having thousands of files makes this task almost impossible. And, like I said, there is always a lot of data.

With the latest version of YOSE, we have solved that for you.

Using a combination of NLP and pattern matching, YOSE can now automatically detect what type of content you have in a column  and match the data to other entity types within YOSE.

The result? 100-1000x faster indexing of CSV and Excel files, with a lot better quality of data. No need to spend time manually settings column types. Select a folder of files and YOSE will automatically process them for you.

But once we had done this—and ended up with more data, more intelligence, and a faster viewer—we ran into our next challenge. We needed to find new ways for investigators to navigate the content.

And so we invented Comparative Analytics. Not only does it let you deal with single entities, it allows you to compare thousands of entities in a split second. Because. There’s always a lot of data.