How Federated Data Lakes Transform Big Data into Business Insight

By Matthew Lempriere, Telstra

Firms operating in financial markets today face increasing challenges around the growth of ‘big data’. With the ability to collect, record, store and retrieve more data than ever before, how do firms make sense of it all in a way that provides useful and useable business insight?

The big data challenge
Whether a firm wants to understand its customers’ needs better by examining interactions across multiple channels, or to satisfy regulatory demands by locating all data relating to a particular set of transactions, the requisite information has probably been captured in some form and is residing somewhere. The challenge is how to find it, access it and use it in a meaningful way.

‘Data lakes’ can help address this challenge by enabling users to rapidly analyze huge quantities of data from multiple sources – all in real time – through a single interface. The result? Meaningful insight from every relevant touch point.

How does a data lake work?
A data lake is a virtualized application framework driven by metadata in which multiple databases are viewed as a single entity. This allows analysis and visualization of both upstream and downstream data from a single point.

When this federated framework is queried, the system identifies which underlying databases contain the required data, passes the request to them, and returns the results to be combined and aggregated, thus providing insights and answers when and where they are needed.

The underlying data can be event-driven or static, structured or unstructured, and accessed on demand from multiple sources. But to the user, it has the appearance and performance of a single data repository.

Use cases
One of the key success factors of working with data lakes is getting the metadata right, so that the federated framework knows how and where to access the underlying source data. Although this requires careful design, there are a growing number of use cases for firms who are able to get this right, including:

Sentiment analysis
Advanced analytics, which combine and monitor relevant voice, email, instant messages and other data sources such as social media feeds, can be used to identify trends that would otherwise be unseen, perform sentiment analysis or changes in market perceptions.

Fraud detection
Data lakes can be integrated with ‘rules engines’ to help detect fraud by identifying data combinations, indicating potential risks of money laundering and emerging fraud types.

Risk control
Faster, more accurate risk analysis can be performed on multi-source data queries, such as those required for bank stress testing, enabling more precise and timely risk control measures to be taken.

Trade reconstruction
All data sources involved in a particular trade can be seamlessly queried by compliance and surveillance teams to recreate the data flow at the moment that trade occurred.


Storage, network, compute and cloud
The federated query engine within a data lake is a virtualized abstraction that can run against most data sources, so firms do not need to re-architect their existing data structures. This means they can leverage their current data capture and storage technology investments.

However, in order to use data lakes most effectively, firms need to consider their current and future infrastructure requirements around storage, network, and compute. Cloud-based services – whether public, private or hybrid – can be a big help in this regard because they enable firms to scale up and down according to demand, without additional capital expenditure.

Virtualized servers, software-defined networking and on-demand compute resources can enable firms to implement their own data lakes quickly and efficiently. This in turn can transform how they use analytics across the enterprise, to turn data into knowledge and facilitate better-informed business decisions.

Matthew Lempriere is Telstra’s Global Head of Financial Services Market Segment based in Hong Kong, where he is responsible for identifying and leveraging opportunities for the company within the sector across Asia Pacific, EMEA and the US.

About Telstra
Telstra is a leading telecommunications and technology company with heritage that is proudly Australian and a longstanding, growing international business. Today, we have around 36,000 employees offering consumer, small business and enterprise services, with more than 3,000 based in 22 countries outside of Australia. Our customers are global, our people are local.
Telstra US: +1 877 835 7872


Follow Us:

Sitemap | Privacy | Copyright © © 2017, WSTA®, All Rights Reserved.