By Peter Chen, Senior Manager of Data Science & Analytics at Mitchell International
It’s undeniable that we are living in the age of Big Data. More data is being generated now across all industries than any other time in human history. The workers’ compensation and property and casualty (P&C) industries are no different.
Experts claim that the three Vs of Big Data are:
- 1. Volume
- 2. Velocity
- 3. Variety
The volume aspect of Big Data is fairly obvious as it’s in the name. However, this may not necessarily be the most critical aspect of Big Data since a lot of companies can claim to have large volumes of data. Velocity can be defined as the speed at which data comes at us, and this is very application dependent. All data isn’t necessarily received in real-time as it is in social network feeds or stock prices. Therefore, velocity as criteria for Big Data can be a moot point for most industries that do not use or need real-time processing of data. Variety of data, however, is perhaps the most underappreciated and potentially the most valuable aspect of Big Data.
There are many potential applications of Big Data and predictive analytics in the workers’ compensation space. However, the most fruitful and exciting ones all involve unifying a variety of data sets to get a richer and more complete picture of the full spectrum of risks and opportunities embedded in claims.
For example, workers’ compensation fraud and opioid risk detection are highly complex and multi-faceted issues that require looking at the problem from multiple angles and views to properly and proactively identify risks early in the process. This might be possible when one incorporates a variety of data beyond only claims information stored in databases. Similarly, parsing text information from emails and social media interactions and images from photos offers a 360-degree view of risk detection using Big Data technologies.
The benefit of unifying various data sets goes far beyond fraud and opioids risk detection, but what’s often not discussed is the immense complexity of implementing this approach using traditional BI (Business Intelligence) technologies. This is where new Big Data technology frameworks and best practices come in play and really shine.
Big Data’s best practices propose bringing all of the various data from disparate sources into a data repository called a ‘data lake’ using open-sourced technology like Hadoop, which runs on low-cost commodity hardware. The storage cost of housing a data lake would be prohibitive in traditional BI databases, not to mention future costs of scaling these storage requirements would be prohibitively expensive.
The real power of a data lake is that it allows flexibility to decide which data elements to connect in order to find interesting insights, which after all is the real promise of Big Data Analytics. On the other hand, traditional BI systems require that users know which questions to ask in advance by specifying the data elements joined beforehand. This is a powerful competitive and implementation edge that Big Data has over traditional BI, because in traditional BI the data has to be pre-categorized at the point of entry to the data warehouse. That requires a lot of upfront time and IT investments to determine the optimal form of data storage that might later be negated by a new slew of questions that the predetermined data structures do not support in the data warehouse.
With proper Big Data best practices, we can further explore and confirm that variety of data is potentially the most valuable V of the three Vs of Big Data for providing richer and deeper levels of insight and success in the workers’ compensation and P&C industry.
About Peter Chen
Peter Chen is the Senior Manager of Data Science & Analytics at Mitchell International. He’s part of the Innovation & Design team working on applications of data science & analytics to help create innovative data products and services. He has an eclectic background and has previously worked in various industries such as quantitative investments, internet startup, energy/utilities, retail, and currently software for P&C. He was profiled in a media article about his analytics and data mining experience. He was the co-author of a published technical paper on using similarity measures in image database for research work conducted while he was still an undergrad. He received his Bachelor of Science from Massachusetts Institute of Technology and his masters at Harvard.