psilva's prophecies: Big Data Getting Attention

Tuesday, July 23, 2013

Big Data Getting Attention

According to IBM, we generate 2.5 quintillion (2.5 followed by 17 zeros) bytes of data every day. In the last two years, we've created about 90% of the data we have today. Almost everything that's 'connected' generates data. Our mobile devices, social media interactions, online purchases, GPS navigators, digital media, climate sensors and even this blog to name a few, adds to the pile of big data that needs to be processed, analyzed, managed and stored. And you think that saving all your movies, music and games is a challenge.

This data growth conundrum is 3 (or 4 - depending on who you talk to) dimensional with Volume (always increasing amount of data), Velocity (the speed back and forth) and Variety (all the different types - structured & unstructured). Veracity (trust and accuracy) is also included in some circles. With all this data churning, security and privacy only add to the concerns but traditional tactics might not be adequate.

Recently the Cloud Security Alliance (CSA) listed the top 10 security and privacy challenges big data poses to enterprises and what organizations can do about them. After interviewing CSA members and security-practitioners to draft an initial list of high priority security and privacy problems, studying the published solutions and characterizing problems as challenges if the proposed solution(s) did not cover the problem scenarios, they arrived at the Top 10 Security & Privacy Challenges for Big Data.

They are:

Secure computations in distributed programming frameworks
Security best practices for non-relational data stores
Secure data storage and transactions logs
End-point input validation/filtering
Real-Time Security Monitoring
Scalable and composable privacy-preserving data mining and analytics
Cryptographically enforced data centric security
Granular access control
Granular audits
Data Provenance

The Expanded Top 10 Big Data challenges has evolved from the initial list of challenges to an expanded version that addresses new distinct issues.

Modeling: formalizing a threat model that covers most of the cyber-attack or data-leakage scenarios
Analysis: finding tractable solutions based on the threat model
Implementation: implanting the solution in existing infrastructures

The idea of highlighting these challenges is to bring renewed focus on fortifying big data infrastructures. The entire CSA Top 10 Big Data Security Challenges report can be downloaded here.