Hacker using laptop. Hacking the Internet.

How Big Data Helps Detect Hacking

Cyberattacks have become increasingly damaging and visible in recent years, in part because of numerous, high-profile instances of hacking affecting everything from your personal files to global election outcomes.

And, in fact, hacking is no longer just about unauthorized access to, or loss of data. As the internet of things continues to expand, so has the threat of so-called “kinetic” cyberattacks, attacks that result in physical damage or worse, the loss of human life. In what was perhaps the highest profile instance of a kinetic cyberattack, the Stuxnet worm destroyed Iranian nuclear centrifuges by manipulating the code that controlled their speed.

Another growing concern is data sabotage, the subtle manipulation of data within transactional databases with the aim of some direct or indirect benefit.  Cyber criminals have realized that they can reap similar benefits from changing information as they can from stealing it, and stand a higher chance of going undetected by current monitoring tools.  This is of particular concern in the financial sector, where tweaking certain numbers, such as the revenue figures on an earnings report, or the price of a recent stock transaction, can send ripples through the stock market and cause billions of dollars in damage.

Thankfully, the good guys are keeping up and developing strategies to thwart modern cyberattacks.  We take a look this week at how cyberattacks have traditionally been detected and how data-centric threat detection is changing the cybersecurity sphere, leading security companies to take a highly contextualized and analytical approach to threat detection.

Scalability and Data Unification: To catch intrusions, you have to look at every piece of data

Traditional security event and information management software does not collect enough data to be able to detect modern, sophisticated attacks.  And, while they do use some historical data, many do not have the storage or processing capabilities to analyze anything older than 30 days, which can lead them to ignore key abnormalities.  Additionally, these tools examine different sources of data individually and not in conjunction (i.e. not correlated) with one another.

So new tools have emerged which take into account the size, speed, complexity and variety of data in order to detect the new generation of cyberattacks.  The new paradigm calls for layering predictive analytics and machine learning algorithms on top of all sources of data in an organization’s cyber infrastructure.

With such quantities of data, well-designed visualization is essential

Visual representations of infrastructure data can help make security vulnerabilities obvious. However, today’s security professionals are not well versed in data visualization.  Typically, they are only formally trained in computer science, statistics, and security.  But in situations where data is captured across much longer time horizons and from multiple, disparate sources, well-designed visualization becomes indispensable to threat analysis.

Those companies that do use visualization tools have traditionally used them for post-attack illustration and not for analysis of real-time threats.  But the integrated platforms described above, when paired with elegant and streamlined visualization, now give users the ability to quickly and accurately identify system vulnerabilities.

Real-time is a necessity

A few weeks ago, we wrote about the importance of knowing when real-time analytics can help your strategic decision making and security is one of the cases where it makes all the difference.  When it comes to data-centric security, it is imperative that your platform possess the ability to process all of the information going in and out of your network in real-time.

Cybersecurity is getting more expensive, making smaller companies more vulnerable

It used to be the case that hackers would target massive corporations with large-scale cyberattacks intended to disrupt thousands of systems and make front page news.  By contrast, the modern cyberattack is more likely to be a low-profile attack on confidential data, intended to go unnoticed.  Smaller companies are the most vulnerable, because they can’t afford to implement and manage systems that track the big data moving through the endpoints of their organizations.

The software and human talent to enable this type of monitoring are not necessarily expensive, but the hardware to handle the processing of such massive amounts of data can be extremely costly.  Thus, your security approach should depend chiefly on the value of the assets you are protecting.

Here are a few options for protecting your IT systems

Platfora and MapR offer a security solution that combines data transformation, visualization and analytics on top of a native Hadoop platform, allowing multiple varieties of data to coexist in a single repository.  By combining the scalable platform with sensors at the gateways of the business IT infrastructure, the algorithms are able to detect irregularities and then present them in a visually digestible way to end-users.

Another platform, Sqrrl Enterprise, uses a three step, data-driven approach to expose threats and intrusions.  It allows users to embark on “hunts” driven by established indicators or also “exploratory hunts,” driven by hypotheses and optimized using their automated analytics and machine learning processes.  Then, by scouring current and historical network data coming in and out of your organization, Sqrrl is able to pinpoint threats that other security solutions would have ignored.  In identifying and disrupting attacks, the system learns to generate new indicators to inform future hunts.  And like all user-friendly platforms, it offers advanced risk scoring and visualization capabilities.

Moving Ahead

Automation and the big data it produces is a double edged sword; it brings huge business benefits but all of that data and reliance on technology also introduces major security challenges. Just like a recursive algorithm, though, it’s fascinating to see how these same technologies are being used to monitor and protect themselves.

Need Help with Your Big Data and Data Science Efforts?

For almost 25 years Dataspace has helped our client navigate the opportunities presented by big data, analytics, and data science. We provide both data strategy consulting and cost-effective, highly talented implementation staff. Want to kick around your needs? Contact us today!

Scroll to Top