The Rise of the Threat Hunter

If you work in cybersecurity, you cannot afford to overlook any warning signs of a potential threat. Every day, new cyber threats, adversaries, and computer hacking tools emerge, each more advanced than the previous. The volume of cybersecurity data is increasing rapidly and overwhelming professionals with noise. Meanwhile, the number of defenders is decreasing while the dark side is growing stronger with organized crime and state funding support. What can save the world? Obviously, superheroes, but behind every successful superhero is a technological assistant - when it comes to cyber threat hunters, a data scientist.

This is the twelfth article in our ongoing "Rise of the Threat Hunter" blog series. You should now be very familiar with our superheroes, our threat hunters. For more information about the series and to find previous posts, please check our series introduction or read last week's article "Equipping Threat Hunters: Advanced Analytics and AI - Part 1".

Data Scientists Among Threat Hunters

At first glance, data scientists may seem out of place among threat hunters, but their complementary skills can elevate a threat hunting team from good to extraordinary. Cyber threat hunting involves more than just technical skills and the following procedures. It requires analytical, inquisitive minds and creative approaches. Interestingly, these are exactly the skills required for a successful data scientist. One reason why data scientists and threat hunters work so well together is precisely this.

Data scientists spend their days interacting with data, uncovering hidden patterns and insights that cannot be found using traditional methods. They are professionals with expertise in various areas such as computer science, mathematics, statistics, and machine learning. Collaboration between threat hunters and data scientists is extremely important for effective cyber threat research. While threat hunters determine what to look for, data scientists figure out how to extract these signals from vast and complex datasets. In addition to effectively articulating problems, thoroughly cleaned and prepared data can provide access to a completely new level of data insights that cannot be reached by directly searching and querying the original data. This can make the difference between missing a match or catching it early before any damage is done.

Skilled Operator or Simple Button Pusher

Cybersecurity tools have changed. Ongoing developments bring together the latest technological advancements such as machine learning, intelligent asset discovery and asset analysis to greatly increase speed and efficiency. However, these advancements have also led to an increase in the complexity and intricacy of stopping threats.

This has made security tools less intuitive and more explanatory. Along with the increasing complexity, there is also a risk that threat hunters may shift from being skilled operators of advanced tools to simple button pushers. The reason for this is not that they do not want to learn the tools, but rather that most threat hunters do not have the time to specialize in a single tool within the security stack.

Matching data scientists with threat hunters in a cyber defense team can solve this problem. Data scientists not only know various methods to extract information from data, but also deeply understand the weaknesses and common pitfalls of these methods. More importantly, they understand the weaknesses of the data these methods are applied to and how to overcome them.

For example, let's look at request engineering. Grammar, length, tone, and sentence structure can make the difference between receiving an understanding response or none at all. Worse than not receiving a response is receiving a wrong one, known as hallucination. Data scientists can work with threat hunters to design effective requests using their domain knowledge and data information. Threat hunters and data scientists working as a team can produce high-quality results effectively.

Asset Analysis Problem

Another example of effective teamwork between threat hunters and data scientists is asset analysis, which involves linking all events and behaviors associated with a single asset to that asset. This problem consists of two parts. First, we need to determine what constitutes an asset and what the best level of detail and separation methods are. Second, we need to maximize the relationship between all activities recorded in various data sources and asset representations associated with one asset. Although this may sound simple, the way both problems are solved will directly affect the quality of the results generated by the data.

Let's say there is a system administrator with two accounts in an organization. A normal user account is used for daily tasks, while the other is an administrator account with special privileges. Should we combine these two accounts under one asset or keep them separate? From a threat hunter's perspective, both accounts belong to the same employee, so grouping them seems natural. However, from a data science and behavioral analysis perspective, these two accounts serve different functions, use different processes, have different privileges, transfer different volumes of data, and even have different activity models. Furthermore, when comparing their behaviors with their peers, for best results, the administrator account should be compared with other administrator accounts and the normal user account with other normal user accounts. Running PowerShell scripts that add or remove users or elevate user privileges is quite normal for an administrator, but extremely abnormal for a regular user.

Integrating Data Science into the Threat Hunting Process

At this stage, I hope you have been convinced that your threat hunting team needs data scientists. There are a few options for acquiring one: you can hire a new data science team member, use external data science consultants or paid services, or you can cultivate the skills of your existing cybersecurity team members. Once you have brought together your multidisciplinary team, they can start by addressing one use case at a time. Since ideas for solutions and use cases will come from both your threat hunters and data scientists within your cyber defense team, having a variety of expertise in your team will be an advantage.

Based on my experience working with threat hunting teams, applied threat hunting experts usually bring details and the methods they use to deal with these situations from their field experiences. After explaining the complexities of the use case, the methods used to detect it, and the limitations of the existing tools, data scientists can examine the data. They consider various aspects of the problem and the available data, sometimes approaching it as a mathematical or logical problem. This forms the basis for brainstorming new detection methods, testing and validating them, and finally permanently integrating these new solutions into the arsenal of frontline defenders.

A member of the data science team can also discover a new use case based on new algorithms, computational methods, or data sources. In this case, the data scientists share their findings with threat hunters and brainstorm together to determine whether the findings are valuable and how they can improve existing solutions. For example, the data science team may discover that command-line inputs have patterns similar to natural language and suggest that threat hunters use language-based models to uncover discovery attacks. Through close collaboration with threat hunters, a new and effective method for command-line discovery was developed and the idea was tested and evaluated through multiple iterations.

In Conclusion

Data scientists help threat hunting teams work not just more, but smarter. When properly integrated and appropriately included, data scientists can help reduce the burden on our superhero threat hunters. Ideas that bounce back and forth among all team members in multiple cycles naturally expand and develop the scope of various cybersecurity use cases. Since this race to protect an organization never ends, and the defenders' team never "ends," this kind of idea flow should never stop. However, this is also the difficulty and beauty of the profession.

Learn More About OpenText Cyber Security

Ready to empower your threat hunting team with products, services, and training to protect your most valuable and sensitive information? For proactive identification, prioritization, and investigation of abnormal and malicious behavior, explore our comprehensive security solution portfolio that offers 360-degree visibility across endpoints and network traffic for threat hunters and security analysts. View our Cyber Security Portfolio.

Source: Equipping Threat Hunters: Advanced Analytics and AI - Part 2 - OpenText Blogs