Threat Hunting: The Ultimate Guide for Beginners

Introduction

Consider a situation where you are a security guard in an office building. The duty of the guard is not to wait for alarms. On the contrary, the guard moves along the corridors, checks the dark areas, and looks for any anomalies. Such a practice is very similar to threat hunting in terms of cybersecurity.

Most organizations use automated tools to identify cyber threats. The firewall will block any suspicious activity. The antivirus program will protect against malware. The SIEM system will raise an alarm in case of any pattern. However, these are reactive measures because they react only in response to something suspicious.

Threat hunting is the opposite strategy. In contrast to automated tools, a threat hunter seeks to detect any threat that is already inside the organization. A person assumes that the intruder is already present there, and they need to find them.

If such a task seems crucial for today’s security teams, that is true. Research confirms that hackers spend several weeks or even months within compromised systems before detection. Threat hunting is one of the best approaches to reduce such a period.

In this guide, we take you through all the basics of threat hunting, from its definition to how it is performed, the tools and frameworks used, and even how to build this process in your company. If you are a security analyst or an executive seeking to know where to invest, then this guide should provide you with a good starting point.

What Exactly Is Threat Hunting?

Threat hunting is the process of actively looking for previously unknown or undetected threats in your environment. This is an intelligence-based operation where the assumption of compromise is made, and attempts are made to find tactics, techniques, and procedures (TTPs) used by the adversary to exploit systems.

The most important term in this sentence is proactive. In contrast to incident response, which begins when there is an alert, threat hunting begins from a hypothesis that is essentially a best guess of what an attacker might do in your network.

Allow me to give you an example to illustrate the concept.

Imagine that your threat intelligence unit discovers that some ransomware gang has been attacking companies in your industry using living-off-the-land techniques. In other words, the attackers have been leveraging legitimate Windows tools such as PowerShell and Bitsadmin for their operations without getting detected by antivirus signatures.

The threat hunter will then analyze the information gathered and make a hypothesis: “If this group has breached our system, there is a high probability that unusual PowerShell operations may occur in users’ endpoints outside office hours.”

The next step is to perform queries on EDR logs and SIEM feeds, among others, to verify if the hypothesis is true or not. This process involves finding unusual PowerShell commands in contrast to baseline data and correlating it with network traffic in order to prove or disprove the hypothesis.

Why Threat Hunting Matters Right Now

This should not be news to anyone who follows cybersecurity trends. Ransomware gangs are employing double or even triple extortion approaches. Nation-state hackers are engaging in protracted cyberespionage campaigns. Hackers are conducting supply chain attacks that exploit trusted software before it even makes its way to your network.

Traditional defenses are not sufficient by themselves. Why? For starters, here are some reasons why:

Attackers can evade detection. Current malware employs polymorphic technology to change its code signature and stay undetected. Fileless attacks do not leave any evidence on disk, since they are fully contained in memory. Living off the land exploits software that is already available in your system and, hence, leaves no trace of malicious activity.

Alert fatigue is a major problem in cybersecurity. Teams are inundated with alerts, many of which are false positives. This results in attackers slipping through unnoticed.

Dwell time is still too high. With all of our technological prowess, attacks can often go undetected for lengthy amounts of time. Threat hunting solves this issue by actively searching out these unknown threats.

This isn’t only about discovering live threats. Even if there aren’t any live threats found during a hunt, value is being created.

Threat Hunting vs. Incident Response: Understanding the Difference

This is a point of great confusion, and we shall try and make sense of it here. Incident response is reactive; it starts once an alarm sounds or a report is filed, and the objective is to contain, eliminate, and restore from the known incident as quickly as possible.

On the other hand, threat hunting is proactive; it starts from a hypothesis and not an alarm, and the objective is to identify the threat that has not yet been detected.

To make it easier to understand, a simple analogy would be Incident response is akin to calling the firefighters when your house is on fire, whereas threat hunting is like walking through your house with a thermal imaging camera in search of hotspots before starting a fire.

However, they are not competing; they are complementary. Insights from threat hunts feed back into incident response while the information from incident response gives rise to new hunting hypotheses.

The Core Components of Effective Threat Hunting

There are some basics that you have to have before you can go hunting. If you skip these, your hunts won’t be very productive.

Comprehensive Visibility (Observability)

You cannot hunt something that you cannot see. This is the most basic requirement.

At least, you need the following:

• Endpoint telemetry: process execution, file creations, registry modifications, and network connections. These are provided by tools such as Sysmon, CrowdStrike Falcon, or Microsoft Defender for Endpoint.
• Identity logs: authentication logs, privilege escalations, and account modifications. Logs from Active Directory, Okta, and Azure AD.
• Network data: DNS queries, HTTP traffic, and lateral movement between hosts. Tools such as Zeek, Suricata, or network flow data.
• Cloud logs: API calls, configuration modifications, and access patterns in AWS, Azure, or GCP.
• Logging in one place: all the above data should be available for querying, typically via a SIEM or data lake.

Otherwise, you’re hunting blind.

A Baseline of “Normal”

How will you detect the abnormal without knowing what normal is? What is the pattern of a regular login attempt? Who among the administrator accounts carries out privilege tasks, and at what time? What destinations does your system connect to over the network regularly? What processes usually operate within your web server?

The baseline builds up over time, but you must consciously create it. Every hunting effort adds to your knowledge about the environment.

Skilled Analysts

Threat hunting requires more than just skill with the tools. It requires an understanding of attacker tactics, the ability to generate hypotheses that can be tested, and to be able to trace evidence from one data source to another.

Not all SOC analysts have these skills yet, and this is okay. Threat hunting is a skill that takes practice. All that matters is having the right people—curious, analytical, and eager to learn.

The Threat Hunting Process: From Hypothesis to Outcome

Although there are many frameworks for threat hunting, the process usually follows a standard cycle. This is how it happens.

Step 1: Form a Hypothesis

Every hunting journey begins with a hypothesis: that is, a scientifically informed guess regarding what an attacker may be doing.

Good hypotheses are concrete and testable. For example, “We have some kind of malware in our system” is not a good hypothesis. Instead, try: “The attacker gained persistence through scheduled tasks running on our Windows servers.” So where do we get hypotheses?

• Threat intelligence: New intelligence on attacks targeting your industry.
• MITRE ATT&CK: Techniques for which you have not validated coverage.
• Environmental anomalies: A subtle anomaly seen in your data.
• Previous attacks: Attacks that have succeeded against you in the past.
• Industry trends: Trends that are successful against other companies in your industry.

Step 2: Find Sources of Data

Once you formulate your hypothesis, figure out what evidence needs to be gathered for its testing.

For example, if your hypothesis is related to PowerShell abuse, then you will require process execution data with command-line arguments; if your hypothesis is related to lateral movement, then you will need data related to network connections and authentication.

You need to document what data sources are available to you and where there are gaps in visibility.

Step 3: Gather and Prepare the Data

This is where the querying will take place. Logs will be gathered from the SIEM, EDR, or other sources and filtered into the data that is relevant.

This usually includes:

• Time-based filtering: Limiting data to the time frame of interest.
• Scope limitations: Specifying specific hosts, users, or network ranges.
• Normalization: Normalizing timestamp formats, IP addresses, or field naming conventions.
• Contextual enrichment: Including things such as asset criticality or threat intelligence.

Query tools used for this include SPL or KQL.

Step 4: Analysis and Investigation

Here comes the part where you find patterns. Here is where human judgment counts the most.

What are we looking for?

• Anomalies: Actions that don’t fit the baseline.
• Correlations: Several pieces of information that form a story.
• TTP matches: Actions of the technique of known bad guys.
• Chains of actions: Initial access, persistence, lateral movement, and so on.

If the hypothesis is proved wrong, it doesn’t matter; it is a good hunt anyway. You have checked it out and can proceed to other hypotheses.

Step 5: Take Action Based on Discoveries

In the case where there are any discoveries made, the hunt turns into an incident response operation. The threat will be contained, eliminated, and recovered from.

In the absence of findings or where findings include the existence of gaps in visibility, action should also be taken:

• Document the hunt process: Describe what you were looking for, how you searched, and the results (negative findings).
• Increase detections: Convert the search methods that have yielded results into detection methods.
• Address visibility gaps: Correct the sources of information you wish you had.
• Refine your hypotheses: Utilize your learning in generating future hypotheses.

Frameworks That Make Threat Hunting Repeatable

Ad-hoc hunting is an improvement over doing nothing, but systematic frameworks provide the practice with repeatability, measurability, and scalability. Here they are.

MITRE ATT&CK

MITRE ATT&CK is a collection of adversary tactics and techniques based on observations. This is basically a database of how the attackers work and is segmented by the stages of attack.

The tactics represent the “why,” that is, the objective of the attacker (e.g., initial access, persistence, and exfiltration). The techniques represent the “how,” that is, the particular ways of achieving the objectives (e.g., phishing, scheduled task, PowerShell).

In the context of threat hunting, ATT&CK provides a menu of hunting possibilities. It helps you connect your hypothesis to the relevant technique and assess your detection capability.

The PEAK Framework

The PEAK (Prepare, Execute, Act with Knowledge) framework is a freely available framework that is agnostic to any vendor and classifies hunts into three categories:

• Theory-driven: Beginning from a hypothesis regarding the activity of the attacker.
• Baseline: Determining what is considered normal to detect the anomalies.
• Model-assisted: Detecting outliers using machine learning and statistics.

This classification is useful in that it recognizes the fact that not all hunts are the same. Sometimes you are following up on a certain lead; sometimes you just want to learn more about your environment.

TaHiTI

TaHiTI (Targeted Hunting integrating Threat Intelligence) is a framework that begins from the present threat intelligence information and previous data and continues through initialization, hunting, and finalization phases. The framework is ideal when the hunt process begins with the information about threat intelligence first.

The Hunting Maturity Model

The Hunting Maturity Model (HMM) by David Bianco includes five stages of hunting, ranging from HMM0 (no hunting, purely based on automated detection) to HMM4 (automated and continuous hunting using machine learning). Normally, most organizations begin at either the HMM1 or HMM2 stage.

Practical Examples: How Threat Hunting Really Gets Done

Here are a few practical examples to help you understand the concept.

Example 1: Ransomware Precursors Hunt

The process that leads to ransomware is not something that happens overnight; there is a series of activities that occur before encryption: access, persistence, privilege escalation, lateral movement, data exfiltration, and, finally, the deployment of malware.

In the example above, a threat hunter might have the hypothesis: “If a ransomware crew has access to the system, they may use unusual remote access tools or scan the network.”

The search process would include:

• Search for programs such as AnyDesk, TeamViewer, or SoftPerfect Network Scanner operating from abnormal paths.
• Identification of network scanning behavior (frequent connections to sequential IP addresses or ports).
• Identifying the creation of any new accounts with misspelled names or the addition of new accounts to privileged group accounts.
• Auditing of any suspicious command-line processes such as PowerShell, Bitsadmin, or PSExec.

Any of these signs should be escalated to incident response.

Example 2: Credential Dumping Detection

Credential dumping is a method whereby the attacker extracts the password hash or credentials stored either in memory or on the disk. This is usually done using tools such as Mimikatz.

A hypothesis: “The attacker has dumped credentials from LSASS memory on a Windows host.”

Potential indicators of this activity include:

• Uncommon processes accessing LSASS memory. Processes accessing LSASS memory are uncommon and rare.
• Mimikatz tool signature or API call.
• Credentials access in Windows Security logs.
• Subsequent authentication events that are abnormal.

The SIEM query can be an event accessing a particular process, in this case, lsass.exe, but the accessing process should not be in a known list of security tools.

Example 3: Hunting Living-off-the-Land Methods

Attackers often utilize tools that exist on systems as legitimate tools to evade detection. PowerShell, WMI, Bitsadmin, and certutil are among the popular ones. Hypothesis: “The attacker has used certutil.exe for downloading malicious files.”

Certutil is an actual Windows tool, and it is legitimate as well; however, the -urlcache switch in Certutil is known to download files from the internet, making it a common method in fileless malware attacks.

The hunt: Look for process execution events with certutil.exe having arguments with -urlcache or -decode options. Analyze network events with connections to external IPs for downloading the files.

Tools of the Trade

You do not have to use all the tools mentioned here, but it is good to know them as you construct your own arsenal for hunting.

SIEM (Security Information and Event Management)

Examples of such systems are Splunk, Microsoft Sentinel, Elastic Security, and Sumo Logic. This is where most of your hunting queries will be executed. The system collects logs from different sources and gives you a query language to work with.

EDR (Endpoint Detection and Response)

CrowdStrike Falcon, Microsoft Defender for Endpoint, SentinelOne, and Carbon Black are some examples. EDR tools give you a lot of visibility on endpoints and also have built-in hunting capabilities. EDR information is highly valuable for hunting purposes.

Network Analysis

Tools like Zeek (Bro) and Suricata are used for analyzing network traffic. If you need deeper insights into a particular connection, use Wireshark for packet analysis.

Threat Intelligence Platforms

For gathering threat intelligence, you can make use of MISP, OpenCTI, and AlienVault OTX. These tools offer hypotheses and enrichments for your IOCs.

Hunting Tools

• Velociraptor: Endpoint Visibility and Hunting.
• osquery: SQL-based endpoint interrogation.
• Sigma: Generic format for signature creation, which can be transformed into a query for most SIEMs.
• YARA: Pattern matching for malware detection.

Querying Languages

You must be proficient in at least one of them: SPL (Splunk), KQL (Microsoft Sentinel/Defender), Elasticsearch Querying Language, or SQL.

Threat Hunting Program From Scratch

What if your company hasn’t developed a threat hunting program? What should you do?

Go Slow and Go Forth

There is no need to begin the search for advanced persistent threats throughout your entire global infrastructure on day one. Limit your scope.

Good first hunts:

• IOC Hunt: Select some indicators of the threat from the latest threat report, which is pertinent to your industry, and see if any of them are present within your environment.
• Authentication Hunt: Search for any kind of anomalies within login activity performed by any privileged account.
• Search for specific technique: Choose one ATT&CK technique and hunt for it.

Document Everything

Though any hunt may not be fruitful, they are still valuable only when documented. Document what you were looking for, how you searched, what information sources you used, and what you discovered.

Knowledge Accumulates Over Time

Each hunt gives you an opportunity to learn about your surroundings. The sources of information that provide you with accurate information, queries that prove to be effective, and your visibility shortcomings. This information accumulates.

Develop a Loop Using Detection Engineering

The successful queries should be automated in order to become detections. The moment you see a pattern that shows malicious behavior on your part, you need to make a detection out of it, either an SIEM rule or an EDR detection.

Integration with Incident Response

Threat hunting and incident response need to go hand in hand. Information gained from threat hunting is fed into the incident response process. The artifacts generated during the incident response process (such as lateral movements or unknown binaries) become threat hunting leads.

Common Pitfalls and Misconceptions

Now let’s look at some common misunderstandings. “Threat hunting is just about running IOC searches.”

False. The value of IOCs is not to be underestimated, but the problem with using IOCs is that by the time they’re released, the attacker may well have already left the scene. Threat hunting is all about behavior-based hypotheses.

“We have an EDR/SIEM, so we’re doing threat hunting.” While tools facilitate threat hunting, they don’t conduct it. Threat hunting involves hypotheses, queries, and investigations. Tools are only part of it. “If we come out of this without finding anything, the hunt failed.”

Incorrect. A hunt that debunks a hypothesis has been successful. You have ruled out one possibility. You know something new about your environment. Your baseline has been refined. The value lies in the process itself and not in the outcome.

“Threat hunting is only relevant to large companies with dedicated teams.”

Small companies can hunt as well. You can start with analysts from your SOC who possess investigation skills. Identify your high-risk zones and work on them. Just a couple of hunts every month will add significantly to your security posture.

“We must hunt everywhere, at all times.”

This is not sustainable. Limit your scope to the risk to your business and to threat intelligence. Hunt on crown jewels, your internet-exposed systems, and privileged access levels.

Advanced Corner: Going Deeper on Threat Hunting

If you feel comfortable with the fundamentals, here are some areas for you to pursue further.

Statistical and Machine Learning Methods

It’s unrealistic to do everything by manual threat hunting. Statistical approaches may help identify irregularities you’d otherwise overlook.

Time series modeling (such as SARIMA) can detect anomalies in network traffic trends. Cluster analysis may find common patterns in events and help discover anomalies. Graph analysis may uncover connections between objects hidden in tabular data.

Remember: it’s all about combining statistical methods with human expertise, not replacing the latter with the former.

Automated Hypothesis Generation

One new approach currently being studied is the process of automated generation of hunting hypotheses based on threat intelligence information. Combining knowledge graphs that relate vulnerabilities, attack techniques, and threat actors could help identify suitable hunts.

This is a relatively new concept, but it is an area that holds promise for the future of threat intelligence driving hunts.

Purple Teaming

An approach to purple teaming, which involves cooperation between red teams (offensive teams) and blue teams (defensive teams), is an excellent method for validating hunting capabilities. Blue teams can hunt for red team techniques once the red team simulation has ended, and if this is unsuccessful, a detection gap exists.

Game Theory and Adversarial Modeling

Other scientists have started using game theory to study the interactions between attackers and defenders. The point is to learn what optimal strategies there are and how to apply this to prioritizing defensive actions.

This is a rather theoretical approach for most companies, but it is a fascinating way to think about it.

References for Pursuing Threat Hunting

Below are a few references that will be useful.

Books

• “Cyber Threat Hunting” by Nadhem AlFardan: A practical book containing real-life case studies and downloadable data sets
• “Practical Threat Hunting” by S. Lee: Practical threat hunting strategies are discussed.
• “The Practice of Network Security Monitoring” by Richard Bejtlich: Understanding network visibility is key.

Frameworks and Documentation

• MITRE ATT&CK (attack.mitre.org)—The ultimate resource on TTPs of adversaries.
• PEAK Framework—Free and vendor-independent hunting framework.
• The Threat Hunter’s Cookbook—Hunting techniques from SANS.

Hands-On Labs

• TryHackMe: SOC Level 1 & 2 tracks and threat hunting rooms.
• Blue Team Labs Online: Defensive exercises in real-world scenarios.
• CyberDefenders: Blue team CTF-based labs.
• Splunk Boss of the SOC (BOTS): Threat Hunting CTF with Splunk.

What to Do Next

If you’ve decided that threat hunting makes sense to invest in, here’s how to get started.

This week: Evaluate where you stand. What kinds of information do you collect now? What’s missing? Determine which valuable data sources would be good candidates for your hunt.

This month: Launch your first hunt. Choose an easy hypothesis based on recent threats that may affect your industry. Record everything. Tell your team about the results.

This quarter: Develop an efficient process. Develop templates for hunt documentation. Establish processes and roles and responsibilities. Begin translating successful hunts into detections.

This year: Refine your approach. Move toward continuous coverage of the riskiest parts of your organization by hunting. Incorporate the outcomes of hunts into the workflow of detection and incident response teams.

Hunting is not a thing you purchase and put in place once. It’s an activity and a discipline that develops over time. Successful hunters are those who start and continue to improve their efforts. All you have to do is start looking.

Explore Our Cybersecurity Category. And if you are reading it up to here, leave a sweet comment to motivate us to write blog everyday.