Malware Analysis Fundamentals

Hash Analysis

The first step is always to compute the file hash (MD5, SHA1, SHA256) and look it up in threat intelligence sources (VirusTotal, Malware Bazaar, your TIP). A known-malicious hash gives you an instant verdict and existing analysis reports.

String Extraction

The strings command (Linux) extracts printable ASCII and Unicode strings from a binary. Malware often contains useful strings: C2 URLs, registry key paths, API function names, encryption keys (sometimes hardcoded), error messages, and debug artifacts.

# Extract printable strings (minimum 6 characters)
strings -n 6 malware.exe

# Extract Unicode strings
strings -e l malware.exe

# Extract strings and search for URLs
strings malware.exe | grep -E "https?://"

# Extract strings and search for registry paths
strings malware.exe | grep -i "HKCU\|HKLM"

PE Header Analysis

Windows executables follow the PE (Portable Executable) format. The PE header contains rich metadata:

• Compilation timestamp — when the binary was compiled (often manipulated by malware authors, but still useful)
• Import Address Table (IAT) — which Windows API functions the malware imports. Imports like VirtualAllocEx, WriteProcessMemory, CreateRemoteThread indicate process injection
• Export table — functions exported by DLLs; useful for analyzing injected DLLs
• Section names — standard sections (.text, .data, .rsrc). Non-standard or high-entropy sections suggest packing or encryption
• Compiler and linker artifacts — often reveal the programming language or framework used

YARA Rules

YARA is a pattern-matching tool for malware identification. A YARA rule defines conditions based on strings, byte patterns, and PE header attributes. When a file matches a YARA rule, it is flagged as matching a known malware family or malicious pattern.

rule AgentTesla_Keylogger {
    meta:
        description = "Detects Agent Tesla keylogger based on string patterns"
        author = "Tacuns Academy (example)"
        date = "2026-05-26"

    strings:
        $s1 = "smtp.gmail.com" ascii nocase
        $s2 = "KeyLogger" ascii
        $s3 = "GetKeyboardState" ascii
        $pdb = "AgentTesla" ascii nocase

    condition:
        uint16(0) == 0x5A4D and      // MZ header (Windows PE)
        filesize < 5MB and
        (
            ($s1 and $s2) or
            ($s3 and $pdb)
        )
}

What to Monitor During Dynamic Analysis

• Network traffic — capture all connections using Wireshark or the sandbox's network monitor. Look for C2 communications, DNS queries, HTTP POSTs exfiltrating data
• Registry changes — which keys are created, modified, or deleted? Persistence mechanisms often use HKCU\Run or HKLM\Run
• File system changes — which files are created, modified, or deleted? Look for dropped payloads, encrypted files
• Process creation — which child processes are spawned? cmd.exe, powershell.exe, wscript.exe spawned by Office or browser is suspicious
• API calls — using process monitoring tools, record which Windows API functions are called and with what parameters

Sandbox Tools

Persistence Mechanisms

• Registry Run Keys — HKCU/HKLM\Software\Microsoft\Windows\CurrentVersion\Run — executes on every user logon
• Scheduled Tasks — schtasks.exe creates tasks that run on schedule, at logon, or on system event
• Services — malware registers itself as a Windows Service for automatic startup with SYSTEM privileges
• Startup Folder — drop executable in %AppData%\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\
• DLL Hijacking — place a malicious DLL in a location searched before the legitimate DLL path

Anti-Analysis Techniques

• VM detection — check for VMware/VirtualBox artifacts (registry keys, MAC addresses, running processes) and exit cleanly to avoid sandbox analysis
• Sandbox evasion — delay execution (sleep for 5+ minutes), check for user activity (mouse movement, foreground window), check for minimum RAM or processes
• Code packing — compress or encrypt the malicious payload; decompress only at runtime to defeat static signature detection
• Obfuscation — encode strings as XOR or Base64, split C2 URLs into fragments, use indirect API calls via GetProcAddress
• Anti-debugging — detect attached debuggers via IsDebuggerPresent, NtQueryInformationProcess, timing checks

C2 Communication Patterns