Skip to main content
Version: Next

Malware analysis

  • Reverse engineering of a malware program
  • Purpose is to
    • determine how the malware works
    • assess the potential damage it could cause
  • Helps find and remove the infections that exist in a system through using designed tools an techniques.

Malware analysis types​

Static malware analysis​

  • Analyzing the malware without running or installing it
  • Malware's binary code is examined
  • Checks for any data structures or function calls that have malicious behavior.

Dynamic malware analysis​

  • Requires the malware program to be running in a monitored environment such as sandbox or a virtual machine.
  • Helps in understanding how the malware works by monitoring its activities on the system.

Windows integrity monitoring​

Port monitoring​
  • Involves monitoring services running on different ports.
  • Features can include
    • analytics for packet rates, CPU, power, and bandwidth of ports
    • mirroring the traffic from one port to another
  • 📝 Tools include
    • netstat (terminal)
      • Displays network connections, available on many OSes
      • E.g. netstat -an to display all connections and listening ports (-a) in a numerical format -n
    • TCPView (GUI)
      • Windows tool to enumerate network connections and owner processes
      • Refreshes automatically
    • CurrPorts (GUI)
      • View open ports and connections per process on Windows
  • See also • Common ports to scan | Scanning networks • Common ports and services to enumerate
Process monitoring​
  • Use e.g. Process Monitor to see what processes malware starts
  • Built-in sc command provides all sorts of information about running services on a Windows machine.
    • E.g. sc query to lists the running services
Registry monitoring​
  • Registry contains information, settings, options, and other values for programs and hardware installed on all versions of Microsoft Windows operating systems.
  • Malware modifies registry including keys such as Run, RunServices, RunOnce, RunServicesOnce, HKEY_CLASSES_ROOT\exefile\shell\open\command "%1" %*.
  • Use native regedit or e.g. RegScanner, Registry Viewer, Active Registry Monitor to monitor registry changes.
Windows services monitoring​
Startup programs monitoring​
Event logs monitoring/analysis​
Installation monitoring​
Files and folder monitoring​
Device drivers monitoring​
  • Malware installs with some infected drivers
  • Drivers can be seen by: Run -> msinfo32 -> Software Environment -> System Drivers
  • Tools include DriverView, Driver Booster
Network traffic monitoring/analysis​
DNS monitoring/resolution​
  • DNSChanger is a DNS hijacking Trojan that can point DNS entries toward malicious name servers.
  • Use e.g. DNSQuerySniffer, DNSstuff.
API calls monitoring​
  • Malware use Windows APIs to perform malicious task
  • API call monitoring tools include API Monitor, Runscope
System baselining​
  • Allows monitoring security configuration changes over time
  • Flow
    1. Take snapshots before and then after malware execution.
    2. Compare the snapshots to understand changes made by the malware.

Unix integrity monitoring​

  • Display processes: ps -ef
    • -e: selects all processes
    • -f: switch provides a full listing

Sandboxing​

  • Technique in which you create an isolated test environment
    • Allows secure experimentation
    • Nothing (no harm) can be spilled out of the environment.
      • If something happens, the damage is confined to that sandbox
  • Examples
    • Chrome web-browser
      • Sandboxing through multi-process architecture.
      • One or more processes are assigned to run scripts of each site.
      • Each Chrome extension and app runs in its own process
    • Virtual machines
      • Good for testing / reverse engineering malware
      • E.g. YouTubers messing with scammers utilizes virtual machines, video, video
      • 💡 Good hypervisor is important to ensure nothing goes out of the environment.
        • E.g. KVM (used by AWS) is good on AWS, and Hyper-V in Windows
          • KVM installation in Fedora: dnf install @Virtualization and then virt-manager to start a GUI.
        • VirtualBox is not as feature rich.
      • 💡 Make sure host environment is safe in first place
        • E.g. in Linux you can enable Security-Enhanced Linux (SELinux).
          • Supported by Fedora, Debian, Ubuntu, used by default by Android.
          • setenforce 1 to enable, getenforce to query status

Anti-malware software​

  • Includes e.g. antivirus, anti-spyware, anti-trojans, anti-spamware, anti-phishing, and email scanners.
  • Helps detecting, mitigating, preventing and repairing any damage by malware.
  • Looks for behavior typical to viruses and give warnings.
  • Looks for already known virus signatures and warns the user if a threat is found.
  • E.g. Kaspersky, McAffee, AVG, Norton, Avira, Bitdefender

Detection types​

  • Signature-based
    • Compare file hash and malware hash
    • ❗ Anything new or custom written will not be detected
  • Rule-based (behavior-based)
    • 📝 Relies on differentiating expected vs anomalous behavior
    • Analyzes certain characteristics of a program.
      • E.g. application accessing user login file. Why?
    • Can utilize AI & ML to decide whether something is a malware.
  • Sandboxing
    • Creates environment, lets program run and examines its behavior.
    • Good to find out behavior of e.g. self-modifying code, encrypted code.

📝 Virus detection methods​

  • Scanning
    • Scans malware for known signatures (characteristics)
    • ❗ Only known and predefined viruses can be detected
  • Integrity checking
    • Verifies files against their recorded integrated data
  • Interception
    • Intercepts the virus if it detect suspicious behavior (e.g. network access) and asks user if the user wants to continue.
    • Useful for logic bombs (only executed if certain conditions are met) or trojans
  • Code emulation
    • Executes a virtual machine mimicking CPU and memory
    • Useful against encrypted, polymorphic or metamorphic viruses
  • Heuristic analysis
    • Helps in detecting new or unknown viruses
    • Static: anti-virus decompiles and analyzes the binary
    • Dynamic: anti-virus runs code emulation to determine if the code is viral
    • Prone to many false positives

Malware countermeasures​

  • Use up-to-date anti-virus, firewall and intrusion detection software with regular scans
  • Block all unnecessary ports at the host and firewall.
  • On Windows
  • Enable Address space layout randomization (ASLR)
  • Do not open files with more than one file type extension
  • Use anti-malware software
  • Avoid accepting executables sent as messages or downloaded from untrusted sources.
  • Inspect network packets using protocol monitoring tools

Data Execution Prevention (DEP)​

  • 📝 Marks memory regions as non-executable, such that an attempt to execute machine code in these regions will cause an exception
  • Executable space protection in Windows
  • Read more on Data Execution Prevention | Microsoft Docs

Address space layout randomization (ASLR)​

  • 📝 Prevents exploitation of memory corruption vulnerabilities.
  • Involves randomly positioning the base address of an executable and the position of libraries, heap, and stack, in a process's address space
  • Breaks assumptions that attackers could make about where programs and libraries would lie in memory at runtime