Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 37 ยท Behavioural Indicators and YARA

YARA Conditions and Rule Tuning

Pick distinctive strings, test YARA rules against clean and malicious corpora, measure false positives and misses, and use PE modules and performance tips.

A rule that matches the sample in front of you proves little. The real question is whether it will match other samples of the same family (few misses) while staying silent on the millions of clean files it will meet (few false positives). Writing a rule is the easy half; testing and tuning it is the work. This lesson shows how to choose strings, build a test set and measure both kinds of error.

The rules are read-only examples because real YARA cannot run on this site. The experiments use Python stand-ins over generated data, so the idea of a corpus test is runnable, but the numbers are from invented files.

Two ways to be wrong

False positive Rule fires on a clean file.
False negative Rule misses a family member.

Tuning trades one against the other.

1 / 2

Choosing distinctive strings

Not all strings are equal. The useful ones are present in all (or most) family samples and rare elsewhere. Strings extracted from malware include a lot of noise: common API names, runtime library text, compiler boilerplate and Windows resource strings that appear in thousands of clean programs.

Ways to find the good ones:

  • Compare strings across several samples of the family and keep what they share.
  • Subtract everything that appears in a goodware corpus (a folder of clean programs and libraries from a fresh system install).
  • Prefer program-specific text: mutex names, debug paths (D:\work\demo\src\gate.cpp), custom error messages, unusual protocol keywords, rare constants.
  • Prefer code over data when the author reuses code between builds: a hex string of the core routine with wildcards for addresses survives recompilation with different strings.

The first stand-in automates the "share and subtract" idea on small made-up sets of strings.

Python Find strings shared by the family but absent from clean files

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The result keeps the mutex and the gate.php page, drops CreateFileW and the generic error text, which clean files contain too. Updater is also left out of the partial list because a clean program uses it, so it is not distinctive on its own. Real goodware sets are far larger, and tools such as yarGen automate this idea, but you must still review what they suggest.

Testing rules on a corpus

Keep two collections: a malicious set (samples of the family you are targeting) and a clean set. Run each candidate rule against both and count:

  • True positives: family files that match.
  • False negatives: family files that do not match.
  • False positives: clean files that match.

Then compute precision (of the files that match, how many are really malicious) and recall (of the malicious files, how many matched). A rule for blocking in production needs very high precision; a hunting rule for researchers may accept lower precision for better recall.

The next stand-in compares three condition styles on generated corpora.

Python Measure precision and recall of three rule variants

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

You should see the pattern clearly: any of has the best recall but a few false positives (the clean tool with a gate.php page), all of is precise but misses variants that dropped a marker, and 2 of is the usual sweet spot. The Updater rule shows what happens with a weak indicator, many false positives, which is exactly why shared text such as that gets rejected. The exact figures vary only with the seeded generator, so they are repeatable.

Useful conditions for real tuning

Beyond string counts, YARA offers ways to be more specific without losing coverage:

  • Position: $decode in (0..filesize/2) or $mz at 0 rules out the same bytes appearing in unrelated places.
  • Size bounds: filesize < 500KB for a small loader excludes big installers that happen to include the same string.
  • Counts: #cmd > 5 helps for scripts where one occurrence is normal.
  • File type checks: magic numbers (MZ, %PDF, PK) first, for speed and accuracy.
  • Negations: not $vendor_string excludes a known clean product that shares code, but use sparingly because it can hide evasion.

The PE module

YARA ships modules that parse formats. The pe module exposes headers, sections, imports and resources, which lets a rule use structure rather than raw bytes (read-only example):

TEXT
import "pe"

rule Demo_Packed_Loader
{
    condition:
        pe.is_32bit() and
        pe.number_of_sections <= 4 and
        pe.imports("kernel32.dll", "VirtualAlloc") and
        pe.imports("kernel32.dll", "VirtualProtect") and
        for any s in pe.sections : (
            s.name == ".pack" and s.raw_data_size == 0
        )
}

This rule echoes the packing clues covered earlier: few sections, memory-permission APIs and an odd section with no raw data. Other modules include math (compute math.entropy(offset, size) to apply entropy analysis in a rule), hash (match an md5 of a section or resource) and dotnet for .NET metadata. Imports-based rules using pe.imphash() find files that share import tables.

Performance

Rules run over millions of files, so efficiency matters:

  • YARA looks for fixed substrings first, so strings starting with unusual bytes are fast. Strings starting with ?? or very short atoms (one or two bytes) slow the scanner and generate warnings.
  • Order the condition so the cheap checks come first: size, magic number, then strings, then expensive loops over sections.
  • Avoid unbounded regular expressions such as /.*example.*/ and wide gaps [0-5000] in hex strings.
  • Keep a regression suite: every rule ships with the samples it should match and the clean files it must not. Re-run the suite whenever you change a rule.

A rule that matches yesterday's sample only is a hash in disguise. Aim to match the behaviour of the code, and expect to maintain rules as families evolve.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A rule matches 95 of 100 malicious files and 3 of 10,000 clean files. Which statement is right?

    Clean files that match are false positives (3); malicious files that do not match are false negatives (5).
  2. 2. Which string is the best candidate for a family rule?

    Distinctive strings appear in family members but rarely in clean software.
  3. 3. Why does a goodware corpus matter?

    Testing against clean files exposes weak strings before the rule reaches production.
  4. 4. Which pe module function would you use to find files sharing an import table?

    pe.imphash() returns the import hash, which links similar builds.

Summary

  • Choose strings shared by the family and rare in clean software; subtract a goodware corpus.
  • Measure true positives, false negatives and false positives; track precision and recall.
  • N of (...), positional checks, size bounds and magic-number checks tune a rule.
  • The pe, math and hash modules let rules use structure, entropy and hashes.
  • Put cheap checks first, avoid slow patterns and keep a regression suite.
  • The corpus code is a Python stand-in on invented data.

Next: Behavioural Patterns and API Sequences