Semgrep (Open Source Edition)

Semgrep is an open-source, multi-language static analysis tool that matches code against patterns written in a syntax resembling the target language itself, rather than requiring you to learn a separate AST-query language. It's a practical, widely-adopted implementation of the SAST technique covered conceptually in SAST and SCA — this article is the hands-on companion: installing it, running it, integrating it into CI without a hosted account, and writing custom rules for your own codebase's specific patterns.

Scope: this article covers the free Semgrep CLI (Community/Open Source Edition) that runs entirely locally. It notes where functionality crosses into Semgrep's paid AppSec Platform (semgrep login, semgrep ci, the Pro rule engine) so you know the boundary, but the practical guidance here is built around what works without an account or a subscription.

Installation

# pipx (recommended — isolates the tool from any project virtualenv)
pipx install semgrep

# pip
pip install semgrep

# Homebrew (macOS/Linux)
brew install semgrep

# Docker — no local install needed
docker run --rm -v "${PWD}:/src" semgrep/semgrep semgrep scan --config auto

Your First Scan

cd my-project/
semgrep scan --config auto

--config auto detects the languages and frameworks in your project and pulls a relevant rule set from the Semgrep Registry over the network — no login is required to use registry rules this way. If you're in an air-gapped or network-restricted environment, auto and named registry packs (below) won't work, since they require reaching the registry; point --config at a local directory of rules instead (covered later).

Choosing Rule Sets

semgrep scan --config p/ci
semgrep scan --config p/owasp-top-ten --config p/secrets   # combine multiple configs

Start with p/ci or auto for a first pass — broader packs like p/security-audit surface more findings but also more noise, and are better introduced after you've tuned out false positives from a narrower starting set (see the rollout guidance in SAST and SCA).

Fully Local / Offline Rule Sets

For air-gapped environments, or simply to pin an exact rule set under version control instead of depending on the live registry, point --config at a local file or directory:

semgrep scan --config ./semgrep-rules/

A directory of .yml/.yaml rule files works the same way a registry name does — this is also how you run your own custom rules (see below) alongside or instead of registry ones.

Output Formats

semgrep scan --config auto --json --output results.json          # machine-readable JSON
semgrep scan --config auto --sarif-output results.sarif           # SARIF — consumed by GitHub code scanning and similar
semgrep scan --config auto --error                                 # exit code 1 if any findings — for CI gating

--error is what turns a scan into an actual CI gate — without it, Semgrep reports findings but always exits 0, and a pipeline step that ignores its own tool's findings isn't a gate at all.

Scoping What Gets Scanned

semgrep scan --config auto --include="*.py" --include="*.js"
semgrep scan --config auto --exclude="tests/fixtures/*"

For anything scanned repeatedly (which is to say, anything in CI), put persistent exclusions in a .semgrepignore file at the project root, using gitignore-style syntax:

# .semgrepignore
vendor/
node_modules/
dist/
**/*_test_fixtures/

Exclude vendored/third-party code you don't control and can't act on findings in, build output, and any directories of intentionally-vulnerable sample/fixture code — these are pure noise sources that erode trust in the tool's output over time if left in scope.

Diff-Aware Scanning Without a Hosted Account

Semgrep's fully-featured CI command, semgrep ci, only reports findings introduced in the current diff when run in a pull/merge-request context — but it requires semgrep login and ties into the Semgrep AppSec Platform, which is a separate hosted product outside this article's "Open Source edition" scope.

The pure-CLI equivalent is --baseline-commit, which needs no account:

# Only report findings not already present at this commit
semgrep scan --config auto --baseline-commit=$(git merge-base HEAD origin/main) --error

This is exactly the "gate new code only, not pre-existing debt" rollout phase described in SAST and SCA — implemented entirely with the free CLI, no platform account needed. Run it as your PR-triggered CI gate; run a plain semgrep scan --config auto (without --baseline-commit) on a schedule against the full codebase to keep visibility into existing debt without blocking on it.

Writing Custom Rules

Registry rules cover common, general patterns. The real value of a static analysis tool over time comes from encoding your own codebase's dangerous patterns — a deprecated internal helper function, a footgun-prone internal API, a pattern from a past incident you never want repeated.

Basic Pattern Matching

Every rule needs id, message, severity (ERROR, WARNING, or INFO), languages, and exactly one pattern operator:

rules:
  - id: insecure-md5-hash
    message: >-
      MD5 is cryptographically broken and unsuitable for password hashing or
      integrity checks against a malicious adversary. Use SHA-256 for
      non-password integrity checks, or Argon2id/bcrypt for passwords.
    severity: ERROR
    languages: [python]
    pattern: hashlib.md5(...)

The ... is Semgrep's ellipsis operator — it matches any arguments, so this rule fires on hashlib.md5(data), hashlib.md5(data, usedforsecurity=False), or any other call shape.

Combining Patterns

rules:
  - id: sql-query-missing-parameterization
    message: "String-built SQL query — use parameterized queries instead"
    severity: ERROR
    languages: [python]
    patterns:
      - pattern-either:
          - pattern: $CURSOR.execute(f"...")
          - pattern: $CURSOR.execute("..." + $X + "...")
      - pattern-not: $CURSOR.execute(..., $PARAMS)

Metavariables

$X-style metavariables capture a value. Within a single patterns (AND) block, the same metavariable name must match the same value in every sub-pattern it appears in; across pattern-either (OR) branches, each alternative is evaluated independently. Constrain a metavariable's actual value with metavariable-regex or metavariable-comparison:

      - metavariable-regex:
          metavariable: $VARNAME
          regex: (?i)(api_key|secret|token|password)

Taint Mode — Tracking Data Across Statements

Plain pattern matching only sees a single expression shape. When a vulnerability depends on untrusted data flowing from one place to another — potentially across several lines — use mode: taint with pattern-sources, pattern-sinks, and optionally pattern-sanitizers:

rules:
  - id: flask-request-to-os-system
    message: "Untrusted Flask request data reaches os.system() — command injection risk"
    severity: ERROR
    languages: [python]
    mode: taint
    pattern-sources:
      - pattern-either:
          - pattern: request.args.get(...)
          - pattern: request.form.get(...)
          - pattern: request.values.get(...)
    pattern-sanitizers:
      - pattern: shlex.quote(...)
    pattern-sinks:
      - pattern: os.system(...)

This flags any path where data from a Flask request reaches os.system() without passing through shlex.quote() first — regardless of how many lines or intermediate variables sit between the source and the sink, as long as they're in the same function.

Know the limitation this implies: the free/Community Edition engine performs intraprocedural analysis only — taint tracking works within a single function. If the tainted request value is passed into a helper function defined elsewhere, or the sink is in a different file, the OSS engine won't connect them; that cross-function and cross-file (interprocedural) analysis is specifically what Semgrep's paid Pro Engine adds. Write taint rules with this boundary in mind — the closer your source and sink are to the same function, the more reliable OSS-edition taint mode will be. This is the same fundamental limitation flagged generally for SAST tools in SAST and SCA, just concretely visible here as an editor-vs-engine distinction rather than an abstract caveat.

Testing Custom Rules

Don't ship a custom rule without a regression test — an untested rule is as likely to silently stop matching after a refactor as it is to actually catch anything.

Create a test file with the same base name as the rule, using the target language's extension:

rules/insecure-md5-hash.yaml
rules/insecure-md5-hash.py

Annotate expected matches and non-matches directly in the test file:

# ruleid: insecure-md5-hash
h = hashlib.md5(data)

# ok: insecure-md5-hash
h = hashlib.sha256(data)

Run the test suite:

semgrep scan --test --config ./rules/

ruleid: marks a line that must match (protects against false negatives if the rule regresses); ok: marks a line that must not match (protects against false positives). Validate the rule's YAML syntax itself separately with:

semgrep scan --validate --config ./rules/insecure-md5-hash.yaml

Practical CI Pipeline (Tool-Agnostic Pattern)

# Illustrative CI job — pure CLI, no Semgrep account required
semgrep:
  script:
    - pip install semgrep
    - semgrep scan --config auto --config ./rules/
        --baseline-commit=$(git merge-base HEAD origin/main)
        --sarif-output=semgrep.sarif
        --error
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'

This combines registry rules (auto) with your own custom rules (./rules/), gates only on findings introduced since the merge base (not pre-existing debt), emits SARIF for downstream tooling, and fails the build (--error) when something new is found — the same phased, diff-aware approach recommended generally in SAST and SCA.

Practical Tips

• Start narrow, expand deliberately. Begin with p/ci or a hand-picked set of custom rules, not p/security-audit plus every language pack at once — a flood of findings on day one is the fastest way to get the tool ignored.
• Write rules from real incidents. The highest-value custom rules encode a mistake your organisation has actually made before, not a hypothetical.
• Keep .semgrepignore current. Vendored code, generated code, and test fixtures are the most common source of ongoing false positives in an otherwise well-tuned setup.
• Prefer pattern-either over broad regex. Semgrep's structural matching understands code syntax (whitespace, argument order, equivalent literal forms); a metavariable-regex or pattern-regex fallback is useful but more brittle than a structural pattern where one is possible.
• Don't assume taint mode "just works" across a whole request lifecycle. Given the OSS engine's intraprocedural limitation above, verify a taint rule actually fires against your real code shape — test it, don't assume it.

Reference Checklist

References

• SAST and SCA
• Semgrep CLI Reference
• Semgrep Rule Syntax
• Semgrep Taint Analysis Overview
• Semgrep — Testing Rules
• Semgrep Community Rules Registry
• Semgrep Pro Engine (cross-file/cross-function analysis)