Release v1.81.0

1.81.0 - 2024-07-24

Changed

• The --debug option will now display logging information from the semgrep-core
  binary directly, without waiting that the semgrep-core program finish. (incremental_debug)

Fixed

• C++: Scanning a project with header files (.h) now no longer causes a
  spurious warnings that the file is being skipped, or not analyzed. (code-6899)

• Semgrep will now be more strict (as it should be) when unifying identifiers.

  Patterns like the one below may not longer work, particularly in Semgrep Pro:

  patterns:
    - pattern-inside: |
        class A:
          ...
          def $F(...):
            ...
          ...
        ...
    - pattern-inside: |
        class B:
          ...
          def $F(...):
            ...
          ...
        ...
  

  Even if two classes A and B may both have a method named foo, these methods
  are not the same, and their ids are not unifiable via $F. The right way of doing
  this in Semgrep is the following:

  patterns:
    - pattern-inside: |
        class A:
          ...
          def $F1(...):
            ...
          ...
        ...
    - pattern-inside: |
        class B:
          ...
          def $F2(...):
            ...
          ...
        ...
    - metavariable-comparison:
        comparison: str($F1) == str($F2)
  

  We use a different metavariable to match each method, then we check whether they
  have the same name (i.e., same string). (code-7336)

• In the app, you can configure Secrets ignores separately from Code/SSC ignores. However, the
  files that were ignored by Code/SSC and not Secrets were still being scanned during the
  preprocessing stage for interfile analysis. This caused significantly longer scan times than
  expected for some users, since those ignored files can ignore library code. This PR fixes that
  behavior and makes Code/SSC ignores apply as expected. (saf-1087)

• Fixed typo that prevented users from using "--junit-xml-output" flag and added a tests that invokes the flag. (saf-1437)

Release v1.80.0

1.80.0 - 2024-07-18

Added

• OSemgrep now can take --exclude-minified-files to skip minified files. Additionally --no-exclude-minified-files will disable this option. It is off by default. (cdx-460)

• Users are now required to login before using semgrep scan --pro.

  Previously, semgrep will tell the users to log in, but the scan will still continue.

  With this change, semgrep will tell the users to log in and stop the scan. (saf-1137)

Fixed

• The language server no longer scans large or minified files (cdx-460)

• Pro: Improved module resolution for Python. Imports like from a.b import c where
  c is a module will now be resolved by Semgrep. And, if a module cannot be found
  in the search path, Semgrep will try to heuristically resolve the module by matching
  the module specifier against the files that are being scanned. (code-7069)

• A scan can occasionally freeze when using tracing with multiprocesses.

  This change disables tracing when scanning each target file unless the scan runs in a single process. (saf-1143)

• Improved error handling for rules with invalid patterns. Now, scans will still complete and findings from other rules will be reported. (saf-789)

• The "package-lock.json" parser incorrectly assumed that all paths in the "packages" component of "package-lock.json" started with "node_modules/".

  In reality, a dependency can be installed anywhere, so the parser was made more flexible to recognize alternative locations ("node_modules", "lib", etc). (sc-1576)

Release v1.79.0

1.79.0 - 2024-07-10

Added

• Preliminary support for the Move on Aptos language
  (see https://aptos.dev/move/move-on-aptos for more info on this language).
  Thanks a lot to Zhiping Liao (ArArgon) and Andrea Cappa for their contributions! (move_on_aptos)
• The language server now reports number of autofixes and ignores triggered throught IDE integrations when metrics are enabled (pdx-autofix-ignore)
• Added support for comparing Golang Pseudo-versions. After replacing calls to the
  packaging module with some custom logic, Pseudo-versions can now be compared against
  strict core versions and other pseudo versions accurately. (sc-1601)
• We now perform a git gc as a side-effect of historical scans. (scrt-630)

Fixed

• tainting: Fixed bug in --pro-intrafile that caused Semgrep to confuse a parameter
  with a top-level function with no arguments that happened to have the same name:

  def foo
    taint
  end
  
  def bar(foo)
    sink(foo) # no more FP here
  end (code-6923)
  

• Fixed fatal errors on files containing nosemgrep annotation without
  any rule ID after. (nosemgrep_exn)

• Matching explanations: Focus nodes now appear after filter nodes, which is
  the correct order of execution of pattern nodes. Filter nodes are now
  unreversed. (saf-1127)

• Autofix: Previews in the textual CLI output will now join differing lines
  with a space, rather than joining with no whitespace whatsoever. (saf-1135)

• Secrets: resolved some rare instances where historical scans would skip blobs
  depending on the structure of the local copy of the repository (i.e., blobs
  were only skipped if the specific copy of the git store had a certain
  structure). (scrt-630)

Release v1.78.0

1.78.0 - 2024-06-27

Added

• Matching of fully qualified type names in the metavariable-type operator has
  been improved. For example:

  from a.b import C
  
  x = C()
  

  The type of x will match both a.b.C and C.

    - pattern: $X = $Y()
    - metavariable-type:
        metavariable: $X
        types:
          - a.b.C  # or C
  ``` (code-7269)
  
  
  

Fixed

• Symbolic propagation now works on decorator functions, for example:

  x = foo
  @x() # this is now matched by pattern `@foo()`
  def test():
    pass (code-6634)
  

• Fixed an issue where Python functions with annotations ending in endpoint,
  route, get, patch, post, put, delete, before_request or
  after_request (i.e., ones we associate with Flask) were incorrectly analyzed
  with the Code product in addition to the Secrets product when present in a file
  being ignored for Code analysis but included for Secrets. (scrt-609)

Release v1.77.0

1.77.0 - 2024-06-24

Added

• Semgrep will now report the id of the organization associated with logged in users when reporting metrics in the language server (cdx-508)

• Pro: taint-mode: Improved index-sensitive taint tracking for tuple/list (un)packing.

  Example 1:

   def foo():
       return ("ok", taint)
  
   def test():
        x, y = foo()
        sink(x)  # nothing, no FP
        sink(y)  # finding
  

  Example 2:

   def foo(t):
        (x, y) = t
        sink(x)  # nothing, no FP
        sink(y)  # finding
  
   def test():
        foo(("ok", taint)) (code-6935)
  

• Adds traces to help debug the performance of tainting. To send the traces added in the PR, pass
  --trace and also set the environment variable SEMGREP_TRACE_LEVEL=trace. To send them to a
  local endpoint instead of our default endpoint, use --trace-endpoint. (saf-1100)

Fixed

• Fixed a bug in the generation of the control-flow graph for try statements that
  could e.g. cause taint to report false positives:

  def test():
      data = taint
      try:
          # Semgrep assumes that `clean` could raise an exception, but
          # even if it does, the tainted `data` will never reach the sink !
          data = clean(data)
      except Exception:
          raise Exception()
  
      # `data` must be clean here
      sink(data) # no more FP (flow-78)
  

• The language server (and semgrep --experimental) should not report anymore errors from
  the metrics.semgrep.dev server such as "cannot read property 'map' of undefined". (metrics_error)

• Fixed a bug in the gemfile.lock parser which causes Semgrep to miss direct
  dependencies whose package name does not end in a version constraint. (sc-1568)

Release v1.76.0

1.76.0 - 2024-06-17

Added

• Added type inference support for basic operators in the Pro engine, including
  +, -, *, /, >, >=, <=, <, ==, !=, and not. For numeric
  computation operators such as + and -, if the left-hand side and right-hand
  side types are equal, the return type is assumed to be the same. Additionally,
  comparison operators like > and ==, as well as the negation operator not,
  are assumed to return a boolean type. (code-6940)

• Added guidance for resolving token issues for install-semgrep-pro in non-interactive environments. (gh-1668)

• Adds support for a new flag, --subdir <path>, for semgrep ci, which allows users to pass a
  subdirectory to scan instead of the entire directory. The path should be a relative path, and
  the directory where semgrep ci is run should be the root of the repository being scanned.
  Unless SEMGREP_REPO_DISPLAY_NAME is explicitly set, passing the subdirectory
  will cause the results to go to a project specific to that subdirectory.

  The intended use case for semgrep ci --subdir path/to/dir is to help users with very large
  repos scan the repo in parts. (saf-1056)

Fixed

• Language Server will now send error messages properly, and error handling is greatly improved (cdx-502)

• Pro: Calling a safe method on a tainted object should no longer propagate taint.

  Example:

  class A {
      String foo(String str) {
          return "ok";
      }
  }
  
  class Test {
      public static void test() {
          A a;
          String s;
          a = taint();
          // Despite `a` is tainted, `a.foo()` is entirely safe !!!
          s = a.foo("bar");
          sink(s); // No more FP here
      }
  } (code-6935)
  

• Fixing errors in matching identifiers from wildcard imports. For example, this
  update addresses the issue where the following top-level assignment:

  from pony.orm import *
  db = Database()
  

  is not matched with the following pattern:

  $DB = pony.orm.Database(...)
  ``` (code-7045)
  

• [Pro Interfile JS/TS] Improve taint propagation through callbacks passed to $X.map functions and similar. Previously, such callbacks needed to have a return value for taint to be properly tracked. After this fix, they do not. (js-taint)

• Rust: Constructors will now properly match to only other constructors with
  the same names, in patterns. (saf-1099)

Release v1.75.0

1.75.0 - 2024-06-03

Added

• Pro: Semgrep can now track taint through tuple/list (un)packing intra-procedurally
  (i.e., within a single function). For example:

  t = ["ok", "taint"]
  x, y = t
  sink(x) # OK, no finding
  sink(y) # tainted, finding
  ``` (code-6935)

• Optional type matching is supported in the Pro engine for Python. For example,
  in Python, Optional[str], str | None, and Union[str, None] represent the
  same type but in different type expressions. The optional type match support
  enables matching between these expressions, allowing any optional type
  expression to match any other optional type expression when used with
  metavariable-type filtering. It's important to note that syntactic pattern
  matching still distinguishes between these types. (code-6939)

• Add support for pnpm v9 (pnpm)

• Added a new rule option decorators_order_matters, which allows users to make decorators/ non-keyword attributes matching stricter. The default matching for attributes is order-agnostic, but if this rule option is set to true, non-keyword attributes (e.g. decorators in Python) will be matched in order, while keyword attributes (e.g. static, inline, etc) are not affected.

  An example usage will be a rule to detect any decorator that is outside of the route() decorator in Flask, since any decorator outside of the route() decorator takes no effect.

  bad: another.func() takes no effect

  @another.func("func")
  @app.route("route")
  def f():
  pass

  ok: route() is the outermost decorator

  @app.route("route")
  @another.func("func")
  def f():
  pass (saf-435)

Fixed

• Pro: taint-mode: Fixed issue causing findings to be missed (false negatives)
  when a global or class field was tainted, and then used in a sink after two
  or more function calls.

  For example:

  class Test {
      string bad;
  
      void test() {
          bad = "taint";
          foo();
      }
  
      void foo() {
          bar();
      }
  
      void bar() {
          sink(bad); // finding no longer missed
      }
  } (saf-1059)
  

• [Mostly applicable to Pro Engine] Typed metavariables will now match against the inferred type of a binding even if a constant is propagated for that binding, if we are unable to infer a type from the constant. Previously, we would simply fail to match in this case. (saf-1060)

• Removed the URLs at the end of the log when semgrep ci --dryrun is ran because dry run doesn't interact with the app so the URLs don't make sense. (saf-924)

Release v1.74.0

1.74.0 - 2024-05-23

Fixed

• One part of interfile tainting was missing a constant propagation phase, which causes semgrep to miss some true positives in some cases during interfile analysis.

  This fix adds the missing constant propagation. (saf-1032)

• Semgrep now matches YAML tags (e.g. !number in !number 42) correctly rather
  than ignoring them. (saf-1046)

• Upgraded Semgrep's Dockerfile parser. This brings in various
  fixes from
  tree-sitter-dockerfile
  including minimal support for heredoc templates, support for variables in keys
  of LABEL instructions, support for multiple parameters for ADD and COPY
  instructions, tolerance for blanks after the backslash of a line continuation.
  As a result of supporting variables in LABEL keys, the multiple key/value
  pairs found in LABEL instructions are now treated as if they each had they own
  LABEL instruction. It allows a pattern LABEL a=b to match LABEL a=b c=d
  without the need for an ellipsis (LABEL a=b ...). Another consequence is
  that the pattern LABEL a=b c=d can no longer match LABEL c=d a=b but it
  will match a LABEL a=b instruction immediately followed by a separate
  LABEL c=d. (upgrade-dockerfile-parser)

Release v1.73.0

1.73.0 - 2024-05-16

Added

• Added new AWS validator syntax for Secrets (scrt-278)

Fixed

• Fix couldn't find metavar $MT in the match results error, which may occur
  when we capture FQN with the metavariable and use metavariable-type filter on
  it. (code-7042)
• Fixes the crash (during scan) caused by improper handling of unicode characters present in the source code. (gh-8421)
• [Pro Engine Only] Tainted values are now tracked through instantiation of React functional components via JSX. (jsx-taint)

Release v1.72.0

1.72.0 - 2024-05-08

Fixed

• Dockerfile support: Avoid a silent parsing error that was possibly accompanied
  with a segfault when parsing Dockerfiles that lack a trailing newline
  character. (gh-10084)

• Fixed bug that was preventing the use of metavariable-pattern with
  the aliengrep engine of the generic mode. (gh-10222)

• Added support for function declarations on object literals in the dataflow analysis.

  For example, previously taint rules would not have matched the
  following javascript code but now would.

  let tainted = source()
  let o = {
      someFuncDecl(x) {
          sink(tainted)
      }
  }
  ``` (saf-1001)
  

• Osemgrep only:

  When rules have metavariable-type, they don't show up in the SARIF output. This change fixes that.

  Also right now dataflow traces are always shown in SARIF even when --dataflow-traces is not passed. This change also fixes that. (saf-1020)

• Fixed bug in rule parsing preventing patternless SCA rules from being validated. (saf-1030)