Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace lockfile with dependency source in targets #362

Merged
merged 5 commits into from
Mar 11, 2025

Conversation

mmcqd
Copy link
Member

@mmcqd mmcqd commented Mar 11, 2025

  • I ran make setup && make to update the generated code after editing a .atd file (TODO: have a CI check)
  • I made sure we're still backward compatible with old versions of the CLI.
    For example, the Semgrep backend need to still be able to consume data
    generated by Semgrep 1.50.0.
    See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades
    Note that the types related to the semgrep-core JSON output or the
    semgrep-core RPC do not need to be backward compatible!

@mmcqd
Copy link
Member Author

mmcqd commented Mar 11, 2025

I'm confused by the proto conflict, not sure what's causing that

@mmcqd mmcqd requested a review from aryx March 11, 2025 05:36
@aryx
Copy link
Collaborator

aryx commented Mar 11, 2025

just pull back the latest main, rebase, and rerun make to regenerate everything

@aryx aryx requested a review from bkettle March 11, 2025 07:12
@@ -2023,7 +2023,7 @@ type analyzer <ocaml attr="deriving show"> = string wrap <ocaml module="Analyzer
*)
type target <ocaml attr="deriving show"> = [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also change the targets type and have there a list of code_target and a list of dependency_source.
But let's go with this approach.
Note that @mjambon added also a new Scanning_roots cases in the targets type where pysemgrep relies on
osemgrep file targeting to compute the targets, but I think we're fine with your change; just in the futur for those
cases where pass the Scanning_roots, it will also be the job of the OCaml code to compute the dependency_source (using @bkettle subproject matcher probably).

mmcqd added 4 commits March 11, 2025 14:57
In semgrep/semgrep-proprietary#2872, we updated
the SCA subproject logic to scan only changed subprojects in diff scans.
Subprojects that remained unchanged were intentionally left unresolved
with the reason UnresolvedSkipped.

Previously, we didn't include this information in our subproject stats,
leaving us without visibility into why a subproject was unresolved. This
PR introduces two optional fields, unresolved_reason and errors,
allowing us to capture more granular stats and better understand
subproject resolution failures.

- [x] I ran `make setup && make` to update the generated code after
editing a `.atd` file (TODO: have a CI check)
- [x] I made sure we're still backward compatible with old versions of
the CLI.
For example, the Semgrep backend need to still be able to *consume* data
	  generated by Semgrep 1.50.0.
See
https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades
	  Note that the types related to the semgrep-core JSON output or the
	  semgrep-core RPC do not need to be backward compatible!
To make things more concrete about the interface we want
with the backend.

test plan:
make

- [x] I ran `make setup && make` to update the generated code after
editing a `.atd` file (TODO: have a CI check)
- [x] I made sure we're still backward compatible with old versions of
the CLI.
For example, the Semgrep backend need to still be able to *consume* data
	  generated by Semgrep 1.50.0.
See
https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades
	  Note that the types related to the semgrep-core JSON output or the
	  semgrep-core RPC do not need to be backward compatible!
Copy link

Backwards compatibility summary:

Checking backward compatibility of semgrep_output_v1.atd against past version v1.100.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.101.0
Skipping v1.102.0 because commit 1c82453e89e0b569630e48ddde015e201df0e5f9 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.103.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.104.0
Skipping v1.106.0 because commit 5e0c767ec323f3f2356d3bf8dbdf7c7836497d8a has already been checked
Skipping v1.107.0 because commit 5e0c767ec323f3f2356d3bf8dbdf7c7836497d8a has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.108.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.109.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.110.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.111.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.75.0
Skipping v1.76.0 because commit 9102031608aa4154e1c37f557550ec4eabc8780c has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.77.0
Skipping v1.78.0 because commit dcb5d77b420ddee61f58aadd3c2c7aef38778154 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.79.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.80.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.81.0
Skipping v1.82.0 because commit 9e0f3bec26b07b4fb6753a32cb75277f45f2572c has already been checked
Skipping v1.83.0 because commit 9e0f3bec26b07b4fb6753a32cb75277f45f2572c has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.84.0
Skipping v1.84.1 because commit 3daef49297ada205359cc1d2996354c94b628b0d has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.85.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.86.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.87.0
Skipping v1.88.0 because commit 512c0bd97db59c48a5705b2741662a338776e438 has already been checked
Skipping v1.89.0 because commit 512c0bd97db59c48a5705b2741662a338776e438 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.90.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.91.0
Skipping v1.92.0 because commit 2351c5e528cb7430422208dc66707894c066b508 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.93.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.94.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.95.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.96.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.97.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.98.0
Skipping v1.99.0 because commit 60809032a2e39742f42910d46b3e5dd305b8b8cf has already been checked

@mmcqd mmcqd force-pushed the matthew/dependency-source-target branch from 993f80d to 6e9ad16 Compare March 11, 2025 22:08
@mmcqd mmcqd merged commit 6edf9cf into main Mar 11, 2025
3 checks passed
@mmcqd mmcqd deleted the matthew/dependency-source-target branch March 11, 2025 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants