Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide common infrastructure for writing Sway code analyzers and generators #6836

Open
ironcev opened this issue Jan 16, 2025 · 0 comments
Open

Comments

@ironcev
Copy link
Member

ironcev commented Jan 16, 2025

A need for common infrastructure for writing Sway code analyzers and generators, or in general, tools built on top of the Sway compiler, was first pointed out by @tritao in this comment.

Description

Static analyzers and code generators built on top of Sway can have some, or all of these common needs:

  • finding elements in the lexed or typed trees.
  • navigating between the lexed and typed trees. E.g., finding a trait impl in the lexed tree and asking for the typed declaration of the trait.
  • navigating between related elements. E.g., finding usages of a symbol.
  • modifying lexed or typed trees.
  • generating source code out of lexed or typed trees.

A typical example of a tool having all these needs would be the Sway equivalent of Rust's Clippy.

Current Examples

sway-analyzer implements higher level abstractions, Context structs, that wrap sway_ast types and decorates them with additional contextual information, like, e.g., access to the module in which they are defined. Finding elements of interest is done via combination of the Visitor pattern and numerous helper functions found in the util module. The analyzer does not modify code.

Charcoal, a Solidity-to-Sway transpiler, implements a higher level abstraction that represents parts of the Sway AST. While transpiling, it creates instances of those higher level abstractions based on Solidity code. The resulting tree is in the end formatted to text in what resembles a simplified version of the swayfmt.

forc-migrate provides a simple abstraction for matching parts of lexed or typed trees, locating equivalent elements across the trees, as well as an abstraction for modifying lexed trees. All the abstractions operate directly on sway_ast types. It generates code out of modified lexed trees by using swayfmt. The major constraint in implementing the API for matching, locating, and modifying was invested effort. The goal was to come up with a simple yet extensible API, that will simplify writing code migrations.

Prior Art

  • ast-grep - a polyglot tool for code structural search, lint, and rewriting.
  • Clang LibTooling - a library to support writing standalone tools based on Clang.
  • Clang Transformer - a framework for writing C++ diagnostics and program transformations.
  • Clang LibASTMatchers - a domain-specific language to create predicates on Clang's AST.
  • Clippy Syntax Tree Patterns - a domain-specific language to describe Clippy lints using syntax tree patterns.
  • Roslyn - C# and Visual Basic compilers with an API surface for building code analysis tools.
@ironcev ironcev mentioned this issue Jan 17, 2025
8 tasks
IGI-111 pushed a commit that referenced this issue Jan 20, 2025
## Description

This PR introduces `forc-migrate`, a Forc tool for migrating Sway
projects to the next breaking change version of Sway.

The tool addresses two points crucial for code updates caused by
breaking changes:
- it informs developers about the breaking changes and **assists in
planing and executing** the migration.
- it **automatically changes source code** where possible, reducing the
manual effort needed for code changes.

Besides adding the `forc-migrate` tool, the PR:
- extends `Diagnostic` to support migration diagnostics aside with
errors and warnings.
- changes `swayfmt` to support generating source code from arbitrary
lexed trees. The change is a minimal one done only in the parts of
`swayfmt` that are executed by migration steps written in this PR.
Adapting `swayfmt` to fully support arbitrary lexed trees will be done
in #6779.

The migration for the `references` feature, migrating `ref mut` to
`&mut`, is developed only partially, to demonstrate the development and
usage of automatic migrations that alter the original source code.

The intended usage of the tool is documented in detail in the "forc
migrate" chapter of The Sway Book: _Forc reference > Plugins >
forc_migrate_. (The generated documentation has issues that are caused
by the documentation generation bug explained in #6792. These issues
will be fixed in a separate PR that will fix it for all the plugins.)

We expect the `forc-migrate` to evolve based on the developer's
feedback. Some of the possible extensions of the tool are:
- adding additional CLI options, e.g., for executing only specific
migration steps, or ignoring them.
- passing parameters to migration steps from the CLI.
- not allowing updates by default, if the repository contains modified
or untracked files.
- migrating workspaces.
- migrating other artifacts, e.g., Forc.toml files or contract IDs.
- migrating between arbitrary versions of Sway.
- migrating SDK code.
- etc.
 
`forc-migrate` also showed a clear need for better infrastructure for
writing static analyzers and transforming Sway code. The approach used
in the implementation of this PR should be seen as a pragmatic
beginning, based on the reuse of what we currently have. Some future
options are discussed in #6836.

## Demo

### `forc migrate show`

Shows the breaking change features and related migration steps. This
command can be run anywhere and does not require a Sway project.

```
Breaking change features:
  - storage_domains    (#6701)
  - references         (#5063)

Migration steps (1 manual and 1 semiautomatic):
storage_domains
  [M] Review explicitly defined slot keys in storage declarations (`in` keywords)

references
  [S] Replace `ref mut` function parameters with `&mut`

Experimental feature flags:
- for Forc.toml:  experimental = { storage_domains = true, references = true }
- for CLI:        --experimental storage_domains,references
```

### `forc migrate check`

Performs a dry-run of the migration on a concrete Sway project. It
outputs all the occurrences in code that need to be reviewed or changed,
as well as the migration time effort:

```
info: [storage_domains] Review explicitly defined slot keys in storage declarations (`in` keywords)
  --> /home/kebradalaonda/Desktop/M Forc migrate tool/src/main.sw:19:10
   |
...
19 |     y in b256::zero(): u64 = 0,
   |          ------------
20 |     z: u64 = 0,
21 |     a in calculate_slot_address(): u64 = 0,
   |          ------------------------
22 |     b in 0x0102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f20: u64 = 0,
   |          ------------------------------------------------------------------
   |
   = help: If the slot keys used in `in` keywords represent keys generated for `storage` fields
   = help: by the Sway compiler, those keys might need to be recalculated.
   = help:  
   = help: The previous formula for calculating storage field keys was: `sha256("storage.<field name>")`.
   = help: The new formula is:                                          `sha256((0u8, "storage.<field name>"))`.
   = help:  
   = help: For a detailed migration guide see: #6701
____

Migration effort:

storage_domains
  [M] Review explicitly defined slot keys in storage declarations (`in` keywords)
      Occurrences:     3    Migration effort (hh::mm): ~00:06

references
  [S] Replace `ref mut` function parameters with `&mut`
      Occurrences:     0    Migration effort (hh::mm): ~00:00

Total migration effort (hh::mm): ~00:06
``` 

### `forc migrate run`

Runs the migration steps and guides developers through the migration
process.

## Checklist

- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant