Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add memory profiling and optimization to CLI #1003

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

codegen-sh[bot]
Copy link
Contributor

@codegen-sh codegen-sh bot commented Apr 2, 2025

This PR adds memory profiling capabilities to the Codegen CLI and implements memory optimizations.

New Features

Memory Profiling Command

  • Added a new memprof command that allows profiling memory usage of any CLI command
  • The profiler tracks RSS, VMS, and detailed memory allocations using tracemalloc
  • Generates visualizations and detailed reports of memory usage

Memory Optimizations

  • Added memory usage tracking and reporting to the run command
  • Implemented garbage collection at strategic points to reduce memory footprint
  • Added timing information to help identify performance bottlenecks
  • Improved memory cleanup after operations complete

Implementation Details

  • Created a new memory_profiler.py utility module with reusable profiling tools
  • Added a new memprof command that can profile any other CLI command
  • Modified run_local.py to include memory usage tracking and optimization
  • Updated the main CLI to include the new command

Usage

# Profile memory usage of a command
codegen memprof run my-codemod --arguments '{"param": "value"}'

# Output is saved to ~/.codegen/memory_profiles by default
# Or specify a custom output directory
codegen memprof --output-dir ./profiles run my-codemod

The memory profiling output includes:

  • JSON data of memory usage over time
  • PNG visualization of memory usage
  • Text report with peak memory usage and top memory allocations
  • Runtime performance metrics

These improvements will help identify memory leaks and optimize the CLI's memory usage.

@codegen-sh codegen-sh bot requested review from codegen-team and a team as code owners April 2, 2025 18:08
Copy link

codecov bot commented Apr 2, 2025

Codecov Report

Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/codegen/cli/cli.py 0.00% 2 Missing ⚠️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants