Hello, I am using the yolv8 for underwater dataset (detection) Implementing Fast Geometric Ensembling (FGE) with YOLOv8 - Cyclical Learning Rate Path

[Ishfaz]

I am working on implementing Fast Geometric Ensembling (FGE) with YOLOv8. :
from ultralytics import YOLO
import torch
import math

class FGEScheduler:
def init(self, optimizer, max_lr=1e-3, min_lr=1e-4, cycle_length=10):
self.optimizer = optimizer
self.max_lr = max_lr
self.min_lr = min_lr
self.cycle_length = cycle_length
self.current_step = 0

def step(self):
    # Calculate position in cycle (0 to 1)
    cycle_pos = (self.current_step % self.cycle_length) / self.cycle_length
    
    # Linear piecewise learning rate
    if cycle_pos < 0.5:
        # Linear increase
        lr = self.min_lr + (self.max_lr - self.min_lr) * (2 * cycle_pos)
    else:
        # Linear decrease
        lr = self.max_lr - (self.max_lr - self.min_lr) * (2 * (cycle_pos - 0.5))
    
    # Update learning rate
    for param_group in self.optimizer.param_groups:
        param_group['lr'] = lr
        
    self.current_step += 1
    return lr

class FGECallback:
def init(self, save_dir='runs/fge', cycle_length=10):
self.save_dir = save_dir
self.cycle_length = cycle_length
self.current_step = 0
self.models = []

def on_train_batch_end(self, trainer):
    self.current_step += 1
    
    # Save model at the end of each cycle
    if self.current_step % self.cycle_length == 0:
        model_path = f"{self.save_dir}/model_cycle_{len(self.models)}.pt"
        trainer.model.save(model_path)
        self.models.append(model_path)

def train_fge(model, data_yaml, epochs=100, cycle_length=10):
# Initialize base model
model = YOLO(model)

# Setup FGE scheduler and callback
fge_callback = FGECallback(cycle_length=cycle_length)

# Training with FGE
model.train(
    data=data_yaml,
    epochs=epochs,
    callbacks=[fge_callback]
)

return fge_callback.models 

Is this the correct way to implement a custom learning rate scheduler with YOLOv8? Should I use a different approach to modify the learning rate during training?
How can I properly integrate the FGE callback with YOLOv8's training loop to save model snapshots at the right cyclical intervals?
What's the best way to combine predictions from multiple snapshots for inference with the ensemble?

[glenn-jocher]

👋 Hello @Ishfaz, thank you for sharing your implementation and engaging with the Ultralytics community 🚀! Your use of Fast Geometric Ensembling (FGE) and custom learning rate schedulers with YOLOv8 is very interesting 🌊.

We recommend exploring our Docs for insights into custom training and callback usage. Below are some pointers that might be helpful:

1️⃣ Custom Training and Callbacks: If this is a ❓ custom training question, you may want to verify that your custom scheduler and callback are being properly recognized during the training loop. Please review the documentation on custom training logic for additional guidance.

2️⃣ Bug Report: If this is a 🐛 bug or you're encountering unexpected behavior, we kindly ask that you provide a minimum reproducible example (MRE). Having an MRE greatly helps us debug the issue more effectively.

3️⃣ Learning Rate Adjustment: To modify the learning rate dynamically during training, consider reviewing how YOLOv8 handles its built-in schedulers and callbacks. The base YOLO object may already provide mechanisms you can integrate with or extend.

Your example implementation seems like a good start. To clarify model saving intervals or other cyclical logic, you might find it helpful to review YOLOv8's callbacks or even its source code to ensure full compatibility.

---

Upgrade

Before diving deeper, ensure you have the latest version of the ultralytics package and dependencies correctly installed in a Python>=3.8 setup with PyTorch>=1.8:

pip install -U ultralytics

---

Suggested Environments

Here’s where you can try out your custom implementation in a verified environment with dependencies pre-installed. Just click and start coding:

• Notebooks:
• Google Cloud: See GCP Quickstart Guide
• AWS: See AWS Quickstart Guide
• Docker: See Docker Quickstart Guide

---

Community and Support

Join our active Ultralytics community 🎧 to get real-time help:

• Discord
• Discourse Forums
• Subreddit

An Ultralytics engineer will review this discussion soon and provide additional assistance. Stay tuned! 🚀