Is there a way to perform a single training step on a single query/response pair rather than using an entire dataset?
I'm trying to finetune a model "live" based on its own outputs and a rating from a user, for this I need a way to feed the trainer individual prompts, generations and ratings and have it perform a single training step.
Previously I was using TRL's PPOTrainer.step() function for this, but it appears that it's been [removed](https://github.com/huggingface/trl/issues/3270) in newer versions.
My current idea for a janky workaround is to literally create a new dataset with a single row and re-create a new trainer instance with it every single time, but this seems like it could easily cause problems. Is there a "real" way of doing this using unsloth? If not, how bad of an idea exactly is my janky workaround, and in what way?