[Draft] Add reference-free SimPO loss

[marcusacosta]

Draft - design feedback requested

This PR adds native support for SimPO (Simple Preference Optimization) using
Torchtune's existing paired-preference DPO pipeline.

I am opening this as a draft because SimPO requires response-token-average log
probabilities, while existing DPO and RSO losses consume summed response log
probabilities. The PR preserves existing DPO/RSO behavior and requests review
of the loss-level aggregation contract.

Feedback is especially welcome on:

1. Whether PreferenceLoss.return_average_logprobs is the preferred interface
   for losses that require averaged rather than summed sequence log
   probabilities.
2. Whether SimPO should live in the stable torchtune.rlhf.loss API or under
   torchtune.dev.
3. Whether the dedicated single-device LoRA SimPO config belongs in this PR.

Summary

• Add reference-free SimPOLoss(beta=2.0, gamma_beta_ratio=0.25).
• Add a backward-compatible PreferenceLoss.return_average_logprobs property.
• Preserve summed log-probability behavior for DPO and RSO.
• Use masked average response-token log probabilities for SimPO.
• Add llama3_1/8B_lora_simpo_single_device.
• Add unit tests for the SimPO objective, reference-free inputs, gradients,
  and summed-versus-averaged log-probability aggregation.
• Add a one-step GPU integration test using the registered SimPO config.

SimPO compatibility

SimPO is a reference-free paired-preference objective that uses the average
log probability over unmasked response tokens. Existing Torchtune DPO recipes
currently provide summed sequence log probabilities to preference losses.

Torchtune already supports the required masked averaging through:

get_batch_log_probs(..., return_average_logprobs=True)

This PR exposes that selection through the loss contract while retaining
existing summed-log-probability behavior for DPO and RSO.

Scope

• Reuses existing paired preference datasets and collator.
• Does not change DPO or RSO loss behavior.
• Does not add SimPO's optional SFT term, hinge mode, or label smoothing.
• Does not change full-DPO reference-model setup; SimPO skips the reference
  forward through the existing reference-free path.
• Adds no benchmark, convergence, memory, or performance claims.

Testing

python3 -m pytest \
  tests/torchtune/rlhf/loss/test_dpo_loss.py \
  tests/torchtune/rlhf/test_sequence_processing.py \
  -vv
# 8 passed

python3 -m pytest \
  tests/recipes/test_lora_dpo_single_device.py \
  --collect-only -q
# SimPO integration test collected with integration_test and 1-GPU markers

git diff --check HEAD

The new GPU/S3-backed integration test was not run locally because a GPU and
the required test checkpoint artifacts are unavailable. It is collected with
the existing integration and one-GPU markers for the applicable upstream CI
workflow.

References

• SimPO paper: arXiv:2405.14734
• Official implementation: princeton-nlp/SimPO