[Feature]: pass-through parameter from request to model.forward (already implemented)

[qpwo]

🚀 The feature, motivation and pitch

It took me ages to figure out how to pass a custom parameter from the API through to the model.forward call. This is very useful for running little experiments and making custom logs.

Should I dress it up in a PR?

It would be disabled by default. I guess you'd do aphrodite run --enable-passthrough-param or similar.

Example

## example request:
client.completions.create(
    model='70b',
    prompt='I was',
    extra_body={"passthrough": {"save_activations": True}},
)

## example modification of LlamaForCausalLM.forward:
def forward(
    self,
    input_ids: torch.Tensor,
    positions: torch.Tensor,
    kv_caches: List[torch.Tensor],
    attn_metadata: AttentionMetadata,
    intermediate_tensors: Optional[IntermediateTensors] = None,
    passthrough: Optional[Dict] = None,
) -> Union[torch.Tensor, IntermediateTensors]:
    passthrough = passthrough or {}
    save_activations = bool(passthrough.get("save_activations", False)) # example 1
    scale_gate_up = float(passthrough.get('scale_gate_up', 1.0)) # example 2
    model_output = self.model(
        input_ids, positions, kv_caches, attn_metadata, intermediate_tensors,
        save_activations=save_activations,
        scale_gate_up=scale_gate_up,
    )
    return model_output

Alternatives

I could not find any alternatives but would be quite happy to learn of any.

Additional context

PR is mostly written already, I would just tidy it if you want this feature.

[AlpinDale]

Doesn't seem particularly difficult to maintain, so go ahead!