Skip to content

Can't reproduce some results of GrIPS paper. #3

Description

@YXShen19191

Hi Archiki Prasad,

I want to reproduce the results of GrIPS initializing with task-agnostic instructions with Instruction-Only prompts in your paper.
With the hyperparameters in the paper which is l = 1, m = 5, n = 10, and P = 2, I have achieved the accuracy of model InstructGPT babbage in table 3, which is 54.41. However, I cannot achieve the accuracy of model GPT-2 XL in table 3, which is 54.29. My result is about 53.00.
I have tried several train seeds and different hyperparameters, but still failed to achieve the accuracy 54.29.
I need your help to reproduce the result. Could you give me some suggestions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions