Hi! Thanks for your great work on swe-bench-pro! I have some questions regarding the dataset construction process, and I'd really appreciate it if I could get some clarification.
- Does this repository include any scripts or code for building the dataset (e.g., crawling, cleaning, filtering, generating task instances)? If so, where can they be found?
- What naming convention is used for the
instance_id? For example, given an ID like:
instance_qutebrowser__qutebrowser-c580ebf0801e5a3ecabc54f327498bb753c6d5f2-v2ef375ac784985212b1805e1d0431dc8f1b3c171
does it represent <repo_owner>__<repo_name>-<initial_commit>-<another_commit>? Could you explain the pattern?
Thank you very much for your help!
Hi! Thanks for your great work on swe-bench-pro! I have some questions regarding the dataset construction process, and I'd really appreciate it if I could get some clarification.
instance_id? For example, given an ID like:instance_qutebrowser__qutebrowser-c580ebf0801e5a3ecabc54f327498bb753c6d5f2-v2ef375ac784985212b1805e1d0431dc8f1b3c171does it represent
<repo_owner>__<repo_name>-<initial_commit>-<another_commit>? Could you explain the pattern?Thank you very much for your help!