Submit evaluation results提交测评结果

Follow the steps below to submit through GitHub. Original evidence remains private; only approved result summaries appear on the community leaderboard.按照以下步骤通过 GitHub 提交。原始证据保持私有,社区榜单只公开审核通过的结果摘要。

Haven’t run the benchmark? Start with the evaluation guide ↗.还没有运行测评?先查看测评指南 ↗。

Participant submission参评者提交

  1. Prepare your result files准备结果文件

    After running the benchmark, follow the packaging instructions to create two files: submission.json (result information) and evidence.zip (scores, original traces, run manifest, audit records and receipt). Packaging runs locally and does not upload files.完成测评后,按照打包说明生成两个文件:submission.json(结果信息)和 evidence.zip(分数、原始轨迹、运行清单、审计记录与回执)。打包在本地完成,不会上传文件。

  2. Create your private submission repository创建自己的私有提交仓库

    Sign in to GitHub, use the button below and choose Private. This repository holds your result files. Do not upload evidence to the public benchmark or template repository.登录 GitHub,点击下方按钮并选择 Private。这个仓库用于存放你的结果文件。不要向公开项目或模板仓库上传原始证据。

  3. Authorize review access授权审核访问

    Authorize the review app (GitHub App) to access only your submission repository, then invite @pope8zdt as a collaborator under that repository’s Settings → Collaborators. The app reads your files and PR and posts validation feedback; the maintainer reviews the evidence. Nothing is installed on your computer, and you do not need to configure secrets or GitHub Actions workflows.将审核应用(GitHub App)仅授权给自己的提交仓库,再进入该仓库的 Settings → Collaborators,邀请 @pope8zdt 为协作者。应用读取文件和 PR 并写回校验反馈,维护者负责审核证据。无需在电脑上安装软件,也不需要配置密钥或 GitHub Actions 工作流。

  4. Open a pull request (PR)发起拉取请求(PR)

    In your private repository, create a new branch and add both files at the repository root. Open a non-draft PR from that branch to main in the same private repository. This PR is your submission; the review workflow finds it automatically. No separate form or direct file transfer to the maintainer is needed.在自己的私有仓库中新建分支,将两个文件放在仓库根目录,再从该分支向同一个私有仓库的 main 发起非草稿 PR。这个 PR 就是你提交的结果,审核工作流会自动发现,无需另填表单或向维护者私发文件。

  5. View the review decision查看审核结果

    Validation feedback and rejection reasons appear in your private PR. Push corrections to the same branch; every new commit needs fresh validation and approval. Approved result summaries sync to the community leaderboard every four hours, with possible GitHub scheduling and caching delays. Original evidence remains private after approval.校验反馈与拒绝原因在自己的私有 PR 中查看。需要修改时推送到同一分支,每个新提交都需重新校验和批准。审核通过的结果摘要每四小时同步到社区榜单,GitHub 调度和缓存可能额外延迟。通过后原始证据仍保持私有。

Check your leaderboard result查看上榜结果

After maintainer approval, the next successful scheduled sync publishes your result summary. Sync runs every four hours; GitHub scheduling and page caching can cause delays. You do not need to merge the PR or run any commands.维护者审核通过后,下一次成功的定时同步会自动公开结果摘要。同步每四小时运行一次,GitHub 调度和页面缓存可能造成延迟。你无需合并 PR,也无需执行命令。

  1. Read the receipt on your PR查看自己 PR 中的回执

    Open your submission repository → Pull requests → your PR → Conversation. Read the maintainer’s review and the latest Async-RBench receipt comment. Passing validation is not approval. Status: published means the summary has been written to the public leaderboard data, though the page may still be cached.进入自己的提交仓库 → Pull requests → 本次提交的 PR → Conversation,查看维护者审核意见和最新的 Async-RBench receipt 回执。校验通过不等于审核通过;回执显示 Status: published,才表示摘要已写入公开榜单数据,页面仍可能有缓存延迟。

  2. Find your entry on the community leaderboard在社区榜单找到自己的结果

    Open Leaderboard → Community results. Find your GitHub username and agent/model, then compare the DRS, task success rate and tasks × repetitions with your submitted results. The matching entry confirms that your result is visible on the website; it will not appear in Paper results.打开榜单 → 社区结果,找到自己的 GitHub 用户名和智能体/模型,核对 DRS、任务成功率及任务数 × 重复次数是否与提交结果一致。对应条目出现且信息一致,即可确认结果已在网站展示;它不会出现在“论文结果”中。

  3. If your result is missing审核通过后仍未显示怎么办

    Allow time for the next sync, then refresh the leaderboard. If your result is still missing, leave a comment on your original private PR asking the maintainer to check. Do not create a duplicate PR or make the evidence public. Any new commit requires fresh validation and approval.先等待下一轮同步,再刷新榜单。若仍未显示,在原私有 PR 中留言请维护者检查,无需重复提交 PR,也不要公开原始证据。如果提交了新版本,需要重新校验和审核。

What becomes public哪些内容会公开

Only the approved summary: agent and model names, organization, code revision, scores, coverage, submitter and reviewer usernames, review time and scoring-policy version.仅公开审核通过的摘要:智能体与模型名称、机构、代码版本、分数、覆盖范围、提交者及审核者用户名、审核时间和评分策略版本。

Original evidence, private repository links and rejection reasons stay private, including after approval. Access is limited to the owner, invited collaborators and the authorized review service. Keep your repository private and never include credentials in evidence. GitHub may send notifications according to your account settings.原始证据、私有仓库链接和拒绝原因在批准后也不公开,仅仓库所有者、受邀协作者及已授权审核服务可访问。请始终保持仓库私有,证据中不得包含密钥。GitHub 可能按你的账号设置发送通知。

Community results are stored and displayed separately. They never modify the paper leaderboard.社区结果独立存储与展示,不修改论文榜单。