← Explore Harbor-Index
bixbench
bix-filter-chip-variants
18 trials · 38.9% solve rate · task definition on Harbor Hub ↗
TP 7TN 11
Instruction
Answer the following question.
What is the average number of CHIP variants per sample after filtering out intronic, intergenic, and UTR variants?
Analyze the data files in /workspace/.
Write your final answer — just the number — to /workspace/answer.txt. For example, if your answer is 50:
echo '50' > /workspace/answer.txt
You have up to 20 minutes (1200 seconds) to complete this task. Do not cheat by using online solutions or hints specific to this task.
Trials
| model | harness | outcome | reward | |
|---|---|---|---|---|
| DeepSeek V4terminus-2 | terminus-2 | TP | 1.00 | view →view trajectory → |
| GLM 5.2terminus-2 | terminus-2 | TP | 1.00 | view →view trajectory → |
| GPT-5.5codex | codex | TP | 1.00 | view →view trajectory → |
| GPT-5.5terminus-2 | terminus-2 | TP | 1.00 | view →view trajectory → |
| Kimi K2.6claude-code | claude-code | TP | 1.00 | view →view trajectory → |
| Kimi K2.6terminus-2 | terminus-2 | TP | 1.00 | view →view trajectory → |
| MiMo V2.5terminus-2 | terminus-2 | TP | 1.00 | view →view trajectory → |
| DeepSeek V4claude-code | claude-code | TN | 0.00 | view →view trajectory → |
| Gemini 3.1gemini-cli | gemini-cli | TN | 0.00 | view →view trajectory → |
| Gemini 3.1terminus-2 | terminus-2 | TN | 0.00 | view →view trajectory → |
| GLM 5.2claude-code | claude-code | TN | 0.00 | view →view trajectory → |
| MiMo V2.5claude-code | claude-code | TN | 0.00 | view →view trajectory → |
| MiniMax M3claude-code | claude-code | TN | 0.00 | view →view trajectory → |
| MiniMax M3terminus-2 | terminus-2 | TN | 0.00 | view →view trajectory → |
| Opus 4.8claude-code | claude-code | TN | 0.00 | view →view trajectory → |
| Opus 4.8terminus-2 | terminus-2 | TN | 0.00 | view →view trajectory → |
| Qwen3.7claude-code | claude-code | TN | 0.00 | view →view trajectory → |
| Qwen3.7terminus-2 | terminus-2 | TN | 0.00 | view →view trajectory → |