Article
Bingran You
Table 1 Education
| Institution | Degree | Period |
|---|---|---|
| University of California, BerkeleyBerkeley, California · Haeffner Lab | PhD Candidate in Applied Science & Technology | 2022 — Present |
| University of Chinese Academy of SciencesBeijing, China · GPA 3.95 / 4.00 · Rank 1 / 54 | BS in Physics, Minor in Computer Science | 2018 — 2022 |

Fig. 1 | Bingran You.
Table 2 Projects
| Project | Description | Link |
|---|---|---|
| FrontierPhysics | FrontierPhysics is an open benchmark measuring whether AI agents can carry out authentic, specialist-level physics research. | benchflow.ai/frontierphysics |
| BenchFlow | BenchFlow is a frontier environment lab. We build the environments AI agents learn in. We ship SkillsBench, ClawsBench, PostTrain, and the runtime. | benchflow.ai |
| SkillsBench | SkillsBench evaluates how well skills work and how effective agents are at using them. | github.com/benchflow-ai/skillsbench |
| first-tree | Open-source agent orchestration for engineers. Put Claude Code, Codex, Cursor and your own agents on one backlog — parallel runs on your keys, review before merge, everything lands as a pull request. | first-tree.aigithub.com/first-tree-ai/first-tree |
| DeepTutor | DeepTutorZotero is a research sources manager based on Zotero, with amazing AI capability powered by DeepTutor. | github.com/KnoWhiz/DeepTutorZotero |
| mews | Represent you to finish all the work, when you are sleeping. | github.com/bingran-you/mews |
| smolclaw | High resolution mock environments for testing and improving claw like agents | github.com/bingran-you/smolclaw |
| SBTI CLI | SBTI CLI - Test SBTI for your agents. | github.com/bingran-you/sbti-cli |
| bem | triangulation, boundary element method (BEM), fast multipole method (FMM) code for python | github.com/HaeffnerLab/bem |
| artiq_photonics_integration | ARTIQ Control Framework (ACF) of Photonics Integration | github.com/HaeffnerLab/artiq_photonics_integration |
Agent Evaluation
Li, X.*, Liu, Y.*, Chen, W.*, You, B.* et al. In NeurIPS (2026); preprint at arXiv:2602.12670.
Agent Skills are structured packages of procedural knowledge that augment large language model (LLM) agents at inference time. Despite rapid adoption, there is no standard way to measure whether they actually help. We present SkillsBench, a benchmark whose current
inventory contains 87 tasks across 8 domains paired with curated Skills and deterministic verifiers. Our latest aggregate evaluation runs the 87-task benchmark under matched no-Skills and curated-Skills conditions for 18 model–harness configurations. Curated Skills raise
the average pass rate from 33.9% to 50.5% (+16.6 percentage points; 25.5% normalized gain), with configuration-level gains ranging from +4.1 to +25.7 pp. Focused Skills with at most three modules outperform larger or exhaustive bundles, and smaller models with Skills can match
larger models without them. SkillsBench establishes paired evaluation as the foundation for rigorous measurement of Skill efficacy on agen
Fig. 2 | Agent architecture stack and resolution rates across 18 model–harness configurations on 87 SkillsBench tasks. Each bar stacks the no-Skills baseline and curated-Skills lift; open/filled interval markers denote 95% CIs for the
baseline/total. Bars are ordered by Curated-Skills pass rate and colored by model family. Harness abbreviations: OH = OpenHands, CC = Claude Code, GCLI = Gemini CLI. Figure 1 of arXiv:2602.12670 (CC BY 4.0).
Table 3 Pass rates (%), absolute gain (Δ, pp), and normalized gain (g, %) across the latest 87-task no-Skills vs. curated-Skills aggregate. Δ and g are computed at full precision; configurations are ordered by Curated-Skills pass rate. The Mean-row g macro-averages per-configuration normalized gains rather than recomputing g from the mean pass rates. Each pass cell uses the fixed 87 × 3 trial frame; per-cell public coverage is reported in Appendix I. Differences from rounded Pass cells may differ by ±0.1 pp. Table 2 of arXiv:2602.12670 (CC BY 4.0).
| No Sk. | Curated Skills | ||||
|---|---|---|---|---|---|
| Harness | Model | Pass | Pass | Δ(pp) | g(%) |
| OpenHands | GPT-5.5 | 51.5 | 67.3 | +15.8 | 32.6 |
| Codex | GPT-5.5 | 46.8 | 66.5 | +19.7 | 37.0 |
| Claude Code | Opus 4.7 | 43.0 | 61.2 | +18.2 | 31.9 |
| Gemini CLI | Gemini 3.1 Pro | 36.0 | 60.8 | +24.8 | 38.7 |
| OpenHands | GLM 5.1 | 32.7 | 58.4 | +25.7 | 38.1 |
| OpenHands | Claude Opus 4.8 | 45.7 | 54.1 | +8.4 | 15.5 |
| OpenHands | Kimi K2.6 | 33.4 | 54.0 | +20.6 | 31.0 |
| OpenHands | Claude Opus 4.7 | 42.1 | 53.1 | +11.1 | 19.1 |
| OpenHands | MiniMax M3 | 29.7 | 53.0 | +23.3 | 33.2 |
| OpenHands | Gemini 3.1 Pro | 33.8 | 52.8 | +19.0 | 28.7 |
| OpenHands | DeepSeek V4 Pro | 26.9 | 50.1 | +23.2 | 31.8 |
| OpenHands | Gemini 3.5 Flash | 41.1 | 48.2 | +7.1 | 12.1 |
| OpenHands | Claude Sonnet 4.6 | 33.5 | 47.2 | +13.6 | 20.5 |
| OpenHands | DeepSeek V4 Flash | 27.5 | 44.7 | +17.2 | 23.7 |
| OpenHands | Grok 4.3 | 22.8 | 41.7 | +18.8 | 24.4 |
| OpenHands | GPT-5.4 Mini | 29.9 | 41.4 | +11.5 | 16.4 |
| OpenHands | MiniMax M2.7 | 18.1 | 34.9 | +16.8 | 20.5 |
| OpenHands | Gemini 3.1 Flash Lite | 16.0 | 20.1 | +4.1 | 4.9 |
| Mean | 33.9 | 50.5 | +16.6 | 25.5 | |
Li, X.*, Choe, K. W.*, Liu, Y., Chen, X., Tao, C., You, B. et al. In COLM (2026); preprint at arXiv:2604.05172.
Large language model (LLM) agents are increasingly deployed to auto
LLM agents in realistic productivity settings. It includes five high-
Article
disclosure, and a meta prompt that coordinates behavior across ser
63%), with unsafe action rates from 7% to 23% and no consistent ordering between the two metrics. We identify eight recurring patterns of unsafe behavior, including multi-step sandbox escalation and silent contract modification. We release the trajectories and future dataset at clawsbench.com.
Fig. 3 | ClawsBench evaluation pipeline. Seed data populates five SQLite-
state-based evaluator compares pre- and post-execution database snapshots to score each trial; scores are aggregated into Task Success Rate (TSR), Unsafe Action Rate (UAR), and Safe Completion Rate (SCR). Figure 1 of arXiv:2604.05172.
Table 4 Model comparison on OpenClaw at unscaffolded (off/off) and fully scaffolded (on/on) conditions. Sk = Domain Skills; Mt = Meta Prompt. TSR = Task Success Rate (non-safety, ≥ 0.8). UAR = Unsafe Action Rate (safety, < 0). SCR = Safe Completion Rate (safety, ≥ 0.8). Task-level cluster bootstrap 95% CIs in brackets. Full 33-condition results in Table 11 (Appendix). Table 4 of arXiv:2604.05172.
| Model | Sk | Mt | TSR | UAR | SCR | |||
|---|---|---|---|---|---|---|---|---|
| Gemini 3.1 Flash-Lite | ✗ | ✗ | 0% | [0,0] | 0% | [0,0] | 0% | [0,0] |
| Gemini 3.1 Flash-Lite | ✓ | ✓ | 39% | [21,57] | 23% | [11,35] | 26% | [12,41] |
| Gemini 3.1 Pro | ✗ | ✗ | 8% | [2,16] | 4% | [0,12] | 2% | [0,4] |
| Gemini 3.1 Pro | ✓ | ✓ | 58% | [38,77] | 10% | [3,19] | 48% | [30,65] |
| Claude Sonnet 4.6 | ✗ | ✗ | 0% | [0,0] | 0% | [0,0] | 1% | [0,3] |
| Claude Sonnet 4.6 | ✓ | ✓ | 56% | [36,75] | 13% | [3,27] | 48% | [32,64] |
| Claude Opus 4.6 | ✗ | ✗ | 0% | [0,0] | 0% | [0,0] | 0% | [0,0] |
| Claude Opus 4.6 | ✓ | ✓ | 63% | [42,82] | 23% | [8,40] | 50% | [32,68] |
| GPT-5.4 | ✗ | ✗ | 1% | [0,3] | 0% | [0,0] | 0% | [0,0] |
| GPT-5.4 | ✓ | ✓ | 53% | [34,72] | 7% | [0,15] | 41% | [27,55] |
| GLM-5 | ✗ | ✗ | 3% | [0,6] | 0% | [0,0] | 1% | [0,3] |
| GLM-5 | ✓ | ✓ | 60% | [41,78] | 23% | [10,38] | 48% | [32,63] |
Zheng, S., Di, Z., Liu, Y., …, You, B. et al. Preprint at arXiv:2609.11028 (2026).
LLM-agent benchmarks increasingly function as interactive evalua
hacking: an agent improves its measured score by exploiting the reward-relevant trajectory instead of solving the intended task. Existing defenses rely largely on task-specific patches, prompt instructions, or post-hoc detectors. They do not provide reusable
evidence that a concrete run remained within its intended evaluation boundary. This paper presents BenchShield, a model-backed instru
We construct BenchShield Trajectories, a human-labeled corpus of 456 adjudicated trajectories from more than 31,000 public agent runs across three benchmarks. Compared with an agentic hackability scanner baseline on the same tasks and model, BenchShield improves full-chain recall from 23–94% to 77–100%, same-vector coverage from 16–56% to 43–78%, and reduces per-task cost by up to 65%. Its runtime analysis achieves 96% accuracy in detecting reward hacking from infrastructure-side evidence.
Fig. 4 | Two reward-hacking failures from Terminal-Bench 3 in which an isolated verifier is insufficient. Left: agent-controlled executable content disables a kernel check through the declared patch (I2 → I7). Right: the agent
downloads hidden labels and submits answer-derived clusters (I1 → I7). Both runs retain a native reward of 1.00; BenchShield reports AgentViolation. Figure 1 of arXiv:2609.11028 (CC BY 4.0).
Table 5 Paired static discovery on the same native tasks, adjudicated integrity-link chains, and identical stripped task packages. Recall columns report the union over five trials and the per-trial mean ± standard deviation. Same-vector also requires the reported path to name the channel used by the exploit. Stability is the mean pairwise Jaccard similarity across trials. Cost and wall time are measured per audit with the same model. Table 3 of arXiv:2609.11028 (CC BY 4.0).
| Corpus | System | Dim. recall | Same-vector | Full chain | Stability | Wall | Cost/task |
|---|---|---|---|---|---|---|---|
| SkillsBench | BenchJack | 0.60 (0.37 ± 0.08) | 0.27 (0.21 ± 0.03) | 0.25 (0.25 ± 0.00) | 0.63 | 279 s | $3.34 |
| BenchShield | 0.93 (0.85 ± 0.06) | 0.67 (0.61 ± 0.08) | 0.88 (0.73 ± 0.10) | 0.94 | 150 s | $2.15 | |
| ClawsBench | BenchJack | 0.94 (0.94 ± 0.00) | 0.56 (0.28 ± 0.09) | 0.94 (0.94 ± 0.00) | 0.92 | 460 s | $2.04 |
| BenchShield | 1.00 (0.96 ± 0.02) | 0.78 (0.69 ± 0.06) | 1.00 (0.95 ± 0.03) | 0.91 | 410 s | $1.91 | |
| Terminal-Bench 3 | BenchJack | 0.48 (0.46 ± 0.02) | 0.16 (0.14 ± 0.02) | 0.23 (0.19 ± 0.04) | 0.93 | 357 s | $5.91 |
| BenchShield | 0.80 (0.73 ± 0.03) | 0.43 (0.29 ± 0.11) | 0.77 (0.66 ± 0.04) | 0.79 | 426 s | $2.05 |
Experimental AMO Physics
Xu, S.*, Xia, X.*, Yu, Q., Parakh, A., Khan, S., Megidish, E., You, B. et al. Nature 645, 362–368 (2025). arXiv:2310.00595.
Trapped-ion applications, such as in quantum information, precision measurements, optical clocks, and mass spectrometry, rely on special
arrays of high-performance miniaturized 3D traps. We show that 3D-
Article
Fig. 5 | Paul trap comparison. Schematics and representative pictures of a conventional 3D Paul trap a, a surface trap b, and a 3D-printed vertical Paul trap c. The basic structure of a conventional 3D Paul trap (a) consists of four RF electrodes and two end cap DC electrodes, with a characteristic length scale of 1 mm. It requires precision machining and is thus challenging to scale. Surface traps (b) usually have two RF electrodes with a characteristic length scale on
the order of 100 µm and can be produced by microfabrication techniques allowing for large and scalable trap arrays. However, constraining electrodes to a single plane distorts the desired quadrupole potential reducing its efficiency. 3D-printed Paul traps (c) can have four RF electrodes creating a quadrupole field while of similar size and scalability as surface traps. Figure 1 of arXiv:2310.00595.
Broz, J., You, B., Khan, S., Häffner, H., Kaplan, D. E. & Rajendran, S. Phys. Rev. Lett. 130, 200201 (2023). arXiv:2206.12976.
Kaplan and Rajendran have recently demonstrated that non-linear and state-dependent terms can be consistently added to quantum field theory to yield causal non-linear time evolution in quantum mechanics. Causal non-linear theories have the unavoidable feature that their quantum effects are dramatically sensitive to the full phys
are not well tested by conventional atomic and nuclear spectroscopy. By using a well-controlled superposition of vibrational modes of a 40Ca+ ion trapped in a harmonic potential, we set a stringent limit of 5.4 × 10−12 on the magnitude of the unitless scaling factor ϵ̃γ of the predicted causal, non-linear perturbation.
Fig. 6 | Experimental implementation. (a) A 40Ca+ ion is trapped using a combination of RF and DC electric fields. In a time-averaged sense, the confinement is well modeled by a 3-dimensional harmonic potential. (b) Motion along the x-direction is excited by resonantly coupling to the internal electronic Zeeman sublevels of the 42S1/2 ↔ 32D5/2 transition using narrow band light near 729 nm. The degeneracy of the Zeeman states is broken through application of a strong magnetic field of ≈4 G. Measurement is performed by scattering photons of the short-lived 42S1/2 ↔ 42P1/2, which are then focused onto an EMCCD camera. (c) The experimental pulse sequence. Pulses that address the qubit are colored gray and those that address the blue sideband, blue. After preparing the state |S,0⟩, the first pair of pulses is used to generate
the state |ψ(t = 0)⟩ = |D⟩(α0|0⟩ + α1|1⟩). This is then allowed to freely evolve for a time τ, accumulating a relative phase of Φ(τ), which is sensitive to the proposed causal non-linear perturbation. Afterwards, the information is mapped onto the qubit with a blue sideband pulse and then the expectation value of the Pauli spin operator cos(ξL)σx + sin(ξL)σy is measured. (d) An illustration of the two-step process for generating the state |ψ(t = 0)⟩, as described in more detail in the main text. The key feature is the fact that the state |D,0⟩ is transparent to the resonant blue sideband drive as illustrated in (e). This allows us to map an arbitrary qubit state onto the ground and first excited state of the vibrational mode. Figure 1 of arXiv:2206.12976.
You, B.*, Wu, Q.*, Miron, D., Ke, W., Monga, I., Saglamyurek, E. & Haeffner, H. Phys. Rev. Appl. 26, 014101 (2026). arXiv:2405.10501.
High-rate remote entanglement between photon and matter-based qubits is essential for distributed quantum information processing. A key technique to increase the modest entangling rates of existing long-
Fig. 7 | Schematics of multiplexed ion-photon interface. (a) A nine-ion chain is confined in an RF Paul trap. Controlling DC endcap voltages allows for ion transport. A beam of 397 nm and 866 nm light illuminating all ions is used for Doppler cooling. An objective collects the 397 nm single photons and guides them to a 50/50 beamsplitter, followed by a photomultiplier tube on each exit port for photon detection. An 866 nm beam counter-propagates with the single photons and addresses individual ions. (b), (c) Excitation scheme and pulse sequence for the 397 nm single-photon generation. First, a global 397 nm beam prepares the ions in the 32D3/2 state. Then, the 866 nm addressing beam (resonant with 32D3/2 ↔ 42P1/2) is stroboscopically switched on when the target ion is in the interaction zone to extract photons from the target ion. Figure 1 of arXiv:2405.10501 (CC BY 4.0).
Momenzadeh, M., Sun, K., Wu, Q., You, B., Tang, Y.-L., Häffner, H. & Shcherbakov, M. R. npj Nanophotonics 3, 3 (2026). arXiv:2505.08997.
Trapped-ion quantum computing requires precise optical control for individual qubit manipulation. However, conventional free-space optics face challenges in alignment stability and scalability as the number of qubits increases. Integrated photonics offers a promising alternative, providing miniaturized optical systems on a chip. Here, we propose a design for a multimode photonic circuit integrated with a surface-electrode ion trap capable of targeted and reconfigurable light delivery. Three closely positioned ions can be addressed using a focusing grating coupler that emits multimode light through electrode openings to ions trapped 80 µm above the chip. Simulations show that the couplers achieve diffraction-limited spot with a 4.3 µm beam waist
along the trap axis and 2.2 µm perpendicular to the trap axis. Controlled interference of the TE10 and TE20 modes results in crosstalk of –20 dB to –30 dB at ion separations of 5-8 µm when addressing ions individually, and down to –60 dB when two of the three ions are addressed simultaneously. Additionally, the higher-order TE modes can offer a novel mechanism for driving spin-motion coupling transi
Article
Fig. 8 | (a) Conceptual illustration of the designed surface trap with ions, integrated waveguides and beam forming elements underneath at multiple zones. Ions are trapped at one of the positions marked by the green dots, H= 80 µm above the electrodes, with appropriate potentials applied to the DC and radio-frequency (RF) electrodes. The distances from the grating to the centerline of the trapping region are denoted as d1 = 30 µm and d2 = 100 µm.
Top-left shows the energy-level diagram for 40Ca+. (b) Material stack of final ion trap devices, showing the integrated multimode waveguides and beam forming element beneath the surface-trap electrodes and light diffracting out of the chip by means of an apodized grating. Indium tin oxide (ITO) protects the ions from interface/substrate charging at the cost of reduced out coupling efficiencies. Figure 1 of arXiv:2505.08997.
Klawson, D.*, Zhi, Y.*, You, B.*, Bareian, M.*, Mossman, E.* et al. Preprint at arXiv:2607.25062 (2026).
Fig. 9 | Architecture of the 2D-3D PIC. a, Isometric rendering of the device. The photonic layer is made up of an amorphous Al2O3 core with a symmetric SiO2 cladding, patterned underneath the 200 nm Al surface-electrode layer. In the ion interaction region, a combination of lithographically defined photonics and 2PP-printed optics is used to route and shape the beam delivered to the ion location. b, Side view of the on-chip imaging system. The beam exits from the on-chip planar lens aperture and is focused by the Al-
The planar lens expands and collimates the optical mode from the incoming strip waveguides. The mirror focuses the optical energy onto the ion location. Each incoming waveguide is mapped to a different focal spot by the imaging system. d, Optical micrograph of a finished device showing the planar lens, the μ-mirror, and the surrounding surface-trap electrodes. The mirror has the same potential as the center DC electrode. The alignment accuracy achieved between the substrate and the printed mirror is within 1 μm. Figure 1 of arXiv:2607.25062 (CC BY 4.0).
Trapped ions provide a high-fidelity platform for quantum informa
scale. This implementation can address three individual ions from λ = 405 – 880 nm with -27 dB average intensity crosstalk at 5 μm pitch. We trap Ca and Ba ions above such devices, characterize optical crosstalk with barium ions, and demonstrate individual repumping of calcium ions. This monolithic photonic architecture brings broadband addressing in an on-chip modality to trapped-ion technology. More generally, integrating additive manufacturing into quantum devices is poised to unlock expanded design space for implementing novel quantum architectures.
References
1. Li, X.*, Liu, Y.*, Chen, W.*, You, B.* et al. SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks. In NeurIPS (2026); preprint at arXiv:2602.12670.
2. Li, X.*, Choe, K. W.*, Liu, Y., Chen, X., Tao, C., You, B. et al. ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces. In COLM (2026); preprint at arXiv:2604.05172.
3. Zheng, S., Di, Z., Liu, Y., …, You, B. et al. BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure. Preprint at arXiv:2609.11028 (2026).
4. Xu, S.*, Xia, X.*, Yu, Q., Parakh, A., Khan, S., Megidish, E., You, B. et al. 3D-printed micro ion trap technology for quantum information applications. Nature 645, 362–368 (2025).
5. Broz, J., You, B., Khan, S., Häffner, H., Kaplan, D. E. & Rajendran, S. Test of Causal Nonlinear Quantum Mechanics by Ramsey Interferometry with a Trapped Ion. Phys. Rev. Lett. 130, 200201 (2023).
6. You, B.*, Wu, Q.*, Miron, D., Ke, W., Monga, I., Saglamyurek, E. & Haeffner, H. Temporally multiplexed ion-photon quantum interface via fast ion-chain transport. Phys. Rev. Appl. 26, 014101 (2026).
7. Momenzadeh, M., Sun, K., Wu, Q., You, B., Tang, Y.-L., Häffner, H. & Shcherbakov, M. R. Individual trapped-ion addressing with adjoint-optimized multimode photonic circuits. npj Nanophotonics 3, 3 (2026).
8. Klawson, D.*, Zhi, Y.*, You, B.*, Bareian, M.*, Mossman, E.* et al. A broadband, individually addressing two- and three-dimensional photonic integrated circuit for trapped-ion qubit control. Preprint at arXiv:2607.25062 (2026).
Article
Bingran You
Table 1 Education
| Institution | Degree | Period |
|---|---|---|
| University of California, BerkeleyBerkeley, California · Haeffner Lab | PhD Candidate in Applied Science & Technology | 2022 — Present |
| University of Chinese Academy of SciencesBeijing, China · GPA 3.95 / 4.00 · Rank 1 / 54 | BS in Physics, Minor in Computer Science | 2018 — 2022 |

Table 2 Projects
| Project | Description | Link |
|---|---|---|
| FrontierPhysics | FrontierPhysics is an open benchmark measuring whether AI agents can carry out authentic, specialist-level physics research. | benchflow.ai/frontierphysics |
| BenchFlow | BenchFlow is a frontier environment lab. We build the environments AI agents learn in. We ship SkillsBench, ClawsBench, PostTrain, and the runtime. | benchflow.ai |
| SkillsBench | SkillsBench evaluates how well skills work and how effective agents are at using them. | github.com/benchflow-ai/skillsbench |
| first-tree | Open-source agent orchestration for engineers. Put Claude Code, Codex, Cursor and your own agents on one backlog — parallel runs on your keys, review before merge, everything lands as a pull request. | first-tree.aigithub.com/first-tree-ai/first-tree |
| DeepTutor | DeepTutorZotero is a research sources manager based on Zotero, with amazing AI capability powered by DeepTutor. | github.com/KnoWhiz/DeepTutorZotero |
| mews | Represent you to finish all the work, when you are sleeping. | github.com/bingran-you/mews |
| smolclaw | High resolution mock environments for testing and improving claw like agents | github.com/bingran-you/smolclaw |
| SBTI CLI | SBTI CLI - Test SBTI for your agents. | github.com/bingran-you/sbti-cli |
| bem | triangulation, boundary element method (BEM), fast multipole method (FMM) code for python | github.com/HaeffnerLab/bem |
| artiq_photonics_integration | ARTIQ Control Framework (ACF) of Photonics Integration | github.com/HaeffnerLab/artiq_photonics_integration |
Agent Evaluation
SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
Li, X.*, Liu, Y.*, Chen, W.*, You, B.* et al. In NeurIPS (2026); preprint at arXiv:2602.12670.
Agent Skills are structured packages of procedural knowledge that augment large language model (LLM) agents at inference time. Despite rapid adoption, there is no standard way to measure whether they actually help. We present SkillsBench, a benchmark whose current inventory contains 87 tasks across 8 domains paired with curated Skills and deterministic verifiers. Our latest aggregate evaluation runs the 87-task benchmark under matched no-Skills and curated-Skills conditions for 18 model–harness configurations. Curated Skills raise the average pass rate from 33.9% to 50.5% (+16.6 percentage points; 25.5% normalized gain), with configuration-level gains ranging from +4.1 to +25.7 pp. Focused Skills with at most three modules outperform larger or exhaustive bundles, and smaller models with Skills can match larger models without them. SkillsBench establishes paired evaluation as the foundation for rigorous measurement of Skill efficacy on agentic, expertise-heavy work.

Table 3 Pass rates (%), absolute gain (Δ, pp), and normalized gain (g, %) across the latest 87-task no-Skills vs. curated-Skills aggregate. Δ and g are computed at full precision; configurations are ordered by Curated-Skills pass rate. The Mean-row g macro-averages per-configuration normalized gains rather than recomputing g from the mean pass rates. Each pass cell uses the fixed 87 × 3 trial frame; per-cell public coverage is reported in Appendix I. Differences from rounded Pass cells may differ by ±0.1 pp. Table 2 of arXiv:2602.12670 (CC BY 4.0).
| No Sk. | Curated Skills | ||||
|---|---|---|---|---|---|
| Harness | Model | Pass | Pass | Δ(pp) | g(%) |
| OpenHands | GPT-5.5 | 51.5 | 67.3 | +15.8 | 32.6 |
| Codex | GPT-5.5 | 46.8 | 66.5 | +19.7 | 37.0 |
| Claude Code | Opus 4.7 | 43.0 | 61.2 | +18.2 | 31.9 |
| Gemini CLI | Gemini 3.1 Pro | 36.0 | 60.8 | +24.8 | 38.7 |
| OpenHands | GLM 5.1 | 32.7 | 58.4 | +25.7 | 38.1 |
| OpenHands | Claude Opus 4.8 | 45.7 | 54.1 | +8.4 | 15.5 |
| OpenHands | Kimi K2.6 | 33.4 | 54.0 | +20.6 | 31.0 |
| OpenHands | Claude Opus 4.7 | 42.1 | 53.1 | +11.1 | 19.1 |
| OpenHands | MiniMax M3 | 29.7 | 53.0 | +23.3 | 33.2 |
| OpenHands | Gemini 3.1 Pro | 33.8 | 52.8 | +19.0 | 28.7 |
| OpenHands | DeepSeek V4 Pro | 26.9 | 50.1 | +23.2 | 31.8 |
| OpenHands | Gemini 3.5 Flash | 41.1 | 48.2 | +7.1 | 12.1 |
| OpenHands | Claude Sonnet 4.6 | 33.5 | 47.2 | +13.6 | 20.5 |
| OpenHands | DeepSeek V4 Flash | 27.5 | 44.7 | +17.2 | 23.7 |
| OpenHands | Grok 4.3 | 22.8 | 41.7 | +18.8 | 24.4 |
| OpenHands | GPT-5.4 Mini | 29.9 | 41.4 | +11.5 | 16.4 |
| OpenHands | MiniMax M2.7 | 18.1 | 34.9 | +16.8 | 20.5 |
| OpenHands | Gemini 3.1 Flash Lite | 16.0 | 20.1 | +4.1 | 4.9 |
| Mean | 33.9 | 50.5 | +16.6 | 25.5 | |
ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces
Li, X.*, Choe, K. W.*, Liu, Y., Chen, X., Tao, C., You, B. et al. In COLM (2026); preprint at arXiv:2604.05172.
Large language model (LLM) agents are increasingly deployed to automate productivity tasks (e.g., email, scheduling, document management), but evaluating them on live services is risky due to potentially irreversible changes. Existing benchmarks rely on simplified environments and fail to capture realistic, stateful, multi-service workflows. We introduce ClawsBench, a benchmark for evaluating and improving LLM agents in realistic productivity settings. It includes five high-fidelity mock services (Gmail, Slack, Calendar, Docs, Drive) with full state management and deterministic snapshot/restore, along with 44 structured tasks covering single-service, cross-service, and safety-critical scenarios. We decompose agent scaffolding into two independent levers (domain skills that inject API knowledge via progressive disclosure, and a meta prompt that coordinates behavior across services) and vary both to measure their separate and combined effects. Experiments across 6 models, 4 agent harnesses, and 33 conditions show that with full scaffolding, agents achieve task success rates of 39–64% but exhibit unsafe action rates of 7–33%. On OpenClaw, the top five models fall within a 10 percentage-point band on task success (53–63%), with unsafe action rates from 7% to 23% and no consistent ordering between the two metrics. We identify eight recurring patterns of unsafe behavior, including multi-step sandbox escalation and silent contract modification. We release the trajectories and future dataset at clawsbench.com.

Table 4 Model comparison on OpenClaw at unscaffolded (off/off) and fully scaffolded (on/on) conditions. Sk = Domain Skills; Mt = Meta Prompt. TSR = Task Success Rate (non-safety, ≥ 0.8). UAR = Unsafe Action Rate (safety, < 0). SCR = Safe Completion Rate (safety, ≥ 0.8). Task-level cluster bootstrap 95% CIs in brackets. Full 33-condition results in Table 11 (Appendix). Table 4 of arXiv:2604.05172.
| Model | Sk | Mt | TSR | UAR | SCR | |||
|---|---|---|---|---|---|---|---|---|
| Gemini 3.1 Flash-Lite | ✗ | ✗ | 0% | [0,0] | 0% | [0,0] | 0% | [0,0] |
| Gemini 3.1 Flash-Lite | ✓ | ✓ | 39% | [21,57] | 23% | [11,35] | 26% | [12,41] |
| Gemini 3.1 Pro | ✗ | ✗ | 8% | [2,16] | 4% | [0,12] | 2% | [0,4] |
| Gemini 3.1 Pro | ✓ | ✓ | 58% | [38,77] | 10% | [3,19] | 48% | [30,65] |
| Claude Sonnet 4.6 | ✗ | ✗ | 0% | [0,0] | 0% | [0,0] | 1% | [0,3] |
| Claude Sonnet 4.6 | ✓ | ✓ | 56% | [36,75] | 13% | [3,27] | 48% | [32,64] |
| Claude Opus 4.6 | ✗ | ✗ | 0% | [0,0] | 0% | [0,0] | 0% | [0,0] |
| Claude Opus 4.6 | ✓ | ✓ | 63% | [42,82] | 23% | [8,40] | 50% | [32,68] |
| GPT-5.4 | ✗ | ✗ | 1% | [0,3] | 0% | [0,0] | 0% | [0,0] |
| GPT-5.4 | ✓ | ✓ | 53% | [34,72] | 7% | [0,15] | 41% | [27,55] |
| GLM-5 | ✗ | ✗ | 3% | [0,6] | 0% | [0,0] | 1% | [0,3] |
| GLM-5 | ✓ | ✓ | 60% | [41,78] | 23% | [10,38] | 48% | [32,63] |
BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure
Zheng, S., Di, Z., Liu, Y., …, You, B. et al. Preprint at arXiv:2609.11028 (2026).
LLM-agent benchmarks increasingly function as interactive evaluation infrastructure. Agents observe state, call tools, modify workspaces, submit artifacts, and receive rewards from outcome procedures. This interactivity makes evaluations vulnerable to reward hacking: an agent improves its measured score by exploiting the reward-relevant trajectory instead of solving the intended task. Existing defenses rely largely on task-specific patches, prompt instructions, or post-hoc detectors. They do not provide reusable evidence that a concrete run remained within its intended evaluation boundary. This paper presents BenchShield, a model-backed instrumentation layer for reward integrity in LLM-agent evaluation. BenchShield grounds detection in a finite lifecycle model of an evaluation’s reward-relevant events. Within the benchmark infrastructure, two complementary analyses operate over this model. A static, phase-aware taint analysis exposes reward-hacking paths before a run. Its runtime counterpart uses infrastructure-side evidence to attribute concrete agent use and emit evidence-backed claims.
We construct BenchShield Trajectories, a human-labeled corpus of 456 adjudicated trajectories from more than 31,000 public agent runs across three benchmarks. Compared with an agentic hackability scanner baseline on the same tasks and model, BenchShield improves full-chain recall from 23–94% to 77–100%, same-vector coverage from 16–56% to 43–78%, and reduces per-task cost by up to 65%. Its runtime analysis achieves 96% accuracy in detecting reward hacking from infrastructure-side evidence.

AgentViolation. Figure 1 of arXiv:2609.11028 (CC BY 4.0).Table 5 Paired static discovery on the same native tasks, adjudicated integrity-link chains, and identical stripped task packages. Recall columns report the union over five trials and the per-trial mean ± standard deviation. Same-vector also requires the reported path to name the channel used by the exploit. Stability is the mean pairwise Jaccard similarity across trials. Cost and wall time are measured per audit with the same model. Table 3 of arXiv:2609.11028 (CC BY 4.0).
| Corpus | System | Dim. recall | Same-vector | Full chain | Stability | Wall | Cost/task |
|---|---|---|---|---|---|---|---|
| SkillsBench | BenchJack | 0.60 (0.37 ± 0.08) | 0.27 (0.21 ± 0.03) | 0.25 (0.25 ± 0.00) | 0.63 | 279 s | $3.34 |
| BenchShield | 0.93 (0.85 ± 0.06) | 0.67 (0.61 ± 0.08) | 0.88 (0.73 ± 0.10) | 0.94 | 150 s | $2.15 | |
| ClawsBench | BenchJack | 0.94 (0.94 ± 0.00) | 0.56 (0.28 ± 0.09) | 0.94 (0.94 ± 0.00) | 0.92 | 460 s | $2.04 |
| BenchShield | 1.00 (0.96 ± 0.02) | 0.78 (0.69 ± 0.06) | 1.00 (0.95 ± 0.03) | 0.91 | 410 s | $1.91 | |
| Terminal-Bench 3 | BenchJack | 0.48 (0.46 ± 0.02) | 0.16 (0.14 ± 0.02) | 0.23 (0.19 ± 0.04) | 0.93 | 357 s | $5.91 |
| BenchShield | 0.80 (0.73 ± 0.03) | 0.43 (0.29 ± 0.11) | 0.77 (0.66 ± 0.04) | 0.79 | 426 s | $2.05 |
Article
Experimental AMO Physics
3D-printed micro ion trap technology for quantum information applications
Xu, S.*, Xia, X.*, Yu, Q., Parakh, A., Khan, S., Megidish, E., You, B. et al. Nature 645, 362–368 (2025). arXiv:2310.00595.
Trapped-ion applications, such as in quantum information, precision measurements, optical clocks, and mass spectrometry, rely on specialized high-performance ion traps. The latter applications typically employ traditional machining to customize macroscopic 3D Paul traps, while quantum information processing experiments usually rely on photo-lithographic techniques to miniaturize the traps and meet scalability requirements. Using photolithography, however, it is challenging to fabricate the complex three-dimensional electrode structures required for optimal confinement. Here we address these limitations by adopting a high-resolution 3D printing technology based on two-photon polymerization supporting fabrication of large arrays of high-performance miniaturized 3D traps. We show that 3D-printed ion traps combine the advantages of traditionally machined 3D traps with the miniaturization provided by photolithography by confining single calcium ions in a small 3D-printed ion trap with radial trap frequencies ranging from 2 MHz to 24 MHz. The tight confinement eases ion cooling requirements and allows us to demonstrate high-fidelity coherent operations on an optical qubit after only Doppler cooling. With 3D printing technology, the design freedom is drastically expanded without sacrificing scalability and precision so that ion trap geometries can be optimized for higher performance and better functionality.

Test of Causal Nonlinear Quantum Mechanics by Ramsey Interferometry with a Trapped Ion
Broz, J., You, B., Khan, S., Häffner, H., Kaplan, D. E. & Rajendran, S. Phys. Rev. Lett. 130, 200201 (2023). arXiv:2206.12976.
Kaplan and Rajendran have recently demonstrated that non-linear and state-dependent terms can be consistently added to quantum field theory to yield causal non-linear time evolution in quantum mechanics. Causal non-linear theories have the unavoidable feature that their quantum effects are dramatically sensitive to the full physical spread of the quantum state of the system. As a result, such theories are not well tested by conventional atomic and nuclear spectroscopy. By using a well-controlled superposition of vibrational modes of a 40Ca+ ion trapped in a harmonic potential, we set a stringent limit of 5.4 × 10−12 on the magnitude of the unitless scaling factor ϵ̃γ of the predicted causal, non-linear perturbation.

Temporally multiplexed ion-photon quantum interface via fast ion-chain transport
You, B.*, Wu, Q.*, Miron, D., Ke, W., Monga, I., Saglamyurek, E. & Haeffner, H. Phys. Rev. Appl. 26, 014101 (2026). arXiv:2405.10501.
High-rate remote entanglement between photon and matter-based qubits is essential for distributed quantum information processing. A key technique to increase the modest entangling rates of existing long-distance quantum networking approaches is multiplexing. Here, we demonstrate a temporally multiplexed ion-photon interface via rapid transport of a chain of nine calcium ions across 74 µm within 86 µs. The non-classical nature of the multiplexed photons is verified by measuring the second-order correlation function with an average value of g(2)(0) = 0.060(13). This indicates low crosstalk of about 1% between the multiplexed modes, which can be reduced further once coupling of the photons to a single-mode fiber is incorporated. In addition, we characterize the motional degree-of-freedom of the ion crystal after transport and find that it is coherently excited to n̄α ≈ 110 for the center-of-mass mode. Our proof-of-principle implementation paves the way for large-scale quantum networking with trapped ions, but highlights some challenges that must be overcome.

Individual trapped-ion addressing with adjoint-optimized multimode photonic circuits
Momenzadeh, M., Sun, K., Wu, Q., You, B., Tang, Y.-L., Häffner, H. & Shcherbakov, M. R. npj Nanophotonics 3, 3 (2026). arXiv:2505.08997.
Trapped-ion quantum computing requires precise optical control for individual qubit manipulation. However, conventional free-space optics face challenges in alignment stability and scalability as the number of qubits increases. Integrated photonics offers a promising alternative, providing miniaturized optical systems on a chip. Here, we propose a design for a multimode photonic circuit integrated with a surface-electrode ion trap capable of targeted and reconfigurable light delivery. Three closely positioned ions can be addressed using a focusing grating coupler that emits multimode light through electrode openings to ions trapped 80 µm above the chip. Simulations show that the couplers achieve diffraction-limited spot with a 4.3 µm beam waist along the trap axis and 2.2 µm perpendicular to the trap axis. Controlled interference of the TE10 and TE20 modes results in crosstalk of –20 dB to –30 dB at ion separations of 5-8 µm when addressing ions individually, and down to –60 dB when two of the three ions are addressed simultaneously. Additionally, the higher-order TE modes can offer a novel mechanism for driving spin-motion coupling transitions, potentially enabling alternative approaches to quantum gates and simulations. The proposed integrated platform offers a viable path for constructing large-scale trapped-ion systems, leveraging the benefits of nanophotonic design for precise and reliable ion manipulation.

A broadband, individually addressing two- and three-dimensional photonic integrated circuit for trapped-ion qubit control
Klawson, D.*, Zhi, Y.*, You, B.*, Bareian, M.*, Mossman, E.* et al. Preprint at arXiv:2607.25062 (2026).
Trapped ions provide a high-fidelity platform for quantum information processing, yet delivery of multiple, distinct wavelengths across large networks of interaction zones remains a bottleneck. Conventional free-space light delivery lacks scalability, while on-chip grating couplers suffer from narrow operational bandwidth that increases circuit footprint and optical interfacing complexity. Here we show a broadband photonic integrated circuit capable of addressing individual ions. The circuit combines a planar waveguide lens with a micromirror fabricated using two-photon polymerization at wafer scale. This implementation can address three individual ions from λ = 405 – 880 nm with -27 dB average intensity crosstalk at 5 μm pitch. We trap Ca and Ba ions above such devices, characterize optical crosstalk with barium ions, and demonstrate individual repumping of calcium ions. This monolithic photonic architecture brings broadband addressing in an on-chip modality to trapped-ion technology. More generally, integrating additive manufacturing into quantum devices is poised to unlock expanded design space for implementing novel quantum architectures.

References
1. Li, X.*, Liu, Y.*, Chen, W.*, You, B.* et al. SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks. In NeurIPS (2026); preprint at arXiv:2602.12670.
2. Li, X.*, Choe, K. W.*, Liu, Y., Chen, X., Tao, C., You, B. et al. ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces. In COLM (2026); preprint at arXiv:2604.05172.
3. Zheng, S., Di, Z., Liu, Y., …, You, B. et al. BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure. Preprint at arXiv:2609.11028 (2026).
4. Xu, S.*, Xia, X.*, Yu, Q., Parakh, A., Khan, S., Megidish, E., You, B. et al. 3D-printed micro ion trap technology for quantum information applications. Nature 645, 362–368 (2025).
5. Broz, J., You, B., Khan, S., Häffner, H., Kaplan, D. E. & Rajendran, S. Test of Causal Nonlinear Quantum Mechanics by Ramsey Interferometry with a Trapped Ion. Phys. Rev. Lett. 130, 200201 (2023).
6. You, B.*, Wu, Q.*, Miron, D., Ke, W., Monga, I., Saglamyurek, E. & Haeffner, H. Temporally multiplexed ion-photon quantum interface via fast ion-chain transport. Phys. Rev. Appl. 26, 014101 (2026).
7. Momenzadeh, M., Sun, K., Wu, Q., You, B., Tang, Y.-L., Häffner, H. & Shcherbakov, M. R. Individual trapped-ion addressing with adjoint-optimized multimode photonic circuits. npj Nanophotonics 3, 3 (2026).
8. Klawson, D.*, Zhi, Y.*, You, B.*, Bareian, M.*, Mossman, E.* et al. A broadband, individually addressing two- and three-dimensional photonic integrated circuit for trapped-ion qubit control. Preprint at arXiv:2607.25062 (2026).