Today, the Silicon Integration Initiative (Si2) is announcing the launch of LBC-bench, a public, open-source leaderboard that tracks how large language models and AI agents perform on semiconductor design and verification tasks. The leaderboard is live now at LBC-bench Leaderboards, and the full website codebase is open source on GitHub at si2org/lbc-bench.github.io.
Modern chips are far more complex than they were a decade ago, and design teams are under growing pressure to deliver reliable silicon on tight schedules. LLMs offer promising assistance across this workflow—from drafting RTL to explaining verification failures—but today’s capabilities vary widely from task to task, and claims about model performance can be difficult to compare or reproduce. LBC-bench exists to close that gap: it gives the ecosystem a common, transparent way to measure progress on real design and verification problems, so teams can learn what actually works, compare approaches fairly, and focus investment where it matters most.
Rigorous benchmarking is what ties scientific validity to innovative progress: it’s what makes claims credible, comparisons fair, and advances real rather than aspirational. LBC-bench is built on that same foundation, grounding advances in chip and system design in evidence, transparency, and measurable impact rather than in marketing claims.
Backed by Si2’s LLM Benchmarking Coalition
LBC-bench is the flagship output of Si2’s LLM Benchmarking Coalition (LBC)—a collaborative group of corporate and academic members working to expedite the development of high-quality LLMs for semiconductor design by creating relevant, comprehensive evaluation datasets and metrics. The Coalition’s work spans building and extending benchmark problem sets, maintaining the public leaderboard, refining benchmarking methodology, and publishing interpretation of results so the numbers are meaningful, not just numerous. Because the benchmarks and the leaderboard itself are fully open source, any organization or researcher can inspect exactly how a score was produced, propose new problem categories, or contribute their own evaluation results.
Built on CVDP
The leaderboard’s initial focus is the Comprehensive Verilog Design Problems (CVDP) benchmark, hosted at NVlabs/cvdp_benchmark and described in detail in its introductory paper on arXiv (2506.14074). CVDP is currently organized around categories such as code generation under limited context, code generation with heavier context, and code comprehension. The scope is intentionally growing: CVDP already spans both non-agentic and agentic evaluation datasets, and the Coalition continues to extend it into new problem categories and design domains as the field matures.
How to use it
The leaderboard site lets you browse results by category, filter by tags and by dataset release, and drill into individual submissions to see how a given model or agent was evaluated—including badges that flag open-source submissions and independently verified results. Because the site and its underlying data format are open source, teams can also fork the repository, inspect the leaderboard’s JSON data model, and run the same evaluation pipeline locally to reproduce a result before trusting it.
How to contribute your own results
The Coalition designed LBC-bench to be easy for new participants to contribute to, not just consume. If you’d like to submit your own model or agent’s evaluation results for inclusion on the leaderboard, instructions are provided directly on the leaderboard’s GitHub page—see the contributor guide in the repository for the submission format and process.
We invite the semiconductor design, EDA, and AI research communities to explore the leaderboard, dig into the CVDP benchmark, and— if you’re building or evaluating models for hardware design— submit your results to help the ecosystem build a shared, trustworthy picture of where LLMs stand today, and where they need to go next.
About Si2
The Silicon Integration Initiative Inc. is a global, not-for-profit membership organization with approximately 70 corporate members who collaborate on trusted standards and shared solutions that reduce development costs and increase design productivity for semiconductor foundries, fabless design companies, and EDA software providers. Learn more at https://si2.org/.
Questions? Contact Leigh Anne Clevenger, Vice President, Technology, Si2, le*****************@*i2.org