Search: #LLMBenchmark
MotionGraphicsBench: 100 SVG animation tasks. Only 4 of 32 LLMs exceed 50% pass@1. GPT-6 Astra (max) leads at 67.7%. Verification programs reach 93.3% human agreement. #AI #LLMBenchmark
[6]
8
We benchmarked 27 open-source LLMs for quality, latency and reliability—and discovered why benchmark configuration can completely change the results.
#llmbenchmark #opensourcellms...Show more
1
2
791
GLM features a surprisingly low price, while Grok seems to have struck a decent balance between cost and performance.
#AIvsAI #GrokAI #LLMBenchmark
Grok 4.5 operates at Opus-level intelligence and Sonnet-level pricing!
We tested it against Claude Opus 4.8, Sonnet 5, and GLM 5.2 on a SpaceX mission: liftoff, stage sep, orbit insertion, satellite deploy.
Grok 4.5 animates it more accurately and vividly than Sonnet 5 and GLM 5.2. Opus is the fastest and most expensive. Grok burns more tokens but still lands ~40% cheaper than Opus, and a bit under Sonnet 5.
Time / tokens / cost:
Grok 4.5 — / 35k / $0.69
Opus 4.8 — / 13.7k / $1.10
Sonnet 5 — / 22.1k / $0.78
GLM 5.2 — / 92.7k / $0.13
yet GLM did the whole mission for $0.13 😮
4
138
Read my latest #research on #LLMsafety, #LLMBenchmark, and #TrustwothyLLM, published with @SpringerNature in @NatMachIntell.
We benchmark large language models on potential safety risks in scientific laboratory settings.
🔗 Read & share: rdcu.be/eY08e
Happy to be part of this large scale effort! #LLMBenchmark #PhysicalCommonsenseReasoning
Introducing Global PIQA, a new multilingual benchmark for 100+ languages. This benchmark is the outcome of this year’s MRL shared task, in collaboration with 300+ researchers from 65 countries. This dataset evaluates physical commonsense reasoning in culturally relevant contexts.
5
125
3/12 LLMs are not Zero-Shot Biomedical Reasoners
This paper benchmarks LLMs on biomedical tasks it tests LLMs on Medical Classification and NER
Evaluates standard prompting, CoT, self-consistency, and RAG
#BiomedicalLLMs #AIHealthcare #LLMBenchmark
xitter.twitterapi.xyz/OpenlifesciAI/status/1…
🚨 Medical AI Research alert! 🚨
Can LLMs perform well on structured medical prediction tasks?
ASUS(@ASUS) presents: LLMs are not Zero-Shot Reasoners for Biomedical Information Extraction By (@aishiknagar, @ARCHDrNguyen, @haoli_ii and others)
Here's why it's exciting: 1/8 👇🧵
#AI #MedicalAI #NLP #AI #JohnsHopkins #Bioinformatics #GenAI #MachineLearning #HealthTech #Medical #LLM #GPT #Healthcare #pubmed #arxiv #vision #nlp #research
GPT3.5/4 が解けて、Bardが解けない問題。皆さんは解けますか？ #LLMBenchmark
２％の食塩水１リットルに、５％の食塩水を加えて、４％の食塩水を作りたいと思います。どのくらい加えれば良いですか？