I’m into open science, elegant software architecture, and well-crafted interfaces. Building https://xitter.twitterapi.xyz/t.co/0GX1wFmPoU
San Diego, California
Joined July 2017
- Tweets2K
- Following1K
- Followers5.9K
- Likes7.9K
Raunak retweeted
in my opinion, this is actually the correct way to do benchmarking. nothing beats the user’s own code.
what would folks like to see if we did this for Archil?
Every document parser is #1 on its own benchmark.
So forget static benchmarks. Today, we're releasing ParseJudge, an open-source app that judges parsers on your own documents, free for up to 10,000 pages a day.
parsejudge.com
Raunak retweeted
If you apply for our startup program (reducto.ai/startups) to get $1,500, thats like 1,500,000 pages. what're u waiting for 🫢
Raunak retweeted
I am incredibly excited about this new launch / product. Bye @Microsoft, thanks for the years of docs. Onto the next generation of what document creation and editing looks like 📄
Raunak retweeted
some of the best r-1 evals came from documents we weren’t sure it could handle. think 10m+ pixel construction blueprints, 32k+ output tokens for a single page, and even the muffin-vs-chihuahua captcha.
In this video, @TheOtherGandhi_ and I discussed why the weird tests are often the (most) useful ones and what they taught us about r-1’s capabilities and limitations! we challenge you to keep sending us your strangest documents.
Raunak retweeted
Look guys if you’re building something on a cloud platform and you don’t want to get owned through a platform compromise at some point you have like 4 real options
AWS
Azure
Cloudflare
GCP
You should not trust key infra to a neocloud
Neoclouds may get you faster performance on something like S3 but they just can’t match the availability, durability, and security of a “real” cloud
They don’t and can’t design for as many nines
Raunak retweeted
Hundreds of millions of pages move through Reducto every week 📃 On October 27th, we’re bringing together the people who power the infrastructure behind them.
Join us at our HQ in San Francisco for a chat with Siddhant Pagariya and Lijie Zhou from our Infra Team. They will walk us through real projects and afterwards, we'll turn it over to you to try an interactive Capture the Flag with a live leaderboard.
Expect great food, drinks, and time to connect with fellow infrastructure engineers 🔮
Raunak retweeted
Railway is compromised, rotate all API keys as soon as possible
Deployments are down so don’t wait to redeploy
Attackers are draining keys in real time, we didn’t lose much but can confirm this is still an active attack
Stay safe friends
Raunak retweeted
deepseek posted way more alpha on their zhihu page, if you only read twitter you are reading the laowai approved stuff
"Based on this framework, we have implemented a high-throughput, low-latency large-scale inference service for DeepSeek V4.1 Flash. Currently, the service throughput of a single cluster is 155 trillion tokens/day, and the overall revenue is $38 per card per hour (based on off-peak prices)."
it costs roughtly $5-10 per gpu hour so depseek is claiming 80% serving margin at THEIR prices lololololol
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Raunak retweeted
“Why not do this for everything?” is exactly the founding question for our company
We know what the verification loop is for HTML parsers, but what is it for… tax software? robotics control systems? UX researcher agents? Construction AR? niteshift.dev
Raunak retweeted
Just heard from my coworkers (@jeanghislainbil and @ani_0203_ ) that r-1 is running at 500-800 decoding tps per request.
On simple pages, p50 e2e latency is ~800ms.
We are serving r-1 on Nvidia chips.
Give it a try!
Raunak retweeted
what does "good" document processing even mean?
benchmarking in this space can mean a lot of things:
> accuracy on complex docs
> latency
> throughput for scale
we're not here to tell you what set of documents is a "definitive benchmark", because your own docs are the ones that matter. the more important part is actually deciding how you measure performance.
@archanaahlawat and @jannalulu_ have been working a lot on building and tuning our in-house judge framework, and turns out there's a couple of reoccurring patterns we see that might help you with your eval process:
Raunak retweeted
Jev doesn't support image input yet, so I tested out some open-source Jev-like alternatives.
Seems like DiffusionGemma and reflex are on the pareto for the open source Vision-Jevs!
github.com/vllm-project/vllm… (ty @mmastrac)
github.com/kshetrajna12/refl… (ty @kshetrajna )
"DiffusionGemma as Jev" showcases the power of non-autoregressive architectures.
While Jev demonstrates the value of rapid decision models, running DiffusionGemma in this paradigm leverages canvas diffusion to evaluate structured choices in a single parallel pass:
⚡ ️Massive Parallelism: Denoises across an open canvas in a single step instead of sequential autoregressive token generation (~0.2s on a DGX spark).
🧠 Full Bidirectional Attention: Allows every option to attend to the full context concurrently, yielding well-calibrated decision distributions.
👁️ Multimodal Grounding: Inherits Gemma 4's spatial vision capabilities for complex visual and text decisions.
Read more about this approach here:
github.com/vllm-project/vllm…
xitter.twitterapi.xyz/mmastrac/status/210037…
xitter.twitterapi.xyz/mmastrac/status/210062…
Raunak retweeted
with instinct, you can now sign up for @reductoai and use r-1 to find out how bad your finances are in 20 seconds