Pinned Tweet
Lately, I have been trying to beat Google.
Ok not really xD
I have been building a search engine over niche web data and it's been one of the most fun projects I've ever worked on.
The idea of querying the internet exactly the way I want, at scale, has always fascinated me. Especially what I could do with it: look at 1,000 companies' job postings and find hiring trends using natural language, map competitive landscapes including the lesser known players, find alternative data for investing. Or just a personalized corner of the web, curated for me.
It's been an interesting journey.
Started out with 200 workers on localhost, rewrote the crawler from Python to Rust to get up to 1,600 workers in parallel. Moved it to the cloud. Hit challenges that made the whole Rust rewrite useless xD
Then I woke up to an abuse report from a French website with the words "European Commission", "appeal", "court" in it. For a second I genuinely thought I was about to have legal proceedings against me. Phew. Nevertheless it's been the most fun project I've worked on. It's surprising how you can cover a lot of information starting from a modest number of seed URLs.
I'll be sharing everything I learned in the open. Code will be open sourced soon and the search engine will be live too.
If you're curious how to build a distributed web crawler that covers 10M+ pages under $200/month, do read my blogpost.
Blogpost Link: lnkd.in/g7MsaXYA
hrishikesh kamath retweeted
We looked at existing copy trading behavior on Hyperliquid across a few apps and 170k wallets and found that only 33% of wallets are in profit on their copied trades. Article dives into why.
hrishikesh kamath retweeted
Our first research article dives into copy trading on Hyperliquid and informs how we are building the feature in Tailor.
We believe that starts with lower fees, helping users discover the best traders to copy trade using in house quantitative models, and building in risk controls to avoid just one bad copy trade from liquidating an account.
hrishikesh kamath retweeted
AWS as an RL environment 👀
when agents train in RL environments that actually look like the real world, they transfer better to the real world
this looks pretty amazing! the best way to have product adoption in the future is to make sure agents are good at using your product
so naturally every team will create copies if their product so agents rock at using them
We just shipped AWS Emulator, which gives coding agents a local copy of AWS to test against.
Early in my career, we built test environments like this by hand, one system at a time. Now, agents are writing code far faster than any team can check it in production. Matt Goldberg, a Senior Principal Engineer on our Developer Tools team, wanted to know whether agents could build and test those environments themselves. He gave @kirodotdev a single prompt, and about 15 minutes later it came back with a first version of an SQS emulator. Turing that into a robust solution took much more work
Here’s what I think this means for engineers today: allthingsdistributed.com/202…
"If you don’t own the model (bytedance, google), the durable business is the infrastructure not the wrapper imo" interesting
It’s my job to deal with shade :)
Short version: in video, the frontier is now closed models. If you don’t own the model (bytedance, google), the durable business is the infrastructure not the wrapper imo.
1. We’d had this live quietly for a while (I soft-launch, then amplify). Customer quality is much higher, and we now have enough data to be bullish on the curve and NRR. We’ve also started working with closed model providers directly. Once you’re doing that, running an end-user app on top of their models is a different business, and focus matters at a startup.
2. After seeing a (very) early Seedance 2 video, it took me about a month to figure out what to do. Omnia was strong for what we spent, roughly Veo 3 class, but not SD2. What we did have was the pipeline: for our July 2025 realtime model, we built autonomous distillation and eval when distillation was still unstable.
3. App-layer margins are getting squeezed from three directions (and IMO it's all getting absorbed into muse/chatgpt/claude long term): the shift to closed models, a flood of thin wrappers, and UGC CAC skyrocketing while NRR stays structurally lower than in other sectors. We never leaned on paid media, and now even that CAC is brutal. In contrast, infra & research demand is vertical.
1
132
People who think this will beat $MSFT have no idea why Microsoft still makes billions selling its Office suite, even though $GOOG has offered Docs, Sheets, and Slides for free all these years.
Starting to roll out today: Claude Whiteboards, Claude Sheets and Claude Data.
Think a plan through on a board, build a tracker in a sheet, and tell a story with your data.
160
hrishikesh kamath retweeted
hot take: most "autonomous trading agents" are a stranger with your wallet
tailor isn't a stranger. it remembers what you hold, how you trade, and what you'd never touch.
now it executes on its own, by your rules.
it doesn't trade for you. it trades like you
Where are Claude docs/editable artifacts ? Or having to edit artifacts ? Now I am juggling notion, chat and artifacts to plan and do things.
Starting to roll out today: Claude Whiteboards, Claude Sheets and Claude Data.
Think a plan through on a board, build a tracker in a sheet, and tell a story with your data.
1
3
227
hrishikesh kamath retweeted
It's so weird that people don't understand that marketplaces just don't aggregate "demand" from users, they resolve disputes when something goes south and that's why they're valuable as an arbiter.
marketplaces will collapse
first - marketplaces that are just aggregators like airbnb in the next 365 days
second - then marketplaces with some ops like delivery, stores like amazon, zomato in the next 1000 days
from first principles, marketplaces aggregated demand for their users, but if an agent like muse, @iradotso or @instinct can do the same without charging you 17% , why would that survive?
@amazon @Airbnb and likes are on a downward slope and it's a steep slope
“Opus 5.5 seems dumber, I sense that they are releasing another model”, 1 week later:
Introducing Claude Fable 5.2 and Claude Mythos 5.2, designed to tackle the most daunting technical challenges.
They’re capable collaborators on ambitious projects in many fields, and show improvements in areas like coding, reasoning, and vision.
2
173
Given how the public has been reacting to Temple recently, it’s a great idea to raise money from people who understand what it means to invest in early-stage, high-risk, high-reward startups. That’s usually not the general public.
Rich get richer because only the inner coterie has access to the best companies. This model has been taking to the extreme in US with Stripe, SpaceX, OpenAI etc creating trillions in value for the rich before public gets access. We should avoid this path in India
94
hrishikesh kamath retweeted
I love when AI makes mistakes; it reminds me why I still have a role
I love when AI doesn’t make mistakes; it lets me increase the stakes
hrishikesh kamath retweeted
One of those companies hit $50M ARR
(Some big retailers are their clients)
Reminder that just because Google can casually drop a “labs” feature doing exactly what your entire company does, doesn’t meant they will actually seriously compete with you in the industry.
Wow this still exists, I knew a company called OpenAI existed in 2018-19 only because of this.
1
2
205
hrishikesh kamath retweeted
The forward P/E obsession is absolutely bizarre.
Analyst earnings forecasts peak right before cyclical downturns. In tech/semis, hyperscaler capex has created one-time revenue spikes, masking real valuation risk and making concentrated indexes look deceptively cheap.
Was trying to understand how much AI adoption has happened and where there’s still room to grow.
Interesting how coding and AI assistants are so far ahead of almost every other use case. Like way ahead, didn’t think the magnitude was like 10x. (Source: @MenloVentures)
194
The play is that Baseten is asset-light and could become a more horizontal platform. Over time, securing compute might become less of a competitive advantage.
If AI is going to be as big as everyone expects, with so many players competing, won’t brand, product and engineering matter more in the long run?
Also, public and private markets value things differently. Is private capital already pricing in a future where securing compute is no longer the biggest problem?
Just a hypothesis though.
1
174