Dwarkesh Podcast sponsors and ad reads

Jane Street's back with a new puzzle for my audience. I found all their puzzles super interesting, but this one I am especially excited about. I've cleared this weekend and a buddy and I are going to work on it. They designed an ASIC and sent me the final masks, including all the metal routing and active transistors. They also gave me a small sample of the inputs they typically feed into it, but they left out any information on what the chip is actually used for. So that's the puzzle. Reverse engineer the circuit and figure out the chip's purpose. Jane Street has a bunch of swag ready to send out to the most creative solutions, and they're excited to feature the best write-ups in a blog post they'll post on their website. I have no reason to expect this, but if I can manage to get my solution on there, I would be very, very psyched. And this puzzle is just a warmup for a bigger competition that Jane Street has slated for the fall. That one will involve designing your own ASIC from scratch. More info on that soon, but for now, go to janestreet.com slash thawarkash to download all the files necessary for this puzzle. I'd really encourage you to try it out even if you're not an expert. I certainly am not and that's not going to stop me.

Crusoe gave us early access to their serverless fine-tuning product, which lets you fine-tune open models without having to deal with infra or provisioning. I thought it'd be cool to try fine-tuning a question generator using the transcripts of my old interviews. The models have gotten so good that if they had all my research and prep and they could look at a conversation so far, they could ask a next question better than I would. Crusoe made the implementation super straightforward. I just uploaded the data, picked an open model, and started the run. I didn't have to touch any of the hyperparameters. Crusoe's Applied AI team maintains optimal recipes for each model, so I just set everything on auto. When the run finished, I deployed it as a self-server endpoint and built an eval for my team. I had them choose the best next question out of three anonymized choices, one that was produced by the base model, one that was produced by the fine-tuned model, and one that I actually asked. Fortunately, my team preferred my actual questions about two-thirds of the time. Hopefully, this benchmark doesn't saturate. And in the remaining cases, they almost always preferred the fine-tuned model over the base model. Serverless inference is live now, and serverless fine-tuning goes live next week. Learn more at crusoe.ai.com.

I recently wrote this blog post where I speculated that sample efficiency during training actually hasn't improved that much over the last few years. And rather, we've just dramatically improved and widened the data distribution. And I was having dinner with friends recently, and then I had this idea of how you could get some empirical information on this question. There's this nano-GPT speedrun where people compete to train Karpathy's GPT-2 baseline to a fixed loss with less and less compute. The training data is frozen, so I wondered if the loss curves over time of each record could tell you roughly how fast sample efficiency is improving. So I pulled out my phone, I dumped this idea into a voice note in the cursor app, and I went back to dinner. And then I got a notification about 15 minutes later, the cursor agent had cloned the modded nano GPT repo, it had analyzed all the loss curves for all the records, and it had estimated that sample efficiency had been improving about two to five X every single year. Of course, this is very naive and circumstantial evidence, but it inspired me to start writing a full post with a friend where we investigate this question using many different methods. And the friction really mattered here. The idea would have just floated away if I wasn't able to just kick off the investigation right then and there with the Cursor app. If you want to try Cursor's iOS app, go to cursor.com slash thewarkesh.

I don't think people appreciate the kinds of things that these models can just go handle for you when you equip them with a good harness like Cursor. For example, I started publishing my episodes on Bilibili for a burgeoning Chinese audience, but everything I upload there needs the sponsored segments cut out. Normally, that would have meant asking my editors to go back through all the old episodes, cut out the ads, and re-export everything. But in about just as much time as it would have taken me to send them that Slack message, I can just tell Cursor to do it instead and spare them. For research for the podcast, I have a whole repo where I've put every single book and paper relevant to prepping for any of the recent episodes. I've been able to hodgepodge everything because the Cursor harness is extremely good at helping the model figure out exactly what information to pull, whether that's from my repo or from the web, in order to answer the questions I have while doing research. So whatever you happen to be working on right now, just try pointing Cursor at it. Go to cursor.com/lorkash to get started.

Anybody sends an invoice to that address, Mercury automatically downloads it, scans it, and extracts all the relevant information, things like the contractor name, address, payment amount, invoice number, and due date, and then uses all of this to create a draft payment. Mercury then stores a list of these drafts for me to review. I just go through this list and double check that I've been billed correctly. I don't have to track anything or enter any information myself. Mercury does all the fundamental things for your business extremely well, and it puts them all in one place. If you wanna learn more, go to mercury.com. Mercury is a fintech company, not an FDIC insured bank. Banking services provided through Choice Financial Group and Column NA members FDIC.