Arxiv Dives with Oxen.AI - RWKV-7 "Goose" ๐ชฟ + Q&A with Eugene Cheah
โโHey Nerd, join the Herd!... for a little book/paper review.
โโWHAT TO EXPECT
โโEach week we pick a paper to cover in depth and have open Q/A. Often joined by paper authors themselves! Reading is optional ๐.
โWe present RWKV-7 "Goose", a new sequence modeling architecture, along with pre-trained language models that establish a new state-of-the-art in downstream performance at the 3 billion parameter scale on multilingual tasks, and match current SoTA English language performance despite being trained on dramatically fewer tokens than other top 3B models.
โWant to start nerding out? Grab the paper here...
โTRY OUR NEW NOTEBOOKS FEATURE WITH FREE CREDITS
โOxen.ai makes it easier than ever to compute ebeddings... at 1,000 rows per second in our embedded Marimo notebooks... cheaply spun up on a GPU, all from your browser.
โWant to experiment mooore fast? How about some ๐ free bonus credits! To get yours...
โ$10 if you sign up โก๏ธ here
โWe'll double it if you tag us in a tweet with your own experiment image or RT our post.
โ๐ฌ โJOIN THE CONVO
โDiscord here to share paper recs and more community discussion.
โโSEE PAST SESSIONS
โโTo see past topics head over to our blog which has show notes and links to Youtube videos.
๐ค๐๐ค๐๐ค๐๐ค๐๐ค๐๐ค๐๐ค๐๐ค๐๐ค๐๐ค๐
โโWHO'S AN ARXIV DIVER
โ1.4k+ in Discord and 5.3k+ on Youtube - we've been joined by folks from around the world including leaders from:
โโand many more...
โโSign up
โโWe share datasets relevant to these sessions via Oxen.ai. To get free data:
โโAbout Oxen.ai: Build World-Class AI Datasets, Together. Track, iterate, collaborate on, & discover data in any format.
โโAbout Arxiv Dives
โโEach week we dive deep into a topic in machine learning or artificial intelligence. We break down the content into a digestible format and have an open discussion with the Oxen.ai community. Read more in our Arxiv Dive Manifesto.