Arxiv Dives with Oxen.AI - RWKV-7 "Goose" 🪿 + Q&A with Eugene Cheah
Hey Nerd, join the Herd!... for a little book/paper review.
WHAT TO EXPECT
Each week we pick a paper to cover in depth and have open Q/A. Often joined by paper authors themselves! Reading is optional 🙃.
We present RWKV-7 "Goose", a new sequence modeling architecture, along with pre-trained language models that establish a new state-of-the-art in downstream performance at the 3 billion parameter scale on multilingual tasks, and match current SoTA English language performance despite being trained on dramatically fewer tokens than other top 3B models.
Want to start nerding out? Grab the paper here...
TRY OUR NEW NOTEBOOKS FEATURE WITH FREE CREDITS
Oxen.ai makes it easier than ever to compute ebeddings... at 1,000 rows per second in our embedded Marimo notebooks... cheaply spun up on a GPU, all from your browser.
Want to experiment mooore fast? How about some 🆓 free bonus credits! To get yours...
$10 if you sign up ➡️ here
We'll double it if you tag us in a tweet with your own experiment image or RT our post.
💬 JOIN THE CONVO
Discord here to share paper recs and more community discussion.
SEE PAST SESSIONS
To see past topics head over to our blog which has show notes and links to Youtube videos.
🤓🐂🤓🐂🤓🐂🤓🐂🤓🐂🤓🐂🤓🐂🤓🐂🤓🐂🤓🐂
WHO'S AN ARXIV DIVER
1.4k+ in Discord and 5.3k+ on Youtube - we've been joined by folks from around the world including leaders from:
and many more...
Sign up
We share datasets relevant to these sessions via Oxen.ai. To get free data:
About Oxen.ai: Build World-Class AI Datasets, Together. Track, iterate, collaborate on, & discover data in any format.
About Arxiv Dives
Each week we dive deep into a topic in machine learning or artificial intelligence. We break down the content into a digestible format and have an open discussion with the Oxen.ai community. Read more in our Arxiv Dive Manifesto.