Asia Paper Club : ROPE

​This week we've got @adtygan covering the ROPE paper ( https://arxiv.org/abs/2104.09864 )

​If you've ever wondered how we're achieving such massive context windows today or what suceeded the simple positional encodings of the old transformer model, this is the event for you!