RWKV

community

https://www.rwkv.com/

RWKV_AI

RWKV

Activity Feed

AI & ML interests

None defined yet.

Recent Activity

SmerkyG updated a model about 1 month ago

RWKV/v6-Finch-7B-World3-HF

SmerkyG new activity about 2 months ago

RWKV/v6-Finch-7B-HF:Update README.md

Hazzzardous authored a paper about 2 months ago

Lina-Speech: Gated Linear Attention is a Fast and Parameter-Efficient Learner for text-to-speech synthesis

View all activity

RWKV's activity

BlinkDL

posted an update 19 days ago

Post

1908

RWKV-7 "Goose" 0.4B trained w/ ctx4k automatically extrapolates to ctx32k+, and perfectly solves NIAH ctx16k 🤯 100% RNN and attention-free. Only trained on the Pile. No finetuning. Replicable training runs. tested by our community: https://github.com/Jellyfish042/LongMamba

SmerkyG

updated a model about 1 month ago

RWKV/v6-Finch-7B-World3-HF

Updated Dec 4, 2024 • 192

SmerkyG

in RWKV/v6-Finch-7B-HF about 2 months ago

Update README.md

#1 opened 4 months ago by

SmerkyG

Hazzzardous

authored a paper about 2 months ago

Lina-Speech: Gated Linear Attention is a Fast and Parameter-Efficient Learner for text-to-speech synthesis

Paper • 2410.23320 • Published Oct 30, 2024 • 8

BlinkDL

posted an update about 2 months ago

Post

4435

RWKV-6-world-v3 (+3.1T tokens) is our best multilingual 7B model as of now: BlinkDL/rwkv-6-world

It's 100% RNN and attention-free. MMLU 54.2% (previous world-v2.1 = 47.9%. note: without eval-boosting tricks such as annealing).

RWKV-7-world-v4 soon :)

m8than

in RWKV/v6-Finch-7B-HF 2 months ago

Adding `safetensors` variant of this model

#3 opened 2 months ago by

SFconvertbot

Upload modeling_rwkv6.py

#2 opened 3 months ago by

weili-0234

Hazzzardous

updated a model 2 months ago

RWKV/rwkv-5-world-1b5

Text Generation • Updated Apr 22, 2024 • 79 • 13

ybelkada

authored a paper 3 months ago

Falcon Mamba: The First Competitive Attention-free 7B Language Model

Paper • 2410.05355 • Published Oct 7, 2024 • 31

BlinkDL

posted an update 3 months ago

Post

5470

RWKV-7 "Goose" preview rc2 => Peak RNN architecture?😃Will try to squeeze more performance for the final release. Preview code & model: https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v7

2 replies

m8than

updated 6 models 4 months ago

xianbao

posted an update 4 months ago

Post

1740

With the open-weight release of CogVideoX-5B from THUDM, i.e. GLM team, the Video Generation Model (how about calling it VGM) field has officially became the next booming "LLM"

What does the landscape look like? What are other video generation models? This collection below is all your need.

xianbao/video-generation-models-66c350163c74f60f5c412af6

The above video is generated by @a-r-r-o-w with CogVideoX-5B, taken from a nice lookout for the field!