these Reka models look impressive

(sips tea)

these Reka models look impressive

(sips tea) 
https://www.reka.ai/news/reka-core-our-frontier-class-multimodal-language-model

- multimodal model integrates image, text, video, and audio, resembling Gemini
- it uses the Noam architecture, similar to T5, and incorporates sentinel tokens for masking
- model processes data with a significant mix: 25% code, 30% STEM, 10% math, and 25% web crawl, over an 8K sequence length.

- supports multilingual inputs, and features reverse instruction tuning.
- infrastructure includes 2.5K H100 and A100 GPUs, runs on PyTorch, and uses the Ceph filesystem.
- evals show strength in coding tasks.

technical report:

- supports multilingual inputs, and features reverse instruction tuning.
- infrastructure includes 2.5K H100 and A100 GPUs, runs on PyTorch, and uses the Ceph filesystem.
- evals show strength in coding tasks.

technical report: https://publications.reka.ai/reka-core-tech-report.pdf