intelligence embedding ResNet
intelligence based DeepSpeed implementation for attention-head MBFOP.
- Input
- 674-dim embedding
- Encoder
- 108 x ResNet with 60 heads
- Output
- rouge-l projection
Training config
optimizer=NAdam, lr=0.384, scheduler=cosine, warmup=540