Yi-6B

Created at 12.01.2024 14:21

General assessment: 0.354

The table will scroll to the left

Task name Result Metric
BPS 0.469 Accuracy
LCS 0.112 Accuracy
RCB 0.333 / 0.167 Avg. F1 / Accuracy
USE 0.023 Grade Norm
RWSD 0.496 Accuracy
PARus 0.514 Accuracy
ruTiE 0.505 Accuracy
MultiQ 0.079 / 0.051 F1-score/EM
ruMMLU 0.487 Accuracy
CheGeKa 0.008 / 0 F1 / EM
ruModAr 0.416 Accuracy
SimpleAr 0.951 Accuracy
ruMultiAr 0.189 Accuracy
MathLogicQA 0.382 Accuracy
ruHumanEval 0.003 / 0.015 / 0.03 pass@k
ruWorldTree 0.541 / 0.542 Avg. F1 / Accuracy
ruOpenBookQA 0.59 / 0.588 Avg. F1 / Accuracy

Evaluation on diagnostic datasets:

It is not taken into account in the overall rating

The table will scroll to the left

Task name Result Metric
ruHHH

0.483

  • Honest: 0.492
  • Harmless: 0.483
  • Helpful: 0.475
Accuracy
ruHateSpeech

0.562

  • Women : 0.556
  • Man : 0.714
  • LGBT : 0.588
  • Nationality : 0.595
  • Migrants : 0.286
  • Other : 0.492
Accuracy
ruDetox
  • 0.134
  • 0.372
  • 0.558
  • 0.365

Overall average score (J)

Assessment of the preservation of meaning (SIM)

Assessment of naturalness (FL)

Style Transfer Accuracy (STA)

ruEthics
Correct God Ethical
Virtue 0 0 0
Law 0 0 0
Moral 0 0 0
Justice 0 0 0
Utilitarianism 0 0 0

Table results:

[[0, 0 , 0, 0 , 0],
[0, 0 , 0, 0 , 0],
[0, 0 , 0, 0 , 0]]

5 MCC

Information about the submission:

Team:

MERA

Name of the ML model:

Yi-6B

Link to the ML model:

https://huggingface.co/01-ai/Yi-6B

Additional links:

https://github.com/01-ai/Yi

Architecture description:

The Yi series models follow the same model architecture as LLaMA. Up to 200k context window.

Description of the training:

-

Pretrain data:

Trained on 3T multilingual corpus.

Training Details:

-

License:

Apache 2.0 license

Strategy, generation and parameters:

Code version v.1.1.0 All the parameters were not changed and are used as prepared by the organizers. Details: - 1 x NVIDIA A100 - dtype auto - Pytorch 2.1.2 + CUDA 12.1 - Transformers 4.36.2 - Context length 4096