ruGPT-3 XL 1.3B

ai-forever Created at 27.03.2026 21:59
0.198
The overall result
594
Place in the rating
Weak tasks:
468
RWSD
543
PARus
607
RCB
609
MultiQ
612
ruWorldTree
619
ruOpenBookQA
391
CheGeKa
597
ruMMLU
425
ruHateSpeech
466
ruDetox
536
ruHHH
550
ruTiE
580
USE
552
MathLogicQA
560
ruMultiAr
585
SimpleAr
247
LCS
523
BPS
573
ruModAr
601
MaMuRAMu
+16
Hide

Ratings for leaderboard tasks

The table will scroll to the left

Task name Result Metric
LCS 0.116 Accuracy
RCB 0.324 / 0.243 Accuracy F1 macro
USE 0.028 Grade norm
RWSD 0.488 Accuracy
PARus 0.5 Accuracy
ruTiE 0.502 Accuracy
MultiQ 0.034 / 0.014 F1 Exact match
CheGeKa 0.036 / 0.017 F1 Exact match
ruModAr 0.001 Exact match
MaMuRAMu 0.262 Accuracy
ruMultiAr 0.011 Exact match
ruCodeEval 0 / 0 / 0 Pass@k
MathLogicQA 0.266 Accuracy
ruWorldTree 0.246 / 0.224 Accuracy F1 macro
ruOpenBookQA 0.238 / 0.231 Accuracy F1 macro

Evaluation on open tasks:

Go to the ratings by subcategory

The table will scroll to the left

Task name Result Metric
BPS 0.528 Accuracy
ruMMLU 0.252 Accuracy
SimpleAr 0.012 Exact match
ruHumanEval 0 / 0 / 0 Pass@k
ruHHH 0.478
ruHateSpeech 0.558
ruDetox 0.082
ruEthics
Correct God Ethical
Virtue -0.006 0.013 0.024
Law 0.003 -0.01 0.037
Moral -0.004 -0.013 0.017
Justice -0.044 -0.064 0.02
Utilitarianism 0.004 0.018 0.015

Information about the submission

Mera version
v1.2.0
Torch Version
2.11.0
The version of the codebase
2a537dd
CUDA version
13.0
Precision of the model weights
float16
Seed
1234
Batch
1
Transformers version
4.57.6
The number of GPUs and their type
1 x NVIDIA GeForce RTX 4090 48Гб
Architecture
hf

Team:

ai-forever

Name of the ML model:

ruGPT-3 XL 1.3B

Model size

1.3B

Model type:

Opened

Pretrain

Additional links:

https://habr.com/ru/articles/1016148/

Architecture description:

A 1.3B-parameter GPT-2-style language model for Russian, converted from the original ai-forever/rugpt3xl Megatron-LM checkpoint into a native HuggingFace transformers format.

License:

MIT

Inference parameters

Generation Parameters:
simplear - do_sample=false;until=[" \n"]; \nchegeka - do_sample=false;until=[" \n"]; \nrudetox - do_sample=false;until=[" \n"]; \nrumodar - do_sample=false;until=[" \n"];do_sample=false;until=[" \n"]; \nrumultiar - do_sample=false;until=[" \n"]; \nuse - do_sample=false;until=[" \n","."]; \nmultiq - do_sample=false;until=[" \n"]; \nruhumaneval - do_sample=true;temperature=0.6;until=[" \nclass"," \ndef"," \n#"," \nif"," \nprint"]; \nrucodeeval - do_sample=true;temperature=0.6;until=[" \nclass"," \ndef"," \n#"," \nif"," \nprint"];

The size of the context:
2048

Ratings by subcategory

Metric: Grade Norm
Model, team 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 8_0 8_1 8_2 8_3 8_4
ruGPT-3 XL 1.3B
ai-forever
0 0 0 0 0 0 0 - 0.033 0 0 0.033 0 0 0 0.133 0.067 0 0.3 0 0 0 0 0 0 0 0.033 0.167 0.033 0 0.033
Model, team Honest Helpful Harmless
ruGPT-3 XL 1.3B
ai-forever
0.492 0.475 0.466
Model, team Anatomy Virology Astronomy Marketing Nutrition Sociology Management Philosophy Prehistory Human aging Econometrics Formal logic Global facts Jurisprudence Miscellaneous Moral disputes Business ethics Biology (college) Physics (college) Human Sexuality Moral scenarios World religions Abstract algebra Medicine (college) Machine learning Medical genetics Professional law PR Security studies Chemistry (школьная) Computer security International law Logical fallacies Politics Clinical knowledge Conceptual_physics Math (college) Biology (high school) Physics (high school) Chemistry (high school) Geography (high school) Professional medicine Electrical engineering Elementary mathematics Psychology (high school) Statistics (high school) History (high school) Math (high school) Professional accounting Professional psychology Computer science (college) World history (high school) Macroeconomics Microeconomics Computer science (high school) European history Government and politics
ruGPT-3 XL 1.3B
ai-forever
0.311 0.265 0.211 0.214 0.239 0.239 0.252 0.248 0.253 0.229 0.175 0.286 0.3 0.25 0.285 0.289 0.27 0.229 0.233 0.252 0.277 0.199 0.21 0.266 0.179 0.29 0.229 0.176 0.212 0.21 0.27 0.273 0.227 0.273 0.298 0.278 0.24 0.226 0.285 0.266 0.242 0.298 0.297 0.21 0.234 0.31 0.265 0.244 0.266 0.243 0.24 0.224 0.272 0.277 0.29 0.206 0.264
Model, team SIM FL STA
ruGPT-3 XL 1.3B
ai-forever
0.275 0.761 0.562
Model, team Anatomy Virology Astronomy Marketing Nutrition Sociology Managment Philosophy Pre-History Gerontology Econometrics Formal logic Global facts Jurisprudence Miscellaneous Moral disputes Business ethics Bilology (college) Physics (college) Human sexuality Moral scenarios World religions Abstract algebra Medicine (college) Machine Learning Genetics Professional law PR Security Chemistry (college) Computer security International law Logical fallacies Politics Clinical knowledge Conceptual physics Math (college) Biology (high school) Physics (high school) Chemistry (high school) Geography (high school) Professional medicine Electrical Engineering Elementary mathematics Psychology (high school) Statistics (high school) History (high school) Math (high school) Professional Accounting Professional psychology Computer science (college) World history (high school) Macroeconomics Microeconomics Computer science (high school) Europe History Government and politics
ruGPT-3 XL 1.3B
ai-forever
0.2 0.218 0.317 0.231 0.211 0.293 0.328 0.333 0.288 0.246 0.192 0.267 0.192 0.225 0.187 0.506 0.243 0.244 0.333 0.228 0.211 0.203 0.2 0.254 0.244 0.197 0.256 0.228 0.246 0.222 0.333 0.359 0.375 0.281 0.212 0.304 0.2 0.311 0.281 0.4 0.273 0.333 0.289 0.244 0.31 0.311 0.207 0.136 0.246 0.298 0.2 0.319 0.278 0.169 0.233 0.275 0.256
Coorect
Good
Ethical
Model, team Virtue Law Moral Justice Utilitarianism
ruGPT-3 XL 1.3B
ai-forever
-0.006 0.003 -0.004 -0.044 0.004
Model, team Virtue Law Moral Justice Utilitarianism
ruGPT-3 XL 1.3B
ai-forever
0.013 -0.01 -0.013 -0.064 0.018
Model, team Virtue Law Moral Justice Utilitarianism
ruGPT-3 XL 1.3B
ai-forever
0.024 0.037 0.017 0.02 0.015
Model, team Women Men LGBT Nationalities Migrants Other
ruGPT-3 XL 1.3B
ai-forever
0.519 0.714 0.471 0.595 0.286 0.574