RU EN
Return to the leaderboard

Random submission

MERA Created at 09.01.2024 08:34
0.205
The overall result
The submission does not contain all the required tasks

Ratings for leaderboard tasks

The table will scroll to the left

Task name Result Metric
LCS 0.096 Accuracy
RCB 0.361 / 0.36 Accuracy F1 macro
USE 0.064 Grade norm
RWSD 0.519 Accuracy
PARus 0.482 Accuracy
ruTiE 0.472 Accuracy
MultiQ 0.014 / 0.001 F1 Exact match
CheGeKa 0.002 / 0 F1 Exact match
ruModAr 0.0 Exact match
ruMultiAr 0.0 Exact match
MathLogicQA 0.244 Accuracy
ruWorldTree 0.23 / 0.229 Accuracy F1 macro
ruOpenBookQA 0.245 / 0.245 Accuracy F1 macro

Evaluation on open tasks:

Go to the ratings by subcategory

The table will scroll to the left

Task name Result Metric
BPS 0.5 Accuracy
ruMMLU 0.258 Accuracy
SimpleAr 0.0 Exact match
ruHumanEval 0 / 0 / 0 Pass@k
ruHHH 0.522
ruHateSpeech 0.468
ruDetox 0.382
ruEthics
Correct God Ethical
Virtue 0.013 0.026 -0.022
Law 0.014 0.029 0.016
Moral -0.01 -0.023 -0.017
Justice -0.038 -0.045 -0.053
Utilitarianism 0.014 0.044 0.019

Information about the submission:

Mera version
-
Torch Version
-
The version of the codebase
-
CUDA version
-
Precision of the model weights
-
Seed
-
Butch
-
Transformers version
-
The number of GPUs and their type
-
Architecture
-

Team:

MERA

Name of the ML model:

Random submission

Link to the ML model:

https://github.com/ai-forever/MERA

Architecture description:

Random submission. Basic low baseline to compare with.

Description of the training:

Random submission. For each task we randomly choose the result and score the variant.

Pretrain data:

No data.

Training Details:

No training.

License:

-

Strategy, generation and parameters:

-

Expand information

Ratings by subcategory

Metric: Accuracy
Model, team Honest Helpful Harmless
Random submission
MERA 		0.492 0.525 0.552
Model, team Anatomy Virology Astronomy Marketing Nutrition Sociology Management Philosophy Prehistory Human aging Econometrics Formal logic Global facts Jurisprudence Miscellaneous Moral disputes Business ethics Biology (college) Physics (college) Human Sexuality Moral scenarios World religions Abstract algebra Medicine (college) Machine learning Medical genetics Professional law PR Security studies Chemistry (школьная) Computer security International law Logical fallacies Politics Clinical knowledge Conceptual_physics Math (college) Biology (high school) Physics (high school) Chemistry (high school) Geography (high school) Professional medicine Electrical engineering Elementary mathematics Psychology (high school) Statistics (high school) History (high school) Math (high school) Professional accounting Professional psychology Computer science (college) World history (high school) Macroeconomics Microeconomics Computer science (high school) European history Government and politics
Random submission
MERA 		0.1 0.25 0.4 0.314 0.238 0.2 0.267 0.176 0.1 0.2 0.455 0.4 0.4 0.192 0.136 0.1 0.4 0.37 0.4 0.4 0.1 0.231 0.2 0.235 0.5 0.273 0.313 0.429 0.4 0.091 0.1 0.444 0.1 0.2 0.182 0.3 0.5 0.333 0.5 0.5 0.228 0.2 0.2 0.1 0.25 0.2 0.2 0.1 0.3 0.1 0.182 0.188 0.235 0.4 0.292 0.182 0.259
Model, team SIM FL STA
Random submission
MERA 		0.805 0.558 0.841
Coorect
Good
Ethical
Model, team Virtue Law Moral Justice Utilitarianism
Random submission
MERA 		0.013 0.014 -0.01 -0.038 0.014
Model, team Virtue Law Moral Justice Utilitarianism
Random submission
MERA 		0.026 0.029 -0.023 -0.045 0.044
Model, team Virtue Law Moral Justice Utilitarianism
Random submission
MERA 		-0.022 0.016 -0.017 -0.053 0.019
Model, team Women Men LGBT Nationalities Migrants Other
Random submission
MERA 		0.463 0.543 0.529 0.459 0.857 0.377

Пользовательское соглашение

We use cookies to store data. By continuing to use the site, you consent to work with these files