Kimi-K2-Instruct-0905

MERA Создан 28.10.2025 17:18
0.325
Общий результат

Оценки по задачам лидерборда

Таблица скроллится влево

Задача Результат Метрика
YABLoCo 0.077 / 0.029
EM pass@k
stRuCom 0.232
chrF
RealCode 0.188 / 0.986
pass@k execution_success
UnitTests 0.153
CodeBLEU
ruCodeEval 0.448 / 0.552 / 0.579
pass@k
JavaTestGen 0.233 / 0.507
pass@k compile@1
ruHumanEval 0.448 / 0.554 / 0.579
pass@k
RealCodeJava 0.342 / 0.987
pass@k execution_success
CodeLinterEval 0.423 / 0.548 / 0.573
pass@k
ruCodeReviewer 0.022 / 0.132 / 0.074 / 0.129 / 0.144
chrF BLEU judge@1 judge@5 judge@10
CodeCorrectness 0.571
EM

Информация о сабмите

Версия MERA
v1.0.0
Версия Torch
2.7.1
Версия кодовой базы
c965818
Версия CUDA
12.6
Precision весов модели
bf16
Сид
1234
Батч
1
Версия transformers
4.56.1
Количество GPU и их тип
8 x NVIDIA A100-SXM4-80GB
Архитектура
local-chat-completions

Команда:

MERA

Название ML-модели:

Kimi-K2-Instruct-0905

Ссылка на ML-модель:

https://huggingface.co/moonshotai/Kimi-K2-Instruct-0905

Размер модели

1,000.0B

Тип модели:

Открытая

SFT

MoE

Параметры инференса

Параметры генерации:
rucodeeval - do_sample=true;temperature=0.6;max_gen_toks=1024;until=[" \nclass"," \ndef"," \n#"," \nif"," \nprint"]; \ncodelintereval - do_sample=true;temperature=0.6;max_gen_toks=1024;until=[" \n \n"]; \nrucodereviewer - temperature=0;do_sample=false;max_gen_toks=1000;until=[" \n \n"]; \nruhumaneval - do_sample=true;temperature=0.6;max_gen_toks=1024;until=[" \nclass"," \ndef"," \n#"," \nif"," \nprint"]; \nstrucom - do_sample=false;max_gen_toks=512;until=[" \n \n"]; \ncodecorrectness - until=[" \n \n"];do_sample=false;temperature=0; \nrealcode - do_sample=true;max_gen_toks=4096;temperature=0.7;repetition_penalty=1.05;top_p=0.8;until=["<|endoftext|>","<|im_end|>"]; \nrealcodejava - do_sample=true;max_gen_toks=4096;temperature=0.7;repetition_penalty=1.05;top_p=0.8;until=["<|endoftext|>","<|im_end|>"]; \njavatestgen - do_sample=true;max_gen_toks=4096;temperature=0.2;top_p=0.9;until=["<|endoftext|>","<|im_end|>"]; \nyabloco_oracle - max_gen_toks=2048;do_sample=false;until=["<|endoftext|>","<|im_end|>"," \n \n \n","\sclass\s","\sdef\s","^def\s","^class\s","^if\s","@","^#"]; \nunittests - do_sample=false;max_gen_toks=1024;until=[" \n \n"];

Описание темплейта:
{% macro render_content(msg) -%} {%- set c = msg.get('content') -%} {%- if c is string -%} {{ c }} {%- elif c is not none -%} {% for content in c -%} {% if content['type'] == 'image' or 'image' in content or 'image_url' in content -%} <|media_start|>image<|media_content|><|media_pad|><|media_end|> {% else -%} {{ content['text'] }} {%- endif -%} {%- endfor -%} {%- endif -%} {%- endmacro %} {%- if tools -%} <|im_system|>tool_declare<|im_middle|>{{ tools | tojson(separators=(',', ':')) }}<|im_end|> {%- endif -%} {% for message in messages %} {%- if loop.first and messages[0]['role'] != 'system' -%} <|im_system|>system<|im_middle|>You are Kimi, an AI assistant created by Moonshot AI.<|im_end|> {% endif %} {%- set role_name = message.get('name') or message['role'] -%} {%- if message['role'] == 'user' -%} <|im_user|>{{role_name}}<|im_middle|> {%- elif message['role'] == 'assistant' -%} <|im_assistant|>{{role_name}}<|im_middle|> {%- else -%} <|im_system|>{{role_name}}<|im_middle|> {%- endif -%} {%- if message['role'] == 'assistant' and message.get('tool_calls') -%} {{render_content(message)}}<|tool_calls_section_begin|> {%- for tool_call in message['tool_calls'] -%} {%- set formatted_id = tool_call['id'] -%} <|tool_call_begin|>{{ formatted_id }}<|tool_call_argument_begin|>{% if tool_call['function']['arguments'] is string %}{{ tool_call['function']['arguments'] }}{% else %}{{ tool_call['function']['arguments'] | tojson }}{% endif %}<|tool_call_end|> {%- endfor -%} <|tool_calls_section_end|> {%- elif message['role'] == 'tool' -%} {%- set tool_call_id = message.tool_call_id -%} ## Return of {{ tool_call_id }} {{render_content(message)}} {%- elif message['content'] is not none -%} {{render_content(message)}} {%- endif -%} <|im_end|> {%- endfor -%} {%- if add_generation_prompt -%} <|im_assistant|>assistant<|im_middle|> {%- endif -%}