phuongntc/Multi_EvalSumViet3

Multi_EvalSumViet2 (hierarchical Vietnamese summary evaluator)

What is this?
A Vietnamese multi-criteria evaluator that predicts three scores in [0,1]:

• faithfulness
• coherence
• relevance

for a (document, summary) pair.

Architecture (high-level).

• Vietnamese encoder backbone: Fsoft-AIC/videberta-base
• Hierarchical encoding of the document into overlapping segments, with attention-based aggregation over segments.
• Shared MLP trunk (256-dim) + three scalar heads (one per criterion).

Training (short).

• Multi-task regression objective (MSE) on 3 criteria with scores normalized to [0,1].
• Jointly trained with an in-document pairwise ranking loss (hinge) to encourage correct ordering of summaries within the same document.
• Tokenization: hierarchical doc segmentation + pair(doc_segment, summary) with max lengths:
  • sum_max_len = 256
  • pair_max_len = 512

See:

• arch_config.json – backbone + hierarchical settings + trunk/heads
• training_args.json – optimization & data-related hyper-parameters
• loss_config.json – loss configuration
• package_versions.json – Python / PyTorch / Transformers / Lightning versions

Files

• config.json, model.safetensors, tokenizer.*: encoder backbone + tokenizer
• trunk.pt, head_faith.pt, head_coh.pt, head_rel.pt, seg_attn.pt: task-specific layers
• arch_config.json, training_args.json, loss_config.json, package_versions.json
• modeling_multievalsumviet2.py: hierarchical loader & tokenization helpers

Quickstart

import os, importlib.util, torch

repo_dir = "/path/to/local/Multi_EvalSumViet2_export"  # hoặc snapshot_download(...)
spec = importlib.util.spec_from_file_location(
    "multievalsum", os.path.join(repo_dir, "modeling_multievalsumviet2.py")
)
mse = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mse)

docs = ["Văn bản gốc ..."]
sums = ["Bản tóm tắt ..."]

scores = mse.predict_scores(repo_dir, docs, sums)
print(scores)  # tensor [B, 3] với 3 tiêu chí trong [0,1]