Evaluation info
The evaluation for TalentCLEF-2025 will be conducted on Codabench. Submissions will be ranked using Mean Average Precision (MAP).
TalentCLEF Task A - Codabench
TalentCLEF Task B - Codabench
To Be Published
Evaluation dates
- Start Date: 21st April 2025
- End Date: 5th May 2025
Both Task A and Task B will be evaluated during this period.
Evaluation Criteria
The top-performing teams will be determined based on the following evaluation criteria:
Task A:
- Best Overall Multilingual Performance – The highest-performing system across English, Spanish, and German, measured as the average Mean Average Precision (MAP) in
en-en,es-es, andde-de. - Best Cross-Lingual Performance – The best-performing system in cross-lingual scenarios, calculated as the average MAP in
en-esanden-de. - Best Bias-Controlled Model – The system that minimizes performance differences across job titles in different gender groups.
Additionally, there will be a special mention for the best-performing model in Chinese. During the CLEF workshop, certificates will be awarded to the first and second-best systems in each category.
- Best Overall Multilingual Performance – The highest-performing system across English, Spanish, and German, measured as the average Mean Average Precision (MAP) in
Task B: The highest-performing system based on MAP.