Frequently Asked Questions

What is Mahak?

Mahak is an open community platform measuring AI model competence in Arabic through practical everyday tasks: contracts, correspondence, creative writing, and customer support.

Is my data public?

Yes. Mahak is an open-source project. Prompts, outputs, and evaluations are all public and viewable by anyone. No private data is collected from users. The site uses Cloudflare’s cookieless analytics to measure traffic only.

How do I contribute?

Copy a prompt from the prompts page, run it in any AI model, then submit the output from the contribute page with run details. See the Contribution Guide for full details.

Why blind comparison?

Blind comparison removes bias toward a model’s name or reputation. In the arena, two models’ outputs are displayed without revealing their identities, and the evaluator votes for the better one based on output quality alone. This ensures fair evaluation.

How can I trust the scores?

The methodology is fully open. We use the Wilson score lower bound instead of a simple average to avoid inflating low-vote outputs. The Elo system uses different K-factors for new and calibrated models. All scoring formulas are available in the source code on GitHub.

What are rubric criteria?

Each prompt comes with specific criteria divided into positive (what should be present) and negative (what should not be present). Each criterion has a weight determining its importance. See Domains & Rubric Philosophy for details.

Can I evaluate others’ outputs?

Yes. Any community member can vote on outputs and evaluate them against rubric criteria. Every vote contributes to result accuracy.

What’s the difference between “provisional” and “calibrated”?

Cells with fewer than 5 votes are classified as “insufficient” to compute a reliable ranking. Between 5 and 29 votes, they are “provisional.” At 30 votes or more, they become “calibrated.” This progression ensures statistical accuracy and avoids inflating small-sample outputs.

How do I report an issue?

Open an issue on the project’s GitHub repository. We welcome bug reports and suggestions.