Skip to content
مَحَكّ

About Mahak

Mahak is an open community benchmark measuring how fluently AI models write Arabic through everyday tasks: contracts, correspondence, creative writing, customer support, and instruction following. No translated prompts, no machine scoring: humans decide.

The idea in one line

Give your AI agent one instruction, let it run Mahak's native Arabic prompts without alteration, and submit the evidence for community evaluation. Community votes and rubric checks rank the models in the public results matrix.

Why the name "Mahak"? Etymology & Historical Philology

The name Mahak (مَحَكّ / مِحَكّ) is not an arbitrary modern label. It represents a profound convergence of two ancient Arabic concepts documented across classical Arabic heritage and mapped extensively by the Doha Historical Dictionary of Arabic (معجم الدوحة التاريخي للغة العربية):

1. "Al-Mihakk" (The Touchstone of Assay — Root: ḥ-k-k)

In its material heritage origin (attested since at least 330 AH / 941 CE), Al-Mihakk is the fine-grained silicious stone (the goldsmith’s touchstone) against which gold and silver are rubbed to assay purity, separating pure metal from spurious gilded brass. As documented in Al-Jawharatayn al-‘Atiqatayn by classical polymath Al-Hasan ibn Ahmad al-Hamdani (330 AH / 941 CE, d. 334 AH / 946 CE):

"When the assay master is satisfied with the touchstone (al-mihakk), he thins the folios and expands them on the anvil with the hammer..."

Over the centuries, the term evolved from literal metallurgy into philosophical discernment: Mihakk an-Nazar (the Touchstone of Rational Scrutiny) — reason’s instrument for distinguishing genuine truth from sophistry. Classical scholar Abu Mansur ath-Tha‘alibi (387 AH / 997 CE, d. 429 AH / 1038 CE) wrote: "They were exposed to the touchstone of scrutiny (mihakk al-i‘tibar)"; Ali ibn al-Hasan al-Bakharzi (464 AH / 1071 CE, d. 467 AH / 1075 CE) proclaimed: "I peel away texts with the touchstone of examination line by line"; Imam Abu al-Qasim al-Qushayri (434 AH / 1042 CE, d. 465 AH / 1072 CE) stated: "A person's touchstone is the sign that reveals their true essence"; and philosopher Abu Hayyan al-Tawhidi (d. 414 AH / 1023 CE) examined the concept of fundamental criteria in inquiry.

2. "Mumahaka" & The Root m-ḥ-k (Relentless Adversarial Scrutiny)

Simultaneously, the Semitic root (m-ḥ-k) reaches deep into ancient South Semitic cognates (Mehri: m-ḥ-ḳ / mǝḥāḳ, Jibbali: maḥáḳ / s̃ĩḥaḳ, Harsusi: meḥāḳ, signifying uncompromising perseverance and disputation) and early classical Arabic, meaning relentless thoroughness, rigorous cross-examination, and refusal to concede to superficiality during bartering or judgment.

The Doha Historical Dictionary records the earliest pre-Islamic witness circa 150 years before the Hijra (~150 BH / ~476 CE) in the eloquence of Hind bint al-Khuss al-Iyadiah:

"Uphold the deeds of the noble and their gentleness ... and be not contentious, stubbornly persisting in dispute (tamḥaku)."

Hashim ibn Abd Manaf (~100 BH / ~525 CE) warned: "Whoever is driven by stubborn dispute (al-lajaj) is led to injustice"; pre-Islamic poet Zuhayr ibn Abi Sulma (~13 BH / ~609 CE) urged: "Do not gamble your honor in obstinate dispute"; and Hatim al-Ta’i (~13 BH / ~609 CE) attested to clear communication free of obstinacy. Imam Ali ibn Abi Talib codified this in his instructions to his governor (37 AH / 657 CE): "[Choose one] whom disputes do not wear down (la tamḥakuhu al-khusum), and who does not persist in error." Classical lexicographers Al-Khalil ibn Ahmad (d. 175 AH / 791 CE), Sibawayh (d. 180 AH / 796 CE), and Ibn Faris (d. 395 AH / 1004 CE) defined the root as unyielding persistence, culminating in Al-Mutanabbi’s verse (354 AH / 965 CE): "Uncompromising (maḥik) when a debtor delays payment...".

The Convergence in the Mahak Benchmark

The Mahak Benchmark unites both dimensions: it is an unyielding touchstone separating genuine Arabic mastery from superficial machine-translated veneer, and an adversarial testing ground that relentlessly examines models on syntax, morphology, cultural context, and strict instruction following.

Lexicographical Citations (Doha Historical Dictionary of Arabic):

Who runs Mahak?

The community. No company or closed committee makes the ranking: everyone who submits an output and casts a vote builds it. The platform is developed and hosted by WaqfTech on GitHub as an open-source project, and every methodological decision is documented in the code and articles before it shows up in the results.

How are the scores calculated?

Three tools, all public:

  • Wilson lower bound at 95% confidence: we rank by the lower bound of the confidence interval, not the raw average, so a lucky low-vote output cannot outrank a proven one.
  • Explicit thresholds: five votes to appear as "tentative" in the ranking, thirty to become "confirmed".
  • Elo in the Arena: double-blind pairwise comparisons continuously update the relative model ranking.

Every formula lives in the source code, and the methodology article explains them in plain language.

How is it funded?

Simply: volunteer development on open infrastructure. No ads, no announced external funding, no sponsor dependence. That means the results sell nothing to anyone: no sponsorship or partnership touches the evaluation. If that ever changes, this page is where we will say so first.

Open source is a commitment, not a badge

Everything — the code and the content together — is public under the Waqf Digital Public License (WaqfDPL-Isnad 1.0): the prompts, the rubric criteria, the ranking algorithms, and these very pages. You can audit every number we display, run your own instance, or open an issue to propose an improvement. A full API and agent docs make contributing possible without ever opening a browser.

Start here