Arabic prompts and benchmarks

Test Arabic with real work, not slogans.

Start with prompts that look like the work your team actually does. Qabas will publish benchmark scores only after running and documenting the tests itself.

Three useful starting prompts

Office task

Write a clear email

اكتب رسالة بريد إلكتروني رسمية وقصيرة لتأكيد موعد اجتماع يوم الأحد الساعة العاشرة صباحاً بتوقيت دبي.

Customer service

Use a Gulf-friendly tone

رد على العميل بلغة عربية واضحة وودية. اعتذر عن التأخير واشرح أن الطلب سيصل غداً.

Document work

Summarise without adding facts

لخّص النص التالي في خمس نقاط. لا تضف أي معلومة غير موجودة في النص، واذكر أي شيء غير واضح.

Arabic and English in one request

For technical teams, it is often clearer to keep an English term and explain it in Arabic.

{
  "model": "qwen-2.5-7b",
  "messages": [
    {
      "role": "user",
      "content": "اشرح API rate limit بالعربية البسيطة، واحتفظ بالمصطلح الإنجليزي بين قوسين."
    }
  ]
}

Qabas benchmark results are not published yet

Vendor scores are useful background, but they are not Qabas results. We are preparing independent tests for the exact model and service configuration customers use.

Test	What it helps measure	Status
ArabicMMLU	Knowledge and reasoning in Modern Standard Arabic.	Planned
DialectalArabicMMLU	Performance across Arabic dialects, including Gulf varieties.	Planned
AraTrust	Safety, truthfulness and trustworthy behaviour in Arabic.	Planned
Qabas operational set	Emails, support replies, summaries and mixed Arabic-English work.	In design

What every published result must include

The exact model revision and serving configuration
Hardware, quantisation, context length and test date
Prompt set, sample count and scoring method
Time to first token, tokens per second and p50/p95 latency
Known limitations and a link to raw results where possible