Research
Independent AI research in Abu Dhabi.
I am Syamjith NK, an independent AI researcher and developer based in Abu Dhabi, UAE. I evaluate Arabic language technology and publish the tools and evidence behind the findings. Every finding ships with an open test set, the scorer that produced it, the raw per-item results, and a plain statement of what the measurement does not establish.
When OCR changes the script, does it preserve the number?
A human-led evaluation of five OCR-capable models separates numeric value, numeral-script fidelity and joint success. The public preprint includes raw records, corrected scoring and calibration diagnostics. Stronger inferential claims are withheld because uncertainty calibration was inadequate. These synthetic probes are not production error rates.
Is the Arabic that AI models train on already broken?
The recipe that corrupts Arabic appears in 3,168 indexed Python files. If it ever ran before a corpus was saved, every model trained on that corpus learned from text no Arabic reader would accept. Nobody had measured whether it did.
276 datasets readable, 26,318 rows, 119,517 text fields. One stores contextual
glyph forms in every label sampled — 1,000 of 1,000, across ten windows
spanning its whole split — and it is an OCR set, so the affected field is the
ground‑truth label, which is the path from a rendering bug into model weights.
Only the shaping half of the recipe ran there, so word order is intact and
NFKC alone recovers the text; I first reported it as reordered too, which
was wrong, and the correction is on the dataset card.
Twenty more carry isolated stray glyph forms: not the same defect, but measured against
four real vocabularies, one stray form in a four‑letter word is enough for an
Arabic‑specific tokenizer to emit [UNK] and discard the word rather
than split it. To be precise about scale, that dataset has 17 downloads. This
shows the mechanism reaches training data. It is not evidence that widely‑used
Arabic corpora are affected.
65 datasets could not be read, and the viewer fails most often on the largest corpora,
so this covers the mid and small end well and the head of the distribution poorly. The
scanner ships with the results, so the numbers can be re‑derived rather than trusted.
The audit is deposited and citable:
doi:10.5281/zenodo.22733934, An audit of presentation-form corruption
in 341 public Arabic datasets, 13 September 2026 — so a paper can cite the
version it actually read rather than a repository that may have moved on.
arabic-corpus-audit
·
doi.org/10.5281/zenodo.22733934
·
arabic-lint (pip install arabic-lint)
One of these reads ٢٠٢٦ as a year. The other reads it as noise.
Arabic is written with two numeral systems and both appear constantly in real copy. The same sentence written with Arabic-Indic digits rather than Western drops from 11/15 (73%) to 1/15 (7%) recovered on fish in the archived automated evaluation. Apple scores 13/15 (87%) for each digit form after the documented 21 August 2026 scorer correction. These are recovery scores through TTS, Whisper and a number parser, not human listening accuracy; errors at any stage can affect the result.
The Arabic fix everyone recommends is now the thing breaking your Arabic
arabic_reshaper + python-bidi is the advice in nearly every
tutorial. On any renderer that already does complex-text layout it produces
مكب ابحرم where مرحبا بكم belongs, because
the work happens twice. 15/15 correct without it, 0/15 with it. Filed
upstream against a package downloaded 5.4 million times a month.
Your Arabic PDF is fine. What reads it is not.
A PDF looks right, so it is trusted — but almost nothing that consumes one looks at it. Portals parse it, ATS systems read it, compliance tools scan it. Across four generation routes and three extractors, reversed, ligature-mangled and space-collapsed are three distinct failure modes, not one.
Upstream contributions
The findings above did not stay on this site. Where one turned out to be a defect in a library rather than a fact about Arabic, it was filed against that library — which is the only version of this work anyone can check without taking my word for it, and the only place it can be rejected by someone who maintains the code.
nmt_nfkc is the default normaliser in SentencePiece, which sits under a
large share of language-model text pipelines. It mapped U+200E
LEFT-TO-RIGHT MARK and U+200F RIGHT-TO-LEFT MARK to a space, and left
U+061C ARABIC LETTER MARK in place. All three are zero-width invisible bidi
controls, and ALM is the Arabic-script counterpart of the other two, so two Arabic
strings that render identically tokenised differently: on google/mt5-base
the word carrying the invisible mark lost its word-initial piece and went from one piece
to three.
To be precise about who did what: I reported it, and I did not write the
fix. Taku Kudo, the author of SentencePiece, wrote and committed it three days later,
with a regression test covering all three marks. The argument that landed was consistency
rather than frequency: ALM is uncommon, the report says so and measures it, and what made
it a defect is that the normaliser already handled two characters of exactly this class.
And it is not in a release. The newest tag is v0.2.2 from
July 2026, and PyPI serves that same version, so the normaliser anyone installs today
still passes U+061C through. Beyond that, a trained SentencePiece model does
not consult the library's rule table at inference: the rules are compiled and embedded
into the .model file by design, so that training and inference normalise
identically. Models that already exist keep the table they were built with, and this fix
cannot reach them retroactively even once it ships.
google/sentencepiece #1331
· closed as completed · fixed in
c9e4719
· not yet in a release, newest is v0.2.2 from July 2026
pypdf extracted Arabic-Indic digits backwards. The codepoints
U+0660–U+0669 sat inside the library's right-to-left
character range, so each digit was prepended to the output instead of appended:
١٢٣٤ came out of a PDF as ٤٣٢١. Arabic
runs right to left; Arabic numbers do not, and that one-line distinction is the
whole bug. Reviewed, approved and merged by stefan6419846. The full
walk-through is
written
up here.
To be precise about the size: the library change is two lines. The other
81 are a new test.
And it shipped on 16 September in
pypdf 6.19.0,
whose changelog names the pull request and its author.
What this is, and what it is not: it is my first merged code
contribution. The matplotlib pull request below is a thirty-line documentation change, and
I would rather say that here than let a reader take the two for the same thing. What makes
this one worth listing is not its size — it is that a maintainer of a library with
over ten thousand stars read the diff, approved it and merged it. That is someone else's
judgement, not my own measurement of my own claim.
The listing that followed: pypdf's
CONTRIBUTORS.md
now carries my name against “Arabic and RTL text handling”, with a link to
this site. One line in a file, and worth listing only because it is a page on somebody
else's project rather than a page of mine.
py-pdf/pypdf #4077 · merged · released in 6.19.0, 16 September 2026 · listed in CONTRIBUTORS.md
matplotlib 3.11 shapes Arabic itself. That silently reverses the
arabic_reshaper + python-bidi workaround that every tutorial,
Stack Overflow answer and blog post still recommends — no error, no warning, just
backwards labels on a chart nobody who reads Arabic was asked to check. The upgrade guide
now tells 3.11 upgraders to remove that workaround.
To be precise about what was merged: the pull request is a documentation
change, thirty lines. The work behind it was establishing that the reversal happens at
all, that it is unconditional, and that the fix everyone recommends had become the bug.
Three maintainers, including the project lead, engaged with the finding.
matplotlib #32263 · merged
The same class of defect one layer down: under the Raqm layout engine, pre-shaped Arabic is corrupted rather than drawn. Compounded by a trap that costs people days — whether Raqm is active is a property of the runtime, not of the Pillow version, so the identical wheel behaves differently on two machines depending on the host libraries. The patch warns at run time and documents that the layout engine is a build-time decision.
Pillow #9925 · open, 8 of 8 checks passing, no maintainer response since 5 September
A fuller account — including four bug reports filed against projects that ship the broken recipe, of which two were confirmed and fixed by a maintainer and two have had no reply at all — is on the work and evidence page.
Method
Findings come out of production, not a lab: each began with Arabic breaking in delivered work. Test sets are small and public, scorers are published alongside the results, and every claim is reproducible from the repository.
In two of the three benchmark studies the measurement contradicted the assumption it started from, and the write-ups say so. In one, a pass/fail scorer reported a broken configuration as working; adding a middle band is what surfaced the real result. Designing a test that can prove you wrong is most of the work.
Corrections
When four faults were found in the scoring code behind the speech study, a documented correction was published with old-to-new figures per cell rather than a silent revision. A measurement that quietly changes its numbers is worth less than one that never published them.
Elsewhere
Datasets: huggingface.co/syamjithnk · Code: github.com/Syamjith-NK · Work & evidence
Questions about the methods or results: get in touch.