ورقة عمل
A bilingual fine-tuned language model for measuring digital technology integration in Arabic and English annual report narrative
Ahmad Alomari, Anwar Bin Allah Pitchay
الملخص
Researchers who measure how far a firm has integrated digital technology often count keywords in the annual report. Keyword counts do not separate a firm that uses a technology from a firm that only talks about it, and they work poorly when the report is filed in two languages. This paper presents a tool that codes annual report sentences for six digital technologies (artificial intelligence and machine learning, big data analytics, cloud computing, enterprise resource planning, smart accounting and reporting automation, and XBRL) and assigns each technology sentence to a commitment tier. The tool combines a bilingual dictionary screen, a fine-tuned XLM-RoBERTa model that decides whether a screened sentence describes the reporting firm using a technology, and a deterministic rule that gives the commitment tier from two auditable conditions: the firm is the grammatical subject, and a verb from a closed list acts on a technology term. The tool was built on 1,880 hand-coded sentences from Arabic and English XBRL filings of Amman Stock Exchange firms, without translation. In firm-grouped five-fold cross-validation, the hybrid reaches an F1 of 0.857 in English and 0.833 in Arabic on firms unseen in training, against 0.734 and 0.712 for the dictionary screen; on a held-out partition it also exceeds a zero-shot large language model given the same codebook. Applied to 326 paired filings, the tool shows that technology narrative is rare, concentrated, and consistent across the two language versions. The model, code, codebook and coded sentences are released for reuse and extension.
التوثيق
Alomari, A., & Pitchay, A. B. A. (2026). A bilingual fine-tuned language model for measuring digital technology integration in Arabic and English annual report narrative.. https://doi.org/10.2139/ssrn.7517639