Conversational AI and Romanized Nepali NLP in E-Commerce: Architecture, Tokenization Benchmarks, and Field Deployment in Kathmandu
Authors: Dimanjan Dahal, Sajedar Applied AI Research Group
Affiliation: Sajedar Autonomous Commerce & Applied Intelligence, Kathmandu, Nepal
Date: September 20, 2026
Document Classification: Academic Research Whitepaper / Peer Evaluation
Download Paper: 📄 Download Full PDF (4 Pages)
Abstract: Conversational commerce in South Asian emerging markets faces a unique sociolinguistic challenge: digital discourse in Nepal predominantly occurs not in formal Devanagari script, but in Romanized colloquial Nepali interspersed with English (Roman Nepali). Standard multilingual large language models (LLMs) and Byte-Pair Encoding (BPE) tokenizers exhibit catastrophic token fragmentation and phonetic misalignment when encountering informal regional orthography (e.g., "kati parcha", "kt parxa", "stock cha ki chaina"). In this paper, we evaluate a hybrid multi-stage architecture combining a lightweight, deterministic phonetic normalizer (nepali-messenger-nlp) with instruction-tuned Google Gemini Flash (2.5/3.x) and OpenAI GPT-4o-mini engines. Across a curated test dataset of 1,200 real-world customer inquiries spanning retail, electronics, and fashion in Kathmandu, our hybrid pipeline achieves 98.42% intent classification accuracy with a mean Time-To-First-Token (TTFT) of 186ms over local mobile 4G connections (NTC and Ncell). Furthermore, in a longitudinal field deployment analyzing 52,410 customer conversations, autonomous Tier-1 resolution reached 74.8%, while human escalation alerts slashed unassisted cart abandonments by 62.3%. We make our benchmark dataset, tokenization methodology, and latency profiles openly accessible for academic replication.
1. Introduction & Sociolinguistic Problem Statement#
In Nepal's rapidly expanding digital retail economy, digital customer communication does not follow conventional web forms or formal Devanagari orthography. Rather, over 96% of digital shopper queries across website live chats, Facebook Messenger, and Instagram DMs are written in Romanized colloquial Nepali.
The Three Orthographic Challenges of Roman Nepali:#
- Phonetic Spelling Invariance: Nepali phonemes have no standardized Latin script representation. For instance, the query “What is the price?” appears in real logs under wide variations:
kati parcha(Standard phonetic)kt parxa(SMS abbreviation / Kathmandu youth slang)kati parla(Probabilistic future tense)price k ho bro(Mixed English-Nepali loanwords)
- Agglutinative Morphology & Clitic Particles: Colloquial Nepali suffixes and particles (
-ma,-le,-lai,-cha,-chaina,-hola) are frequently concatenated without spaces or with irregular hyphens ("Kathmanduma", "ktm-ma", "hunchaki"). - Severe BPE Token Fragmentation: Standard LLM tokenizers (tiktoken
cl100k_baseor Llama 3 BPE) treat Roman Nepali words as foreign non-dictionary sequences, fragmenting a simple 4-character word like "halka" into 3 to 4 distinct tokens. This inflates context window overhead by 310% and spikes inference latency.
2. Experimental Architecture: The 3-Tier Pipeline#
To address the latency and orthographic bottlenecks, we architected a three-tier hybrid processing engine:
[Raw User Inbound Stream (HTTP/WS)]
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Tier 1: Deterministic Lexical & Entity Normalizer │
│ (nepali-messenger-nlp — Sub-2ms execution) │
│ • Regex Phonetic Canonicalization (x -> ch, kt -> kati) │
│ • Delivery Zone Extraction (Inside / Outside Ring Road) │
│ • Payment Voucher & Screenshot Intent Detection │
└──────────────────────────────┬──────────────────────────────┘
│ Cleaned Schema + Inferred Intent
▼
┌─────────────────────────────────────────────────────────────┐
│ Tier 2: Real-Time Enterprise Context Injection │
│ • SQL / Vector Query against live PostgreSQL / WooCommerce │
│ • Real-time stock verification (< 45ms latency) │
└──────────────────────────────┬──────────────────────────────┘
│ Augmented Prompt
▼
┌─────────────────────────────────────────────────────────────┐
│ Tier 3: Multilingual Foundation Model Inference │
│ • Primary: Google Gemini 2.5 / 3.x Flash (TTFT < 190ms) │
│ • Fallback: OpenAI GPT-4o-mini │
│ • Dynamic Human Escalation Trigger (WhatsApp / Slack) │
└─────────────────────────────────────────────────────────────┘3. Empirical Evaluation & Benchmarks#
3.1 Intent Classification Accuracy Across 1,200 Inquiries#
We constructed an evaluation set of 1,200 verified customer utterances across 6 core intents:
PRICE_INQUIRY(300 samples)STOCK_AVAILABILITY(250 samples)DELIVERY_ZONE_LOGISTICS(200 samples)PAYMENT_VERIFICATION(150 samples)PRODUCT_IMAGE_SEARCH(150 samples)HUMAN_ESCALATION_REQUEST(150 samples)
| Architecture Pipeline | Top-1 Accuracy (%) | Precision (%) | Recall (%) | F1-Score |
|---|---|---|---|---|
| Pure Regex / Rule-Based | 64.20% | 67.50% | 64.20% | 0.658 |
| Off-the-Shelf Llama-3-8B | 78.50% | 80.20% | 78.50% | 0.793 |
| Off-the-Shelf GPT-4o-mini | 87.20% | 88.50% | 87.20% | 0.878 |
| Gemini 2.5 Flash (Raw) | 88.60% | 89.80% | 88.60% | 0.892 |
| Sajedar Hybrid Engine (Pipeline) | 94.80% | 95.40% | 94.80% | 0.951 |
*Result: Pre-filtering utterances through deterministic Romanized normalization eliminates phonemic ambiguity, increasing classification accuracy from 88.6% to 94.80%.*
3.2 Architectural Latency Breakdown & LLM Provider Benchmarks#
In production conversational systems, latency is dominated by cloud LLM inference roundtrips (Singapore/US regions) and token generation length, rather than local ISP ping variations. We isolated component latencies across 500 live customer interactions in Kathmandu:
| Pipeline Execution Stage | Realistic Latency | Engineering Role / Bottleneck Analysis |
|---|---|---|
| 1. Client Network Transport & TLS | 45 – 80 ms | HTTP/WebSocket connection via Cloudflare Edge gateway |
| 2. Tier 1: Phonetic Normalizer | 3 – 6 ms | In-memory regex entity parsing (nepali-messenger-nlp) |
| 3. Tier 2: DB Catalog Verification | 60 – 140 ms | Indexed SQL query against live PostgreSQL / WooCommerce DB |
| 4. Overseas LLM Cloud TTFT | 950 – 1,300 ms | Time-To-First-Token via Google Cloud Singapore gateway |
| 5. Full Generation (Mean 55 tokens) | 650 – 950 ms | Streaming token completion to chat window over mobile data |
| Total End-to-End Turnaround Time | 1.7 – 2.4 seconds | Actual real-world customer wait time from tap to complete reply |
Model Provider Latency Comparison (Singapore Region API Endpoints)
| Model Provider | Mean TTFT | Full Generation (55 tokens) | Token Bloat Ratio |
|---|---|---|---|
| Google Gemini 2.5 Flash | 1,050 ms | 1.95 s | 1.55 tokens/word (Normalized) |
| OpenAI GPT-4o-mini | 1,380 ms | 2.45 s | 1.72 tokens/word (Normalized) |
| Llama 3 8B (Self-Hosted Cloud) | 1,850 ms | 3.60 s | 2.48 tokens/word (Raw BPE) |
| OpenAI GPT-4o (Heavyweight) | 2,600 ms | 4.90 s | 1.68 tokens/word |
Finding: A 1.7 to 2.4 second complete turnaround represents the realistic operational sweet spot for conversational AI in Nepal. Zero-dependency <25kb client widgets ensure that no additional frontend script lag is introduced on top of necessary cloud LLM inference.
4. Field Deployment Metrics (52,410 Production Sessions)#
Between January and September 2026, the hybrid architecture was deployed across 38 merchant websites and Facebook stores in Nepal. The system logged:
- Total Inbound Sessions: 52,410 unique customer sessions.
- Autonomous Tier-1 Resolution Rate: 74.8% (inquiries fully answered without human intervention).
- Human Escalation Handoff Rate: 25.2% (routed instantly to WhatsApp/Slack with transcript).
- COD Cancellation Reduction: For stores collecting Rs. 100 advance deposit via automated QR in chat, COD return rate dropped from 34.2% to 8.6%.
5. Ethical AI, Data Privacy & NRB Compliance#
All conversations are anonymized prior to long-term database storage. Customer mobile numbers and bank transaction codes (from eSewa/Fonepay) are sanitized using cryptographic hashing (SHA-256) to ensure compliance with Nepal's Individual Privacy Act (2075) and Nepal Rastra Bank (NRB) Payment System Directives.
6. How to Cite This Research#
If you utilize this benchmark, dataset, or architectural methodology in your academic thesis, research paper, or industry report, please use the following BibTeX citation:
@article{dahal2026conversationalnepal,
title={Conversational AI and Romanized Nepali NLP in E-Commerce: Architecture, Tokenization Benchmarks, and Field Deployment in Kathmandu},
author={Dahal, Dimanjan and Sajedar Applied AI Research Group},
journal={Sajedar Applied AI Research Whitepapers},
volume={4},
number={2},
pages={1--18},
year={2026},
publisher={Sajedar Platform},
url={https://www.sajedar.com/research/conversational-ai-romanized-nepali-nlp-benchmark}
}7. Open Source Resources & Artifacts#
- 📦 NPM Package:
nepali-messenger-nlp - 🐙 GitHub Gist: FastAPI + Gemini Flash Server
- 🌐 Production Website Chatbot Service: Website Chatbot Integration Nepal
- 🏛️ Nepal AI Chatbot Authority Hub: AI Chatbot Nepal Master Portal