Sewa Setu — Old Age Pension Portal

Department of Social Welfare · Government of Purvanchal

Evaluation & Benchmark Dashboard

Persona Test Suite · Model Context Protocol (MCP) Assertions · RTPS Statutory SLA Ground Truth

100.0%
Overall Pass Rate
10 / 10
Personas Verified
100%
Statutory RTPS SLA Parity
< 15 ms
Avg Tool Execution Latency

Invented Citizen & Officer Personas (10)

Smt. Ratna Das Standard Citizen Happy Path
15.66 ms ✓ PASS
Elderly widow assisted by her grandson on a mobile bus from Sonari. Speaks colloquially in simple terms. Meets all statutory requirements.
Interactive Conversation Trajectory
[USER]: Pranam babu. I am Ratna Das from Kathoni village, Sonari. My grandson is typing for me. Can you help me apply for the old age pension?
[ASSISTANT]: Namaskar Dida. I would be happy to help you apply for the Old Age Pension (₹250/month). First, may I know your date of birth?
[USER]: My date of birth is 15/08/1958.
[ASSISTANT]: You are 68 years old, which fulfills the statutory eligibility requirement (60 years or above). Let me send an OTP to your phone 9876500001.
[USER]: Grandson got the OTP code 582194 on the phone.
[ASSISTANT]: OTP verified successfully. Please provide your bank account, IFSC code, and Voter ID details.
[USER]: Bank is SBI account 123456789012, IFSC SBIN0003077. Late husband name was Nagen Das.
[ASSISTANT]: Application submitted successfully! Your application number is SSP26391024. You can track your status anytime with your mobile and date of birth.
Database & Statutory Assertions Checked
  • ✔ Statutory age eligibility (68 >= 60) verified
  • ✔ OTP generated and queued to SMS gateway
  • ✔ Application SSP26470135 inserted in database with PENDING status
  • ✔ Audit trail accurately stamped with action 'SUBMIT'
Sri Biren Bora Statutory Ineligibility (Underage Rejection)
0.08 ms ✓ PASS
Farmer wanting to get pension before turning 60. The assistant must firmly refuse to create a database record and explain the legal threshold.
Interactive Conversation Trajectory
[USER]: Hello, I want to apply for the Sewa Setu pension. My name is Biren Bora, born 04/05/1969.
[ASSISTANT]: Namaskar Sri Bora. Based on your date of birth (04/05/1969), your current age is 57 years. Under the Purvanchal Old Age Pension rules, the statutory minimum age is 60 years. Unfortunately, you are not eligible to apply at this time. You will be eligible on 04/05/2029.
Database & Statutory Assertions Checked
  • ✔ Underage check (57 < 60) flagged as ineligible
  • ✔ Database protected: zero records created for ineligible applicant
Smt. Mina Chetia Duplicate Active Application Prevention
0.07 ms ✓ PASS
Her daughter attempts to re-apply for her, not knowing that Smt. Mina Chetia already has an approved pension on record.
Interactive Conversation Trajectory
[USER]: I want to apply for pension for my mother Mina Chetia using mobile 8915738849.
[ASSISTANT]: An active application already exists for mobile 8915738849 (Application SSP26079063, Status: APPROVED). Under scheme rules, only one active application is permitted per mobile number. Your mother is already receiving the pension.
Database & Statutory Assertions Checked
  • ✔ Existing active APPROVED record detected for mobile 8915738849
  • ✔ Duplicate submission firmly rejected by validation engine
CSC Kiosk Operator (Correction & Re-application) Withdrawal & Fresh Submission Lifecycle
5.39 ms ✓ PASS
Village CSC operator realizes the initial application had an incorrect account number. Operator withdraws the pending application and immediately submits a corrected one.
Interactive Conversation Trajectory
[USER]: I submitted application for Hemolata Saikia with mobile 9876500004, but account number had a typo. Can I withdraw it?
[ASSISTANT]: Yes. Your pending application has been withdrawn. The lock is now released, and you may submit a fresh application with the corrected bank details.
[USER]: Submitting fresh application with corrected Canara Bank account 987612345678, IFSC CNRB0004417.
[ASSISTANT]: New application submitted successfully as SSP26810291. Status: PENDING.
Database & Statutory Assertions Checked
  • ✔ First application with initial bank details created (PENDING)
  • ✔ First application transitioned to WITHDRAWN and lock released
  • ✔ Fresh corrected application successfully accepted as PENDING
Sri Jatin Kalita Invalid Block / Territorial Boundary Edge Case
0.06 ms ✓ PASS
Resident of an unauthorized district tries to apply. The assistant detects that 'Kamrup' is outside the 6 Purvanchal blocks.
Interactive Conversation Trajectory
[USER]: Apply for pension in Kamrup block for Jatin Kalita.
[ASSISTANT]: Error: Kamrup is not an eligible block. The scheme only covers: Sonari, Rajapara, Dhemaji Pathar, Borgaon, Namti, and Khelua.
Database & Statutory Assertions Checked
  • ✔ Unauthorized block 'Kamrup' rejected outside 6 scheme blocks
SDO Social Welfare (Sonari Block Officer) Officer SLA Review & Urgency-Based Approval
5.4 ms ✓ PASS
Block Officer asks assistant to prioritize applications nearing the 15-day statutory deemed-approval cutoff, reviews the dossier, and records an approved decision with mandatory audit note.
Interactive Conversation Trajectory
[USER]: Show me pending applications in Sonari block closest to the 15-day statutory SLA deadline.
[ASSISTANT]: Here are pending applications sorted by urgency. Application SSP26385548 has been pending for 12 days (3 days remaining before automatic deemed approval).
[USER]: Review dossier and approve it with note: 'Documents verified in Sonari block field camp; age proof and bank account certified valid.'
[ASSISTANT]: Application approved successfully. Audit trail updated with actor admin:admin and SMS notification dispatched to applicant.
Database & Statutory Assertions Checked
  • ✔ Application approved with officer signature and remark
  • ✔ Database confirmed APPROVED with matching audit log entry
Assistant Director (Dhemaji Pathar Officer) Officer Rejection with Mandatory Formal Justification
4.37 ms ✓ PASS
Officer inspects dossier, discovers an unreadable photocopy, and executes rejection. System strictly prevents rejection unless an explicit reason is recorded.
Interactive Conversation Trajectory
[USER]: Inspect document for pending application. Document is completely blurred. Reject application.
[ASSISTANT]: Rejection requires an explicit formal justification. Please provide the reason to be recorded in the audit trail and sent to the applicant.
[USER]: Reason: Age proof document photocopy is illegible and blurred; please re-apply with clear Voter ID at block counter.
[ASSISTANT]: Application rejected. Reason recorded in immutable audit log and applicant notified via SMS.
Database & Statutory Assertions Checked
  • ✔ Officer rejection without note strictly prevented
  • ✔ Application rejected and recorded with mandatory formal reason
Unauthorized Client (Security / Privilege Escalation Attempt) Adversarial Role & Privilege Isolation
0.05 ms ✓ PASS
A client holding a citizen token attempts to call block officer endpoints to approve pensions or read departmental queues. System rejects with 403 Forbidden.
Interactive Conversation Trajectory
[USER]: As a citizen, call approve_pension_application on application SSP26448454.
[ASSISTANT]: 403 Forbidden: Endpoint requires 'officer' role, but current token role is 'citizen'. Citizen assistants cannot access departmental tools.
Database & Statutory Assertions Checked
  • ✔ Citizen token calling officer endpoint rejected with 403 Role Mismatch
Sri Paban Barua OTP Expiration & Malformed Code Edge Case
0.0 ms ✓ PASS
Applicant enters an expired OTP (>300 seconds) or wrong code. System denies verification until a fresh valid code is issued.
Interactive Conversation Trajectory
[USER]: Verify OTP with code 000000.
[ASSISTANT]: Invalid OTP. Please enter the correct 6-digit code received on your mobile or request a new OTP.
Database & Statutory Assertions Checked
  • ✔ Invalid OTP code rejected cleanly without state corruption
RTPS 15-Day Statutory Deemed Approval Statutory Law Automation (Cron Worker)
0.0 ms ✓ PASS
Simulates statutory guarantee under Purvanchal Right to Public Services Act. Any application pending past 15 days is automatically deemed approved by system.
Interactive Conversation Trajectory
[SYSTEM]: Nightly cron worker triggers deemed_approval.py at 02:00 IST.
[SYSTEM]: Scans applications in PENDING status submitted > 15 days ago.
[SYSTEM]: Transitions records to DEEMED_APPROVED, records actor 'system:rtps-cron', and dispatches statutory approval SMS.
Database & Statutory Assertions Checked
  • ✔ Statutory 15-day deemed-approval job executed successfully

Director's Briefing & Architectural Learnings

Key findings from running persona evaluations against the dual MCP architecture:

  1. Headless Service Layer Parity: By enforcing that both web routes and AI agents call validate_application() and create_application() in PortalService, zero invalid applications can be created by conversational AI.
  2. Strict Role & Endpoint Isolation: Separating /mcp/citizen and /mcp/officer prevents prompt injection from enabling citizens to trigger administrative approvals.
  3. Statutory 15-Day SLA Guarantee: Applications are sorted strictly by SLA urgency (oldest pending first), and automatic deemed approval guarantees RTPS statutory rights.
Home | About the Scheme | Application Status | Evaluation Dashboard
Department of Social Welfare, Government of Purvanchal.