Overview

A PBM platform relied on a legacy RM/COBOL system as the system of record, while modern digital channels needed fast access to claims and transaction data for portal experiences, search, and analytics. The requirement was to synchronize COBOL transactions into AWS in near-realtime, ensuring strict ordering (the same sequence as inserted/committed in COBOL) and delivering the data to Application AWS RDS, Amazon Redshift, and Amazon OpenSearch.

External pharmacy claims ingestion was a key upstream dependency. We used Mirth Integration as the integration hub for receiving pharmacy claim transactions, performing validation/normalization/routing, and reliably handing off to COBOL workflows. After COBOL commits, Audit Programs produced auditable change records into an EFS-backed landing zone. A Relativity Driver (orchestration/CDC component) processed these records with SQS-based decoupling for retries and backpressure, then published ordered events into Confluent Kafka. Kafka served as the streaming backbone for fan-out to operational, analytical, and search destinations.

The solution was built with HIPAA-aligned controls: encryption in transit and at rest, least-privilege access, and audit-friendly monitoring. The delivered platform enabled modernization without destabilizing the COBOL environment, while providing fresh, ordered data to modern applications.

Scope and Modules Delivered

• External pharmacy claims integration via Mirth (validation, normalization, routing, reliability patterns)
• COBOL change capture using Audit Programs with an EFS staging/landing zone
• Relativity Driver and SQS orchestration for decoupled processing, retries, and backpressure handling
• Kafka streaming backbone (topic design, partitioning for ordering, retention and replay strategy)
• Multi-destination delivery: Application AWS RDS, OpenSearch indexes, and Redshift analytical model
• Configuration and Administration controls for routing/index rules and operational tuning
• CI/CD pipelines (GitHub Actions, AWS CodeBuild) and operational monitoring (lag, latency, failures, DLQ)

Architecture Highlights

• End-to-end PBM claims flow: Pharmacy -> Mirth -> COBOL -> AWS
• Ordered event processing to preserve claims/financial correctness
• One streaming source of truth (Kafka) feeding OLTP (RDS), OLAP (Redshift), and Search (OpenSearch)
• Replayable and auditable pipeline with deterministic reprocessing
• Resilience by design using EFS staging + SQS buffering + safe retries and DLQ patterns

Key Challenges and Solutions

• Integrating external pharmacy claims reliably across partners/formats:
  Used Mirth as the canonical ingestion point to validate, normalize, and route claim transactions with consistent observability.

• Preserving strict ordering from COBOL to cloud destinations:
  Implemented sequence-aware processing and Kafka partitioning strategy so events are applied deterministically in commit order.

• Near-realtime sync without impacting COBOL performance:
  Decoupled capture using Audit Programs and EFS staging to avoid heavy read pressure on COBOL and smooth spikes.

• Consistency across RDS, OpenSearch, and Redshift with retries:
  Implemented idempotent upserts and version/sequence checks per destination to prevent duplicates and out-of-order updates.

• Operational resilience under spikes, partial failures, and throttling:
  Used SQS buffering, DLQ/error topics, and controlled Kafka replay procedures to recover safely while meeting near-realtime goals.

Security and Compliance

Designed with HIPAA-aligned controls: encryption in transit and at rest, restricted access to integration components and data stores, least-privilege IAM, and audit-friendly logs and metrics for traceability and incident response.

Outcomes

Delivered an ordered, near-realtime PBM integration platform ingesting COBOL transactions into RDS, Redshift, and OpenSearch.

• Enabled Web Portal self-service and search using OpenSearch, and Tableau reporting on Redshift.
• Improved reliability via EFS staging, SQS decoupling, and Kafka replayability for safe recovery and backfills.

Skills

• Mirth Connect
• Confluent Kafka
• Kafka Connect
• AWS RDS
• Amazon Redshift
• Amazon OpenSearch
• AWS Lambda
• Amazon EKS
• Python
• Amazon SQS
• Amazon EFS
• Data Integration
• Streaming Architecture
• Ordered Event Processing
• CI/CD
• HIPAA-aligned Security
• Healthcare PBM