eSocial S-5002 Report Processing System

Complete system for processing and converting eSocial S-5002 XML files (income statements) into consolidated PDFs, developed with distributed architecture on AWS. The solution allows uploading ZIP files containing multiple XMLs, asynchronous large-scale processing, and automatic PDF generation by CPF.

The system consists of three main components: orchestration API developed with Bun and ElysiaJS that manages the entire processing pipeline and authentication, modern React frontend with TypeScript for job management and file uploads, and serverless AWS processing pipeline composed of Lambdas and SQS queues for distributed processing.

Technology stack includes backend with Bun, ElysiaJS, PostgreSQL, Drizzle ORM, JWT authentication with bcrypt, AWS SDK v3, Zod validation and RabbitMQ. Frontend uses React 18 with TypeScript, Vite, React Router v7, TanStack Query, Axios, Tailwind CSS, shadcn/ui and React Hook Form. AWS processing uses Python 3.x with Lambda, SQS, S3, CloudFormation, ReportLab and boto3.

Main features include authentication and authorization system with JWT and refresh tokens, job management with direct S3 upload via pre-signed URLs, real-time tracking with configurable polling, XML processing with automatic ZIP extraction, intelligent CPF grouping, consolidation of multiple statements and formatted PDF generation according to eSocial standards.

The processing flow uses a distributed pipeline: user uploads via frontend which requests pre-signed URL for S3, file is sent directly to S3, Lambda 1 extracts XMLs from ZIP, Lambda 2 groups XMLs by CPF, Lambda 3 processes each CPF in parallel consolidating statements and generating PDFs, and API updates status in real-time for frontend tracking.

Hybrid architecture combines monolithic API (Bun) with serverless processing (AWS Lambda), enabling horizontal scalability through asynchronous pipeline with SQS. Direct S3 upload reduces API load, intelligent consolidation groups multiple XMLs from the same CPF into a single PDF, and CNPJ filtering system allows processing only XMLs from specific companies. Complete type safety with TypeScript throughout frontend and backend.