Data Engineer

Data Engineer to design, develop and deploy Prevalent AI’s Security Data Science (SDS) platform modules across data collection, ingestion, processing and analytics. Build and optimize data ingestion, parsing, scheduling, and data management components using open-source big data technologies (e.g., Spark, Scala, PySpark) on cloud (AWS/Azure) and on-premise, with emphasis on end-to-end data pipelines, testing (unit/system/integration), code quality, and security. Collaborate on researching and integrating big data tools, orchestrate data workflows with Airflow, and implement scalable data storage and processing architectures. Required to work with Spark Framework, SQL, stream-processing, messaging, and API services; exposure to LLM integrations is a plus. Strong communication, project management, and Agile methodology experience expected. Educational background in Computer Science (Master’s or Bachelor’s).