Data Engineer
$100,000–$115,000 year
HybridManhattan, New York, United States
Job Summary
Design, build, and operate cloud-based data pipelines supporting consumer websites, mobile apps, and information services using AWS and GCP technologies like Vertex AI, DynamoDB, and BigQuery. Develop and maintain real-time and batch ETL pipelines for large-scale content and event data while writing clean, testable Python code. Collaborate with product, editorial, and data science teams to deliver infrastructure for analytics, machine learning, and AI answer services. Monitor, optimize, and improve system reliability, security, and cost efficiency. Participate in code reviews, architecture discussions, and agile sprint ceremonies. Document functional requirements for technical and non-technical audiences. This role supports the New York Post Digital Network's multi-platform ecosystem spanning web, mobile, video, and commerce.
Required Qualifications
- MS or BS in computer science, or related field
- 2+ years of professional software engineering experience
- Proficiency in Python, SQL, and PySpark
- Working knowledge of common AWS and/or GCP services
- Experience building and deploying data pipelines (e.g., Airflow, Dataflow, Glue, or similar tools)
- Knowledge of data warehouse technologies (BigQuery, Snowflake, Redshift, etc.)
- Firm grasp of the tools of the trade including AI tooling, IDEs, Git, JIRA, and agile methodologies
- Familiarity with web standards and protocols
- Comfort with code reviews, writing unit tests, QA and UAT processes, and exposure to end-to-end testing frameworks
- Strong problem-solving skills, attention to detail, thoroughness, and an eagerness to learn
- Proactive approach to problem-solving
- Highly organized with the ability to manage multiple priorities in a fast-paced environment
- Must be authorized to work in the United States on a permanent basis
- Sponsorship, including employment visa sponsorship or transfer, is not available for this position now or in the future
- This position would be expected to be in the office a minimum of 3 days per week (subject to change)
Desired Qualifications
- Experience Familiarity with content-management systems, media, news, publishing, video technology, ad technology, or other high-traffic consumer-facing industries
- Exposure to personalization and recommendation systems
- Comfortable with the CLI, Linux administration, shell scripting, ssh, crontabs, etc.
- Hands-on experience with generative-AI models, LLM prompting, and/or machine learning
- Experience with monitoring, logging, dashboard, and alarming tools (e.g., CloudWatch, Datadog, Splunk, etc.)
- Familiarity with REST and GraphQL APIs, authentication/authorization, and API best practices
- Scaling techniques in cloud environments
- Troubleshooting and debugging experience with internet services
- Understanding of CI/CD workflows and containerization (Docker, Kubernetes a plus)
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.