AWS Data Engineer

Data Engineer New York, NY (Hybrid) Face to Face Interview Contract Job Description

• Seeking a Senior Data Engineer with 8+ years of experience in data engineering, software engineering, or big data platform development, with a Bachelor s/Master s degree in Computer Science, Engineering, Information Systems, or a related field.
• Design, develop, and optimize enterprise-scale batch and real-time data pipelines supporting analytics, machine learning, reporting, and AI-driven solutions.
• Demonstrate strong hands-on expertise in Apache Spark, PySpark, Spark SQL, Python, SQL, Databricks, Delta Lake, and Snowflake.
• Build modern Lakehouse architectures using Databricks, Delta Lake, and Medallion Architecture (Bronze, Silver, Gold) for scalable data processing.
• Develop high-volume, low-latency streaming and distributed data solutions using Kafka, Spark Structured Streaming, AWS Kinesis, and related technologies.
• Architect and implement cloud-native data platforms on AWS, leveraging services such as S3, EMR, EC2, Athena, Redshift, RDS, Lambda, IAM, SNS, and SQS.
• Optimize large-scale data workloads through partitioning, clustering, caching, query tuning, distributed computing, data modeling, and performance optimization.
• Build data orchestration, automation, and CI/CD frameworks using Apache Airflow, Databricks Workflows, Git, Docker, and Oozie while supporting monitoring and operational reliability.
• Ensure enterprise data platforms meet requirements for data quality, governance, metadata, lineage, security, compliance, fault tolerance, and cost optimization.
• Apply Generative AI, LLMs, RAG, Vector Databases, AI Agents, Agentic AI, and Model Context Protocol (MCP) to build intelligent data platforms and AI-assisted engineering solutions.
• Experience with Azure AI, Microsoft Copilot Studio, Gemini Enterprise, Spark AI development platforms/frameworks/tools, and AI-assisted developer productivity solutions is highly preferred.
• Experience with Java, Shell Scripting, Hive, HBase, Sqoop, Tableau, Splunk, and AWS/Databricks certifications is a plus; domain exposure in supply chain, retail, healthcare, or manufacturing is preferred.