Pyspark Developer

Job Description: Python / PySpark Developer (4 - 8 Years Experience)
Job Title: Python / PySpark Developer
Experience: 4 - 8 Years
Location: Chennai / Bangalore
Employment Type: Full-Time
Job Summary
We are seeking a highly skilled Python / PySpark Developer with 4-8 years of experience in designing, developing, and supporting scalable data processing solutions. The ideal candidate should possess strong expertise in Python, PySpark, and data engineering concepts, with the ability to work in an Agile environment and deliver high-quality solutions for large-scale data migration, transformation, and analytics initiatives.
Key Responsibilities

• Design, develop, and maintain data processing applications using Python and PySpark.
• Build scalable ETL/ELT pipelines for processing large volumes of structured and unstructured data.
• Develop and optimize Spark jobs for performance and reliability.
• Analyze business requirements and translate them into technical solutions.
• Work closely with Business Analysts, Solution Architects, and cross-functional teams.
• Perform code reviews and ensure adherence to coding standards and best practices.
• Troubleshoot production issues and provide timely resolutions.
• Participate in Agile ceremonies, sprint planning, and estimation activities.
• Create technical documentation, unit test cases, and deployment guides.
• Support data migration and modernization initiatives across enterprise platforms.

Required Skills

• 4-8 years of hands-on experience in software development and data engineering.
• Strong programming expertise in Python.
• Strong experience in PySpark and Apache Spark framework.
• Experience in developing and supporting ETL/data migration solutions.
• Strong understanding of SQL and relational databases.
• Experience working with large datasets and distributed computing environments.
• Knowledge of data modeling, data quality, and performance tuning techniques.
• Familiarity with Linux/Unix environments and shell scripting.
• Experience with version control tools such as Git.
• Strong analytical, debugging, and problem-solving skills.

Preferred Skills

• Experience with Hadoop ecosystem technologies.
• Exposure to cloud platforms such as Azure, AWS, or GCP.
• Experience with Azure Data Factory, Databricks, or similar data platforms.
• Knowledge of CI/CD pipelines and DevOps practices.
• BFSI domain experience will be an added advantage.