Data Engineer

Software Developer
Pune, Hyderabad
April 29, 2026
Full Time
Apply Now

Job Description

We are looking for an experienced Technical Lead / Senior Data Engineer to design, build, and manage scalable data pipelines and cloud-based data solutions. The role involves working extensively with AWS services, Python, and SQL to develop efficient data processing systems.

The ideal candidate will have strong expertise in ETL processes, cloud architecture, and data engineering best practices, along with the ability to lead technical initiatives and solve complex data challenges.

Key Responsibilities

Data Engineering & Development

  • Design, develop, and maintain scalable data pipelines using AWS services
  • Build and optimize ETL processes for large-scale data processing
  • Work with Python and SQL for data transformation and analysis
  • Develop data workflows using services like AWS Glue, Lambda, and Step Functions

Cloud & Architecture

  • Implement data solutions using AWS services such as S3, DynamoDB, Athena, and Snowflake
  • Manage cloud resources and ensure high availability and performance
  • Use AWS CDK for infrastructure as code and automation

Data Processing & Optimization

  • Work with PySpark for big data processing
  • Optimize data pipelines for performance, scalability, and cost efficiency
  • Ensure data quality, integrity, and governance

Monitoring & Troubleshooting

  • Monitor systems using CloudWatch and resolve performance issues
  • Troubleshoot data pipeline failures and system bottlenecks
  • Apply strong analytical and logical thinking to solve complex problems

Collaboration & Delivery

  • Work in Agile environments and collaborate with cross-functional teams
  • Use GitHub for version control and CI/CD processes
  • Support team members and contribute to technical decision-making

Requirements

  • Strong experience with AWS services (S3, Lambda, Glue, Step Functions, DynamoDB, etc.)
  • Proficiency in Python and SQL
  • Hands-on experience with ETL processes and data pipelines
  • Experience with Snowflake and cloud-based data warehousing
  • Knowledge of PySpark for large-scale data processing
  • Familiarity with AWS CDK and infrastructure as code
  • Experience working in Agile environments
  • Strong troubleshooting, analytical, and problem-solving skills

Technical Skills

  • AWS (S3, Lambda, Glue, Step Functions, DynamoDB, Athena, CloudWatch)
  • Python
  • SQL
  • ETL & Data Pipelines
  • PySpark
  • Snowflake
  • AWS CDK
  • GitHub

Key Skills

  • Data Engineering
  • Cloud Data Solutions
  • Big Data Processing
  • Data Pipeline Development
  • Performance Optimization
  • Problem Solving

Related Jobs