Job Title: Senior PySpark Data Engineer (5+ Years Experience)
Location: Remote / Hybrid
Experience Level: 5+ Years
Role Overview
We are seeking an experienced Data Engineer to design, build, and optimize our large-scale data pipelines and analytics architecture. You will work extensively with cloud data platforms to transform complex datasets into actionable business insights.
Mandatory Skills
- PySpark & Python: 5+ years of hands-on experience building distributed data processing pipelines and advanced data transformations.
- Databricks: Strong expertise in managing Databricks workspaces, Delta Lake, and optimizing cluster performance.
- Azure Data Factory (ADF): Proven track record in orchestrating ETL/ELT workflows and pipeline automation.
- SQL: Advanced query optimization, database design, and data modeling skills.
Preferred Skills
- Hands-on experience with Microsoft Fabric (OneLake, Synapse Data Engineering, Real-Time Analytics).
Key Responsibilities
- Architect, develop, and maintain robust ETL/ELT pipelines using PySpark and ADF.
- Optimize Databricks jobs and SQL queries for performance and cost efficiency.
- Collaborate with data science and analytics teams to support data delivery requirements.
- Enforce data governance, quality standards, and security best practices across all data assets.
Pay: Up to ₹2,000,000.00 per year
Benefits:
Experience:
- Pyspark: 5 years (Required)
- Python: 5 years (Required)
- Databricks: 5 years (Required)
- Azure: 5 years (Required)
- Fabrics: 1 year (Preferred)
Work Location: Remote
