DGLiger Consulting•11d ago
InstaHyre
Data Engineer
Gurgaon
Full Time
Senior Level
N/A
N/A
N/A
Qualifications & Requirements
Experience Level: Senior Level
Full Job Description
Python Data Engineer - Cloud Cost Optimization
DGLiger Consulting is seeking an experienced Python Data Engineer to join our Gurgaon-based team. This role is crucial for designing, optimizing, and maintaining robust data pipelines, with a specific focus on cloud billing data and cost optimization strategies. You will be instrumental in integrating advanced analytics into DevOps workflows, contributing to efficient cloud resource management and cost reduction.
Responsibilities:
- Design, develop, and maintain scalable ETL/ELT pipelines for processing cloud billing and usage data.
- Create sophisticated Python-based solutions for efficient data processing and insightful analytics.
- Collaborate closely with DevOps engineers to integrate cost optimization tools and methodologies into existing workflows.
- Develop and implement dashboards and reporting mechanisms to visualize cloud resource utilization and key cost metrics.
- Ensure adherence to industry best practices for distributed systems and cloud infrastructure management.
Requirements:
- A minimum of 4-8 years of professional Python development experience.
- Proven expertise in data engineering, with a strong track record in designing and managing ETL/ELT pipelines.
- Hands-on experience working with cloud billing and usage datasets from major cloud providers such as AWS, Azure, or GCP.
- Proficiency with data processing frameworks like pandas, NumPy, PySpark, or similar technologies.
- Familiarity with containerization and orchestration technologies such as Kubernetes, cloud compute primitives, and distributed systems.
- Demonstrated experience in building dashboards or integrating with monitoring and BI tools (e.g., Grafana, Datadog, or internal custom solutions).
- A solid understanding of cloud resource utilization metrics and the primary drivers of cloud costs.
- Ability to thrive in a fast-paced, execution-driven environment.
Preferred Qualifications:
- Previous experience in FinOps or specialized cloud cost optimization roles.
- Familiarity with backend infrastructure components like Elasticsearch/OpenSearch, Redis, Cassandra/Keyspaces, and Kafka.
- Prior experience within a B2B SaaS company, particularly those managing large-scale microservices architectures.
- Experience integrating automation and analytics solutions into DevOps workflows.
- Strong knowledge of time-series analysis techniques and anomaly detection algorithms.
Company
DGLiger Consulting
DGLiger is a dynamic technology consulting firm embodying the essence of digital transformation. Combining the stability of a lion ('LI' for lion) with the strength and agility of a tiger ('GER' for t...
Gurgaon
Posted on InstaHyre