Platform Engineer

Role Overview:

We are seeking a skilled Platform Engineer to architect and maintain an offline, air-gapped Data Lake infrastructure for high-frequency manufacturing data ingestion. This role is critical for our semiconductor assembly operations in Ahmedabad and Jamnagar.

Key Responsibilities:

• Design scalable on-premises solutions using the Hadoop ecosystem (HDFS, Spark, Hive, YARN) suitable for air-gapped environments.
• Develop robust ETL pipelines in Python and Bash to ingest data from upstream systems like MES and SCADA.
• Optimize storage performance via Parquet/ORC compression strategies and efficient indexing.
• Ensure system resilience by managing compute resources, disk I/O, and disaster recovery protocols without external internet dependencies.
• Collaborate with factory IT teams to enable seamless data retrieval across the platform.

Tech Stack:

You will work extensively with Apache Hadoop, Spark, Hive, HBase, Kafka (offline logic), and Python scripting. Knowledge of semiconductor industry standards is highly valued.