Hadoop Administrator

About the Role:

We are seeking an experienced Hadoop Administrator (Linux / System Administrator) to manage complex multi-node cluster environments. This position is critical for supporting our Data Scientists and Engineers by ensuring high availability, security, and performance of big data ecosystems.

Key Responsibilities:

• Install, configure, maintain, and upgrade core Hadoop components including HDFS, YARN, Hive, HBase, Spark, Oozie, and ZooKeeper on Linux platforms.
• Lead migration efforts from Hortonworks Data Platform (HDP) to the open-source Apache Hadoop distribution while managing scalability across multi-node clusters.
• Administer comprehensive Linux server operations: user management, file permissions, disk space optimization, security patching, and rapid troubleshooting of system issues.
• Proactively monitor cluster health, resource utilization, and job performance using industry-standard tools like Ambari and Cloudera Manager alongside native Linux utilities.
• Spearhead the implementation of robust Kerberos authentication for enhanced security, along with designing backup strategies, recovery protocols, and disaster recovery plans.
• Develop automation scripts in Shell scripting and Python to streamline routine tasks and integrate cron jobs for scheduled operations.
• Provide dedicated support for RStudio Server and JupyterHub environments within Docker containers, diagnosing and resolving job failures efficiently.