About us: Provate Inc. is a Managed Service Provider (MSP) specializing in IT, Cloud, and Security solutions to empower businesses for growth in the digital landscape. The company’s mission is to transform IT environments into resilient, adaptable infrastructures, fortify security, and provide scalable support. Provate Inc. partners with clients to tailor solutions to their unique goals, ensuring their operations are protected and optimized.

Job Title: Offshore Hadoop Ops Engineer

Experience: 7+ Years

Job Type: Full-Time

Job Description: We are seeking a highly skilled Hadoop Ops Engineer to join our team. This role requires expertise in managing and maintaining Hadoop infrastructure, monitoring performance, and optimizing workflows. You will collaborate with data engineers, scientists, and infrastructure teams to ensure smooth operations and high availability of our Hadoop clusters.

Key Responsibilities:

• Implement and manage Hadoop infrastructure, ensuring high availability and performance optimization.
• Monitor production clusters daily using tools like Grafana, Cloudera Manager, Resource Manager, and NameNode.
• Analyze and manage various log files (OS, NameNode, Resource Manager, Node Manager, etc.).
• Troubleshoot and fine-tune performance for Hadoop jobs and data workflows.
• Ensure production jobs start on time and address any failures promptly.
• Automate manual tasks using Python, Shell scripting, and Ansible for improved performance.
• Collaborate with data engineers and scientists to optimize data pipelines and workflows.
• Work with Datacenter and Network Engineers to resolve hardware and connectivity issues.
• Install, configure, and maintain Hadoop clusters and related ecosystem components.
• Handle commissioning and decommissioning of Hadoop nodes as required.
• Support end-to-end production cluster maintenance activities.

Required Qualifications & Skills:

• 7+ years of experience as a Hadoop Administrator or similar role.
• Strong expertise in Hadoop ecosystem (HDFS, YARN, Oozie, Hive, Zeppelin, Livy, HBase, Cloudera Manager).
• Hands-on experience with deploying and managing Hadoop clusters, adding/removing nodes, and monitoring job performance.
• Solid troubleshooting skills with an understanding of systems capacity, bottlenecks, memory, CPU, OS, storage, and networks.
• Proficiency in Linux, as Hadoop primarily runs on Linux environments.
• Experience with automation tools like Python, Shell scripting, and Ansible.
• Strong knowledge of Spark (troubleshooting, debugging, performance tuning, and best practices).
• Experience working with GitHub and managing large data loads through automated processes.
• Excellent problem-solving, communication, and documentation skills.

We’d love to hear from you if you are passionate about managing large-scale Hadoop environments and driving efficiency through automation!