IT Hadoop Admin for an AI and Machine Learning Company

Purpose

Providing Administrative Support for the Customers on Hadoop platforms as required by the customers' SLAs.


Responsibilities

  1. Manages and controls the Hadoop System environment for the customers
  2. Must be prepared to work in shifts and also be on-call to support customer site(s)


Requirements

  1. Experience in Managing and Supporting large scale Production Hadoop environments (configuration management, monitoring, and performance tuning) in any of the Hadoop distributions (Apache, Hortonworks, Cloudera, MapR, IBM BigInsights, Pivotal HD)
  2. Experience in Scripting Language (Linux, SQL, Python). Should be proficient in shell scripting
  3. Experience on Administrative activities likes-
  4. Administration, maintenance, control, and optimization of Hadoop capacity, security, configuration, process scheduling, and errors
  5. Management of data, users, and job execution on the Hadoop System
  6. Experience in Backup, Archival and Recovery (BAR) and High availability (HA)
  7. Plan for and support hardware and software installation and upgrades
  8. Experience in Hadoop Monitoring tools (Cloudera Manager, and Ambari, Nagios, Ganglia etc)
  9. Experience may include (but is not limited to) build and support including design, configuration, installation (upgrade), monitoring and performance tuning of any of the Hadoop distributions
  10. Hadoop software installation and upgrades
  11. Experience of workload / performance management
  12. Automation - experience in CI / CD (Continuous Integration / Deployment) – Jenkins, Ansible, Terraform, Puppet, Chef
  13. Implementing standards and best practices to manage and support data platforms as per distribution
  14. Proficiency in Hive internals (including HCatalog), SQOOP, Pig, Oozie and Flume/Kafka
  15. Experience in ElasticSearch, MySQL & PostgreSQL databases


Bonus Points

  1. Experience with DR (Disaster Recovery) strategies and principles
  2. Development or administration on NoSQL technologies like Hbase, MongoDB, Cassandra, Accumulo, etc.
  3. Development or administration on Web or cloud platforms like Amazon S3, EC2, Redshift, Rackspace, Kubernetes, OpenShift etc.
  4. Web/Application Server & SOA administration (Tomcat, JBoss, etc.)
  5. Development, Implementation or deployment experience on the Hadoop ecosystem (HDFS, MapReduce, Hive, Hbase)
  6. Experience on any one of the following will be an added advantage
  7. Hadoop integration with large scale distributed data platforms like Teradata, Teradata Aster, Vertica, Greenplum, Netezza, DB2, Oracle, etc.
  8. Proficiency with at least one of the following Java, Python, Perl, Ruby, C or Web-related development
  9. Knowledge of Business Intelligence and/or Data Integration (ETL) operations delivery techniques, processes, methodologies
  10. Exposure to tools data acquisition, transformation & integration tools like Talend, Informatica, etc. & BI tools like Tableau, Pentaho, etc.
  11. Linux Administrator certified