We provide IT Staff Augmentation Services!

Lead Hadoop Developer Resume

0/5 (Submit Your Rating)

SUMMARY

  • Experience in performing development and data analytics using HADOOP (BIG - DATA) tools and technologies which included HDFS, MAPREDUCE, HIVE, PIG, HBASE, MONGODB, FLUME, SQOOP, Hue and OOZIE.
  • Strong database skills in IDMS, DB2, HIVE, Oracle, MySQL and No-SQL databases like HBASE, MONGODB, familiarity with CASSANDRA.
  • Experienced in installing, configuring, managing and testing Big-data “HADOOP” ecosystem components.
  • Experienced in Data warehouse concepts and ETL tools (Talend).
  • Experience in the area of legacy systems based on Mainframe platform
  • Experienced in UNIX, and Linux desktops such as Ubuntu, CentOS
  • Experienced with work flow schedulers, data architecture including data ingestion pipeline design and data modeling.
  • Experience in leading and managing development teams.
  • Possess functional knowledge in the areas of Insurance systems, Broadcast services, Financial Systems and Material Management System.
  • Good experience in all phases of systems life cycle Development, Testing (Unit test, System test, Integration Testing and Regression Testing) and Pre Production support.
  • Proficient in analyzing and translating business requirements to technical requirements and architecture.
  • Experience in projecting financial forecasts, building staffing plan, preparing Metrics and implementation plans.
  • Performed Knowledge management in the form of AIDs and Project knowledge and change documents.
  • Experienced in handling internal and external functional, process and data audits.

TECHNICAL SKILLS

Big Data Ecosystem: Apache Hadoop, CDH, HDFS, Hive, Impala, Sqoop, Pig, HBase, Phoenix, MongoDB, AVRO, Flume, kafka, spark, Zookeeper, Cassandra, Oozie, Hue, Hortonworks, Ambari, Map-reduce, distcp

Database skills: IDMS, DB2,DMS1100, Oracle, VSAM, MySQL

Programming/Scripting Languages: COBOL, Java, PL/SQL, Unix shell, XML, SQL, HIVEQL, PIG scripts, Morphline, Scala, python, REST, SOAP

Operating environments: MVS/ESA, ZOS, WINDOWS, UNIX, LINUX, AWS, Confidential & Confidential Data lake

Legacy: Mainframe, JCL, VSAM, ADSO, CICS, OLQ, DBOL, DMLO, CULPRIT, CHANGEMAN, PANVALENT, File manager, IBM Debug, ESP, ZEKE, CA-7

ETL/Reporting: Talend, TABLEU

Processes: IBM’s OPAL (On Demand process asset library), QMS, Lean

Other tools: RPM, MPP, MANAGENOW R2, TEST DIRECTOR 8.0, CLARITY, QUALITY CENTER, SERVICE CENTER, FTP, SFTP, MOBIUS, DB Visualizer, Toad, SSH, Eclipse, IntelliJ Idea, SVN, nexus, Maven, putty, pdsh, Tomcat, junit,mrunit.

PROFESSIONAL EXPERIENCE

Lead Hadoop Developer

Confidential

Responsibilities:

  • Completed Addressable Ads, Audience measurement, Hard disk drive health monitoring projects.
  • Completed Hadoop applications migration from Cloudera to Horton works distribution.
  • Built scripts/ flows for exporting large data sets from various data sources into HDFS using Sqoop, flume, kafka, kite-morphline, NFS, SFTP, JSCH, shell scripts.
  • Built applications using Oozie, Kite-Morphline, Java, Map-Reduce, Hive, Pig, Hbase, Phoenix, Flume, kafka, zookeeper.
  • Involved in data modeling in Hadoop.
  • Experienced in analyzing data with Hive using HiveQL, Hue, CLI
  • Written Apache PIG scripts to process and stich the HDFS data.
  • Experience in using AVRO, JSON, XML file formats.
  • Created Java UDFs in PIG and HIVE.
  • Developed Oozie coordinator workflows for scheduling and orchestrating the ETL process
  • Involved in Loading and querying Hbase database using Fume, Java API, Phoenix and shell.
  • Performed Hbase and Hive integration using Hbase storage handlers (Serde).
  • Good working knowledge of Amazon Web Service components like EC2, EMR, S3, EBS, ELB
  • Written distributed copy Jobs to transfer data from AWS cluster to S3 Buckets.
  • Working towards transforming long running hourly Jobs to Spark jobs for faster processing.
  • Worked with the admin team in upgrading Hadoop Version.
  • Experience with MRV1 and MRV2 (YARN) frameworks.
  • Automated tasks using Unix shell scripts and Python scripts.
  • Prepare estimations and Technical Design Specifications for projects.
  • Gathered the business requirements from the Business Partners and Subject Matter Experts.
  • Prepared Builds and release notes.
  • Provide production support and fixes.
  • Lead teams on onshore-offshore model and help deliver quality products as a Team.

Environment: HADOOP, HDFS, MAPREDUCE, HIVE, Hue, PIG, Flume, kafka, SQOOP, HBASE, Phoenix, OOZIE, AVRO, ZooKeeper, Core Java, Oracle, SQL, UBUNTU/UNIX, eclipse, IntelliJ, Maven, JDBC drivers, DB2, MySQL, Linux, AWS, XML, SVN, PDSH, Putty, SSH, Kerberos, ETL tools Talend, Kite-morphline, scala, python, nexus, Spark, Tomcat, REST, SOAP,junit,mrunit, jsch.

Analyst and Consultant/Tech Lead

Confidential

Responsibilities:

  • Involved in installing Hadoop Cluster and Hadoop Ecosystem components.
  • Importing and exporting data into HDFS and Hive using Flume, Sqoop, SFTP, Shell scripts.
  • Experienced in analyzing reporting data with Hive and Pig.
  • Writing Pig/ Hive scripts to process the data.
  • Built applications based on Oozie, Java, Map-Reduce, Hive, Pig, Hbase, Flume, zookeeper.
  • Developed Oozie coordinator workflows for scheduling Jobs.
  • Worked with UDFs in PIG and HIVE.

Environment: HADOOP, HDFS, MAPREDUCE, HIVE, Hue, PIG, Flume, SQOOP, HBASE, OOZIE, AVRO, ZooKeeper, Core Java, Oracle, SQL, UBUNTU/UNIX, eclipse, IntelliJ, Maven, JDBC drivers, DB2, MySQL, Linux, AWS, XML, SVN, PDSH, Putty, SSH, Kerberos, ETL tools Talend, scala, python, nexus.

We'd love your feedback!