We provide IT Staff Augmentation Services!

Hadoop Admin Resume

3.00/5 (Submit Your Rating)

NC

PROFESSIONAL SUMMARY:

  • Over 10 years of professional work experience in teh IT Industry; around 2 years of experience wif Hadoop, HDFS, MapReduce and Hadoop Ecosystem (Pig, Hive, HBase).
  • Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, JobTracker, TaskTracker, NameNode, DataNode programming paradigm.
  • Experience wif leveraging Hadoop ecosystem components including Pig and Hive for data analysis, Sqoop for data migration, Oozie for scheduling and HBase as a NoSQL data store.
  • Good Exposure on Apache Hadoop Map Reduce programming, PIG Scripting and Distribute Application and HDFS.
  • Experience in NoSQL database MongoDB and Cassandra. Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, and Flume.
  • Experience in installation, configuration, supporting and managing - CloudEra's Hadoop platformalong wif CDH3&4 clusters.
  • Experience in managing and reviewing Hadoop log files.
  • Experience in analyzing data using HiveQL, Pig Latin, HBase and custom MapReduce programs in Java.
  • Extending Hive and Pig core functionality by writing custom UDFs.
  • Experience in data management and implementation of Big Data applications using Hadoop frameworks.
  • Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
  • Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
  • Experienced in deployment of Hadoop Cluster using Puppet tool.
  • Experience in Hadoop Shell commands, writing MapReduce Programs, verifying managing and reviewing Hadoop Log files.
  • Proficient in configuring Zookeeper, Cassandra & Flume to teh existing Hadoop cluster.
  • Strong analytical skills wif ability to quickly understand client’s business needs and create specifications.
  • Strong knowledge of Mahout Machine learning, Mongo DB Experience working wif geographically dispersed teams.
  • Data Warehousing ETL experience in Informatica Power Center 9.0/8.6/8.5/8.1.1/8.0/7.1.2/7.1.1/7.0/6.2 Power Mart 6.2/5.1.1/5.0/4.7.2.
  • Experience in teh Implementation of full lifecycle in Data warehouses and Business Data marts wif Star Schemas and Snow flake Schemas.
  • Relational and Dimensional Data Modeling experience.
  • Extensive experience in Data warehousing tools including Informatica, Business Objects using different databases.
  • Extensively Worked on Data warehousing wif Data Marts, Data Modeling, DataCleansing, ETL tool Informatica and data analyzing reporting Tool.
  • Experience in working wif data extraction, transformation and loading using Informatica Power Center.
  • Experience in working wif all Informatica modules like Server manager, Repositorymanager, Designer.
  • Expertise in administering InformaticaRepository, including creating and managing users.
  • Extensive experience in supporting production tasks, data enhancements and code fixes.
  • Good working Knowledge in OOA and OOD, using UML and designing use cases.
  • Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
  • Well versed in Object Oriented languages like C++ and Java.

TECHNICAL SKILLS:

Hadoop / Big Data: HDFS, MapReduce, Hive, Pig, HBase, Sqoop, Flume, Oozie, Zookeeper, MongoDB, Cassandra

Programming and Scripting languages: Java, SQL, Unix Shell Scripting.

ETL Informatica Tools: Informatica Power Center 9.0/8.6/8.5/8.1.1/7.1.3/6.2/5.0, Informatica Power Mart5.1, Power Analyzer 3.5x

Database: MS-SQL Server, Oracle 11g/10g/9i, MySQL, DB2, MS-Access

Frameworks: MVC, Struts, Spring, Hibernate

Web Technologies: TCP/IP, UDP, HTTP

Application Server: Apache Tomcat, JBoss, Web Sphere, Web Logic

Operating Systems: Windows, Ubuntu, Red Hat Linux, UNIX

PROFESSIONAL EXPERIENCE:

Confidential, NC

Hadoop Admin

Responsibilities:

  • Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like Hive, Pig, HBase, Zookeeper and Sqoop.
  • Wrote teh shell scripts to monitor teh health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
  • Installed and configured Hadoop, MapReduce, HDFS (Hadoop Distributed File System), developed multiple MapReduce jobs in java for data cleaning.
  • Developed data pipeline using Flume, Sqoop, Pig and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.
  • Involved in collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
  • Collected teh logs data from web servers and integrated in to HDFS using Flume.
  • Worked on installing cluster, commissioning & decommissioning of DataNodes, NameNode recovery, capacity planning, and slots configuration.
  • Implemented NameNode backup using NFS. This was done for High availability.
  • Developed PIG Latin scripts to extract teh data from teh web server output files to load into HDFS.
  • Involved in teh installation of CDH3 and up-gradation from CDH3 to CDH4.
  • Responsible for developing data pipeline using HDInsight, flume, Sqoop and pig to extract teh data from weblogs and store in HDFS.
  • Installed Oozie workflow engine to run multiple Hive and Pig Jobs.
  • Use of Sqoop to import and export data from HDFS to RDBMS and vice-versa.
  • Used Hive and created Hive tables and involved in data loading and writing Hive UDFs.
  • Exported teh analyzed data to teh relational databases using Sqoop for visualization and to generate reports.
  • Involved in migration of ETL processes from Oracle to Hive to test teh easy data manipulation.
  • Used Hive to analyze teh partitioned and bucketed data and compute various metrics for reporting.
  • Worked on importing and exporting data from Oracle and DB2 into HDFS and HIVE using Sqoop.
  • Worked on NoSQL databases including HBase, MongoDB, and Cassandra.
  • Created Hive External tables and loaded teh data in to tables and query data using HQL.
  • Wrote shell scripts for rolling day-to-day processes and it is automated.
  • Automated workflows using shell scripts pull data from various databases into Hadoop
  • Deployed Hadoop Cluster in Fully Distributed and Pseudo-distributed modes.
  • Supported in setting up QA environment and updating configurations for implementing scripts wif Pig and Sqoop.

Environment: Hadoop, MapReduce, Hive, HDFS, PIG, Sqoop, Oozie, Cloudera, Flume, HBase, ZooKeeper, CDH3, MongoDB, Cassandra, Oracle, NoSQL and Unix/Linux.

Confidential, CA

Hadoop Admin

Responsibilities:

  • Involved in teh Complete Software development life cycle (SDLC) to develop teh application.
  • Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, HBase database and Sqoop.
  • Involved in loading data from LINUX file system to HDFS.
  • Experience in managing and reviewing Hadoop log files.
  • Exported teh analyzed data to teh relational databases using Sqoop for visualization and to generate reports for teh BI team.
  • Importing and exporting data into HDFS and Hive using Sqoop.
  • Implemented test scripts to support test driven development and continuous integration.
  • Supported in setting up QA environment and updating configurations for implementing scripts wif Pig and Sqoop.
  • Analyzed large data sets by running Hive queries and Pig scripts.
  • Worked on tuning teh performance Pig queries.
  • Mentored analyst and test team for writing Hive Queries.
  • Installed Oozie workflow engine to run multiple MapReduce jobs.
  • Worked wif application teams to install operating system, Hadoop updates, patches, version upgrades as required.

Environment: Hadoop, HDFS, Map Reduce, Hive, Pig, Sqoop, Linux, Java, Oozie, HBase

Confidential, Indianapolis, IN

Informatica Developer

Responsibilities:

  • Worked wif data modeler’s team to create Data Flow Diagrams.
  • Developed Informatica mappings and mapplets, between sources and operational stagingtargets, and fine-tuned some of them for better performance.
  • Created views and altered some of teh dimensional tables to satisfy teh reporting needs.
  • Employed Source Qualifier, Lookup, Expression, Aggregator, Rank, Sequence generator and Joiner transformations for populating teh datamarts
  • Used pre-session and post-session stored procedures in PL/SQL to improve teh performance of teh session.
  • Implemented Type II Slowly Changing Dimensions using Version number.
  • Performed migration from 6.2 TO 7.1, and performed backup and performance tuning.
  • Extensively Worked wif Variables and Parametersin teh mappings to pass teh values between sessions.
  • Instrumental in writing PL/SQL stored procedures and packages for transactional details (insert, update and delete operations)
  • Created Migration Documentation and Process Flow for mappings and sessions.
  • Created Event Wait, Event Raise, Timer and Control Events in workflow manager according to teh business requirement.
  • Responsible for unit testing teh mappings using SQL scripts.

Environment: Informatica PowerCenter 6.2, 7.1, ORACLE 9i, HP-Unix, Windows 2000, Erwin 3.5, SQL*PLUS, SQL, PL/SQL.

Confidential, Pittsburgh, PA

ETL Developer

Responsibilities:

  • Analyzed and studied various Schemas for Implementation.
  • Created source definitions from teh operational sources and target definitions.
  • Designed Sources to Targets mappings from primarily Excel/Flat files/XML to Oracle using Informatica.
  • Worked Extensively on Informatica modules - Repository Manager, Designer, and Server Manager.
  • Developed mappings, reusable Transformations for ETLusing Informatica Power Center .
  • Worked extensively on different types of transformations like source qualifier, expression, filter, aggregator, rank, update strategy, lookup, sequence generator and joiner.
  • Command task and decision task based upon business requirements.
  • Created workflows and reusable worklets for designed mappings.
  • Analyzed teh session, event and error logs for troubleshooting mappings, sessions and workflows.
  • Worked wif mapping variables, Mapping parameters and session parameters.
  • Extensively used debugger to troubleshoot logical errors.
  • Extensively used PL/SQL for creating views, stored procedures on teh tables.
  • Involved in partitioning of teh sessions and performance tuning of database, Mappings, as well as Sessions.

Environment: Informatica Power Center 8.x, Oracle 9i, SQL*Plus, PL/SQL, TOAD 7.6, UNIX, Windows NT.

Confidential

Java/J2EE Interface Developer

Responsibilities:

  • Created Use case, Sequence diagrams, functional specifications and User Interface diagrams using Star UML.
  • Involved in complete requirement analysis, design, coding and testing phases of teh project.
  • Participated in JAD meetings to gather teh requirements and understand teh End Users System.
  • Developed user interfaces using JSP, HTML, XML and JavaScript.
  • Generated XML Schemas and used XML Beans to parse XML files.
  • Developed web application called iHUB (integration hub) to initiate all teh interface processes using Struts Framework, JSP and HTML.
  • Developed teh interfaces using Eclipse and JBoss.
  • Involved in integrated testing, Bug fixing and in Production Support

Environment: Java, Servlets, JSP, Java Mail API, Java Script, HTML, MySQL, Swing, Java Web Server, JBoss, RMI, Rational Rose, Red Hat Linux.

Confidential

Java Developer

Responsibilities:

  • Used HTML, JavaScript, Servlets, and JSP in teh presentation tier. Developed business delegates to communicate wif teh Enterprise Java Beans (EJB) and web services residing on teh business tier.
  • Responsible and active in teh analysis, definition, design, implementation, management, and deployment of full software development life cycle of project.
  • Development of teh application dat was used to create dynamic JSPs, given input from a database table containing information about teh contents of teh JSP being developed.
  • Wrote Servlets programming and JSP scripting for teh communication between web browser and server.
  • Responsible for coding SQL Statements and Stored procedures for back end communication using JDBC
  • Coded different deployment descriptors using XML. Generated Jar files are deployed on Web Logic Server.
  • Involved in teh development of presentation layer and GUI framework in JSP. Client Side validations were done using JavaScript.
  • Developed an API to write XML documents from a database. Utilized XML and XSL Transformation for dynamic web-content and database connectivity.
  • Involved in teh development of presentation layer and GUI framework in JSP. Client Side validations were done using JavaScript.

Environment: J2EE, Java, XML, JAXB, JNDI, SOAP, WSAD, Web sphere Application Server, Linux, SQL Server, Rational Rose, and Oracle.

We'd love your feedback!