Hadoop Admin Resume
NC
PROFESSIONAL SUMMARY:
- Over 10 years of professional work experience in teh IT Industry; around 2 years of experience wif Hadoop, HDFS, MapReduce and Hadoop Ecosystem (Pig, Hive, HBase).
- Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, JobTracker, TaskTracker, NameNode, DataNode programming paradigm.
- Experience wif leveraging Hadoop ecosystem components including Pig and Hive for data analysis, Sqoop for data migration, Oozie for scheduling and HBase as a NoSQL data store.
- Good Exposure on Apache Hadoop Map Reduce programming, PIG Scripting and Distribute Application and HDFS.
- Experience in NoSQL database MongoDB and Cassandra. Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, and Flume.
- Experience in installation, configuration, supporting and managing - CloudEra's Hadoop platformalong wif CDH3&4 clusters.
- Experience in managing and reviewing Hadoop log files.
- Experience in analyzing data using HiveQL, Pig Latin, HBase and custom MapReduce programs in Java.
- Extending Hive and Pig core functionality by writing custom UDFs.
- Experience in data management and implementation of Big Data applications using Hadoop frameworks.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
- Experienced in deployment of Hadoop Cluster using Puppet tool.
- Experience in Hadoop Shell commands, writing MapReduce Programs, verifying managing and reviewing Hadoop Log files.
- Proficient in configuring Zookeeper, Cassandra & Flume to teh existing Hadoop cluster.
- Strong analytical skills wif ability to quickly understand client’s business needs and create specifications.
- Strong knowledge of Mahout Machine learning, Mongo DB Experience working wif geographically dispersed teams.
- Data Warehousing ETL experience in Informatica Power Center 9.0/8.6/8.5/8.1.1/8.0/7.1.2/7.1.1/7.0/6.2 Power Mart 6.2/5.1.1/5.0/4.7.2.
- Experience in teh Implementation of full lifecycle in Data warehouses and Business Data marts wif Star Schemas and Snow flake Schemas.
- Relational and Dimensional Data Modeling experience.
- Extensive experience in Data warehousing tools including Informatica, Business Objects using different databases.
- Extensively Worked on Data warehousing wif Data Marts, Data Modeling, DataCleansing, ETL tool Informatica and data analyzing reporting Tool.
- Experience in working wif data extraction, transformation and loading using Informatica Power Center.
- Experience in working wif all Informatica modules like Server manager, Repositorymanager, Designer.
- Expertise in administering InformaticaRepository, including creating and managing users.
- Extensive experience in supporting production tasks, data enhancements and code fixes.
- Good working Knowledge in OOA and OOD, using UML and designing use cases.
- Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
- Well versed in Object Oriented languages like C++ and Java.
TECHNICAL SKILLS:
Hadoop / Big Data: HDFS, MapReduce, Hive, Pig, HBase, Sqoop, Flume, Oozie, Zookeeper, MongoDB, Cassandra
Programming and Scripting languages: Java, SQL, Unix Shell Scripting.
ETL Informatica Tools: Informatica Power Center 9.0/8.6/8.5/8.1.1/7.1.3/6.2/5.0, Informatica Power Mart5.1, Power Analyzer 3.5x
Database: MS-SQL Server, Oracle 11g/10g/9i, MySQL, DB2, MS-Access
Frameworks: MVC, Struts, Spring, Hibernate
Web Technologies: TCP/IP, UDP, HTTP
Application Server: Apache Tomcat, JBoss, Web Sphere, Web Logic
Operating Systems: Windows, Ubuntu, Red Hat Linux, UNIX
PROFESSIONAL EXPERIENCE:
Confidential, NC
Hadoop Admin
Responsibilities:
- Installed/Configured/Maintained Apache Hadoop clusters for application development and Hadoop tools like Hive, Pig, HBase, Zookeeper and Sqoop.
- Wrote teh shell scripts to monitor teh health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
- Installed and configured Hadoop, MapReduce, HDFS (Hadoop Distributed File System), developed multiple MapReduce jobs in java for data cleaning.
- Developed data pipeline using Flume, Sqoop, Pig and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.
- Involved in collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
- Collected teh logs data from web servers and integrated in to HDFS using Flume.
- Worked on installing cluster, commissioning & decommissioning of DataNodes, NameNode recovery, capacity planning, and slots configuration.
- Implemented NameNode backup using NFS. This was done for High availability.
- Developed PIG Latin scripts to extract teh data from teh web server output files to load into HDFS.
- Involved in teh installation of CDH3 and up-gradation from CDH3 to CDH4.
- Responsible for developing data pipeline using HDInsight, flume, Sqoop and pig to extract teh data from weblogs and store in HDFS.
- Installed Oozie workflow engine to run multiple Hive and Pig Jobs.
- Use of Sqoop to import and export data from HDFS to RDBMS and vice-versa.
- Used Hive and created Hive tables and involved in data loading and writing Hive UDFs.
- Exported teh analyzed data to teh relational databases using Sqoop for visualization and to generate reports.
- Involved in migration of ETL processes from Oracle to Hive to test teh easy data manipulation.
- Used Hive to analyze teh partitioned and bucketed data and compute various metrics for reporting.
- Worked on importing and exporting data from Oracle and DB2 into HDFS and HIVE using Sqoop.
- Worked on NoSQL databases including HBase, MongoDB, and Cassandra.
- Created Hive External tables and loaded teh data in to tables and query data using HQL.
- Wrote shell scripts for rolling day-to-day processes and it is automated.
- Automated workflows using shell scripts pull data from various databases into Hadoop
- Deployed Hadoop Cluster in Fully Distributed and Pseudo-distributed modes.
- Supported in setting up QA environment and updating configurations for implementing scripts wif Pig and Sqoop.
Environment: Hadoop, MapReduce, Hive, HDFS, PIG, Sqoop, Oozie, Cloudera, Flume, HBase, ZooKeeper, CDH3, MongoDB, Cassandra, Oracle, NoSQL and Unix/Linux.
Confidential, CA
Hadoop Admin
Responsibilities:
- Involved in teh Complete Software development life cycle (SDLC) to develop teh application.
- Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, HBase database and Sqoop.
- Involved in loading data from LINUX file system to HDFS.
- Experience in managing and reviewing Hadoop log files.
- Exported teh analyzed data to teh relational databases using Sqoop for visualization and to generate reports for teh BI team.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Implemented test scripts to support test driven development and continuous integration.
- Supported in setting up QA environment and updating configurations for implementing scripts wif Pig and Sqoop.
- Analyzed large data sets by running Hive queries and Pig scripts.
- Worked on tuning teh performance Pig queries.
- Mentored analyst and test team for writing Hive Queries.
- Installed Oozie workflow engine to run multiple MapReduce jobs.
- Worked wif application teams to install operating system, Hadoop updates, patches, version upgrades as required.
Environment: Hadoop, HDFS, Map Reduce, Hive, Pig, Sqoop, Linux, Java, Oozie, HBase
Confidential, Indianapolis, IN
Informatica Developer
Responsibilities:
- Worked wif data modeler’s team to create Data Flow Diagrams.
- Developed Informatica mappings and mapplets, between sources and operational stagingtargets, and fine-tuned some of them for better performance.
- Created views and altered some of teh dimensional tables to satisfy teh reporting needs.
- Employed Source Qualifier, Lookup, Expression, Aggregator, Rank, Sequence generator and Joiner transformations for populating teh datamarts
- Used pre-session and post-session stored procedures in PL/SQL to improve teh performance of teh session.
- Implemented Type II Slowly Changing Dimensions using Version number.
- Performed migration from 6.2 TO 7.1, and performed backup and performance tuning.
- Extensively Worked wif Variables and Parametersin teh mappings to pass teh values between sessions.
- Instrumental in writing PL/SQL stored procedures and packages for transactional details (insert, update and delete operations)
- Created Migration Documentation and Process Flow for mappings and sessions.
- Created Event Wait, Event Raise, Timer and Control Events in workflow manager according to teh business requirement.
- Responsible for unit testing teh mappings using SQL scripts.
Environment: Informatica PowerCenter 6.2, 7.1, ORACLE 9i, HP-Unix, Windows 2000, Erwin 3.5, SQL*PLUS, SQL, PL/SQL.
Confidential, Pittsburgh, PA
ETL Developer
Responsibilities:
- Analyzed and studied various Schemas for Implementation.
- Created source definitions from teh operational sources and target definitions.
- Designed Sources to Targets mappings from primarily Excel/Flat files/XML to Oracle using Informatica.
- Worked Extensively on Informatica modules - Repository Manager, Designer, and Server Manager.
- Developed mappings, reusable Transformations for ETLusing Informatica Power Center .
- Worked extensively on different types of transformations like source qualifier, expression, filter, aggregator, rank, update strategy, lookup, sequence generator and joiner.
- Command task and decision task based upon business requirements.
- Created workflows and reusable worklets for designed mappings.
- Analyzed teh session, event and error logs for troubleshooting mappings, sessions and workflows.
- Worked wif mapping variables, Mapping parameters and session parameters.
- Extensively used debugger to troubleshoot logical errors.
- Extensively used PL/SQL for creating views, stored procedures on teh tables.
- Involved in partitioning of teh sessions and performance tuning of database, Mappings, as well as Sessions.
Environment: Informatica Power Center 8.x, Oracle 9i, SQL*Plus, PL/SQL, TOAD 7.6, UNIX, Windows NT.
Confidential
Java/J2EE Interface Developer
Responsibilities:
- Created Use case, Sequence diagrams, functional specifications and User Interface diagrams using Star UML.
- Involved in complete requirement analysis, design, coding and testing phases of teh project.
- Participated in JAD meetings to gather teh requirements and understand teh End Users System.
- Developed user interfaces using JSP, HTML, XML and JavaScript.
- Generated XML Schemas and used XML Beans to parse XML files.
- Developed web application called iHUB (integration hub) to initiate all teh interface processes using Struts Framework, JSP and HTML.
- Developed teh interfaces using Eclipse and JBoss.
- Involved in integrated testing, Bug fixing and in Production Support
Environment: Java, Servlets, JSP, Java Mail API, Java Script, HTML, MySQL, Swing, Java Web Server, JBoss, RMI, Rational Rose, Red Hat Linux.
Confidential
Java Developer
Responsibilities:
- Used HTML, JavaScript, Servlets, and JSP in teh presentation tier. Developed business delegates to communicate wif teh Enterprise Java Beans (EJB) and web services residing on teh business tier.
- Responsible and active in teh analysis, definition, design, implementation, management, and deployment of full software development life cycle of project.
- Development of teh application dat was used to create dynamic JSPs, given input from a database table containing information about teh contents of teh JSP being developed.
- Wrote Servlets programming and JSP scripting for teh communication between web browser and server.
- Responsible for coding SQL Statements and Stored procedures for back end communication using JDBC
- Coded different deployment descriptors using XML. Generated Jar files are deployed on Web Logic Server.
- Involved in teh development of presentation layer and GUI framework in JSP. Client Side validations were done using JavaScript.
- Developed an API to write XML documents from a database. Utilized XML and XSL Transformation for dynamic web-content and database connectivity.
- Involved in teh development of presentation layer and GUI framework in JSP. Client Side validations were done using JavaScript.
Environment: J2EE, Java, XML, JAXB, JNDI, SOAP, WSAD, Web sphere Application Server, Linux, SQL Server, Rational Rose, and Oracle.
