We provide IT Staff Augmentation Services!

Sr Hadoop Developer Resume

0/5 (Submit Your Rating)

Lincoln, NE

SUMMARY

  • Dedicated Software Engineer with solid experience in full life - cycle Big Data and Hadoop development, including requirements definition, prototyping, design, implementation, and maintenance of data. A skilled project developer able to direct multiple tasks effectively and readily master innovative software and tools.
  • Technically accomplished professional with 8+ years of total experience in software development and requirement analysis in agile work environment and 5+ years of experience in Big Data and Hadoop Ecosystem like PIG, HIVE, SPARK, SQOOP, OOZIE, KAFKA, IMPALA,HBASE, JIRA, ZOOKEEPER and FLUME.
  • 3 years of experience in JAVA programming with various technologies like Java, J2EE, JavaScript and Data Structures, JDBC, HTML, XML, Junit, JQuery, Struts MVC, JSP, Servlets.
  • Excellent understanding / knowledge of Big Data and Hadoop architecture.
  • Good experience in writing Map Reduce jobs using Java native code, Pig, Hive for various business use cases.
  • Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Hive, Spark, Kafka, Sqoop, Pig, Zookeeper, Oozieand Flume.
  • Hands on experience with Amazon EMR, Cloudera (CDH4 & CDH5), and Horton WorksHadoop Distributions.
  • Experienced in data loading and retrieving from different data sources into HDFS systems using Kafka producers, consumers and Kafka brokers.
  • Built multiple POCs using NoSQL database clusters like Cassandra, MongoDB and HBase.
  • Experience in working with different Spark components like Spark transformations, Mlib, Streaming and Spark QL.
  • Experience in developing customized UDF’s in java to extend Hive and Pig Latin functionality.
  • Developed various UNIX Shell and Python scripts to automate various Processes & good experience in resolving errors making enhancements to existing scripts. Experience in managing Hadoop cluster using Cloudera Manager.
  • Good knowledge of Oozie concets like design, development, and execution of workflowsin Oozie.
  • Implemented the industry's largestDataLakeon theBigDataplatform. Sourced and loaded Petabyte ofdatafrom multiple source system to create enterprisedatahub to support needs of various Business Units and programs.
  • Have experience in installing, configuring and using various Hadoop distributions like Cloudera and Horton Works.
  • Knowledge of Data Warehousing and Analytics, Data migration, Data profiling, ETL Processes and Data mining.
  • Well versed in designing and implementing MapReduce jobs using JAVA on Eclipse to solve real world scaling problems.
  • Experience in Object Oriented Analysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.
  • Experience in back-end procedure development, for RDBMS, Database Applications using SQL and PL/SQL. Hands on experience on writing Queries, Stored procedures, Functions and Triggers by using SQL.
  • Hands on experience working with Non-Relationaland NoSQLdatabases.
  • Implemented connectivity to database using JDBC API from Servlets and JSP through Java Beans & Enterprise Java Beans (EJB).
  • Solid understanding of the high volume, high performance systems.
  • Have flair to adapt to new software applications and products, self-starter, have excellent communication skills and good understanding of business work flow.
  • Skilled in numerous SDLCmethodologies, including Agile Scrum, XP and RUP.
  • Experience with operating systems: Windows, Solaris, Red-hat, Cent OS, Mac OS and Ubuntu.

TECHNICAL SKILLS

HADOOP/BIG DATA: HDFS, MapReduce, YARN, HBase, Spark, Kafka, Flume, Pig, Hive, Impala, Sqoop, Flume, Oozie, Zookeeper, Jira.

HADOOP DISTRIBUTIO: Cloudera (CDH3, CDH4, and CDH5), Hortonworks and Map Reduce.

CODE REPOSITORY: Subversion, GitHub.

DATABASE: SQL Server 2012, MySQL, Oracle, Teradata.

PROGRAMMING LANGUAGES: Core Java, Servlets, JSP, JDBC, Java Beans, Jenkins, PL/SQL, Python, Scala, Shell Scripts

WEBTECHNOLOGIES: JSON, XML, XSLT

DEVELOPMENT TOOLS: Eclipse, Maven, Ant, JUNIT, Log4j.

OPERATING SYSTEMS: Windows XP/NT/2007/2010/, Unix, Linux, Mac OS

TOOLS: Microsoft Office: Excel, Visio, Word, PowerPoint,, Outlook, Lync, OneNote, Access.

METHODOLOGIES: Agile, Singleton, Waterfall

PROFESSIONAL EXPERIENCE

Confidential | Lincoln, NE

Sr Hadoop Developer

Responsibilities:

  • Developed Map reduce programs to perform extraction, transformations and load operations.
  • Written Hive and SparkUDFs to perform day to day data validations, Quality Analysis and counters during imports and exports
  • Used Scala in Spark programs to improvise the report generation, data traceability and data quality checks.
  • Created many B-teq scripts to perform insert, delete and update operations in Teradata.
  • Developed Oozie/Sqoop job to import data from multiple RDBMS sources like Teradata, SQLServer and Netezza using JDBC drivers with parallel load processes and export to HDFS location as well as Linux file systems.
  • Developed Oozie workflows with different oozie actions to run Hive, Spark, Sqoop, Pig and Shell scripts using XML
  • Worked in On-Call Production Support to provide L3 support during Monthly and Quarterly releases.
  • Experience in creating, managing and deploying release branches, tags across multiple systems using Jenkins, SVN and ShellScripts
  • Hands-on experience with version control and code deployment and troubleshooting.
  • Have a working knowledge with Hadoop security components like Kerberos and Sentry architectures.
  • Have hands on experience with Python coding and its integration with oozie workflows.
  • Build Hadoop sandbox environments with components like Hive, Sqoop, Pig to work with other RDMSs and applications such as Teradata, SAS, R and SQL Server to provide the Hadoop exposure and introduction to the business users.
  • Managing and scheduling Jobs on a Hadoop cluster using Oozie work flows and Autosys job schedulers.
  • Handled importing of data from various sources, performed transformations using Pig and loaded data into HDFS and Extracted the data from SQL Server and Teradata into HDFS using Sqoop.
  • Developed POCs to deploy the latest codesautomatically to various clients using Java and VersionOne APIs.
  • Performed importing data from various sources to theCassandracluster using Java/Python APIs.
  • Build POCs like sourcing data from HDFS to generate monthly reports and infrographics, data tracing and data quality for every month that can be included to our in-house Web Portal as new features so that Business Users will have better transparency without any manual interventions.

Environment: CDH 5.8.3, HDFS, MapReduce, Hive, Oozie, Sqoop, Spark, Jenkins, Teradata, SQL Server, Subversion, Eclipse Mars.2, JDK 1.8.0 101 and Autosys.

Confidential | Westlake, TX

Hadoop Consultant/Developer

Responsibilities:

  • Developed Map Reduce application using Hadoop Map reduce programming, a framework for processing.
  • Injected large data sets in parallel across the Hadoop cluster for pre-processing.
  • Hands on experience with continuous monitoring and managing of Hadoop cluster using Cloudera Manager.
  • Developed Map Reduce programs to process the different file formats for performing transformations.
  • Developed Map Reduce Program to analyze the production log files for application issues and performances.
  • Experienced in implementing different kind of joins to integrate data from different data sets like Map and reduce side join.
  • Implemented Map Reduce programs to handle semi/unstructured data like text, xml, json files.
  • Developed the code for Importing and exporting data into HDFS and Hive using Sqoop.
  • Responsible for writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL).
  • Extensive hands on experience with HBaseand Cassandra.
  • Configured HBase for automatic sharding of tables, automatic failover support between RegionServers.
  • Created integration between Cassandra and Hadoop clusters for analytic jobs.
  • Involved in defining job flows using Oozie for scheduling jobs to manage apache Hadoop jobs
  • Importing data from Oracle to HDFS & Hive for analytical purpose.
  • Analyzing imported data in HDFS & Hive using HiveQL and custom Map Reduce programs in Java
  • Developing Hive User Defined Functions in java, compiling them into jars and adding them to the HDFS and executing them with Hive Queries.
  • Responsible to manage data coming from different sources.
  • Assisted in monitoring the Hadoop cluster using Ganglia tool.
  • Tested and reported defects in an Agile Methodology perspective.
  • Consolidate all defects, report it to PM/Leads for prompt fixes by development teams and drive it to closure.

Environment: Hadoop, HDFS, MapReduce, Hive, Oozie, MongoDB, Java (jdk1.6), NoSQL, MySQL, SQL, Perl and Ganglia

Confidential | Chicago, IL

Hadoop Consultant/Developer

Responsibilities:

  • Worked on analyzing, writingHadoopMapReduce jobs using JavaAPI, Pig and Hive.
  • Responsible for building scalable distributed data solutions usingHadoop.
  • Involved in loading data from edge node to HDFS using shell scripting.
  • Created HBase tables to store variable data formats of PII data coming from different portfolios.
  • Exported the analysed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Analyze large and critical datasets using Cloudera, HDFS, Hbase, MapReduce, Hive, Hive UDF, Pig, Sqoop, Zookeeper, & Spark.
  • Developed custom aggregate functions using Spark SQL and performed interactive querying.
  • Used Pig to store the data into HBase.
  • Creating Hive tables, dynamic partitions, buckets for sampling, and working on them using HiveQL.
  • Used Pig to parse the data and Store in Avro format.
  • Stored the data in tabular formats using Hive tables and Hive SerDes.
  • Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
  • Worked with NoSQL databases like Hbase in creating Hbase tables to load large sets of semi structured data coming from various sources.
  • Implemented a script to transmit information from Oracle to Hbase using Sqoop.
  • Worked on tuning the performance Pig queries.
  • Involved in writing the shell scripts for exporting log files toHadoopcluster through automated process.
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Implemented MapReduce programs to handle semi/unstructured data like XML, JSON, and sequence files for log files.
  • Installed Oozie workflow engine to run multiple Hive and pig jobs.
  • Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.

Environment: Hadoop, HDFS, MapReduce, Hive, Oozie, MongoDB, Java (jdk1.6), NoSQL, MySQL, SQL, Perl and Ganglia.

Confidential

Java Developer

Responsibilities:

  • Developed programs by following the agile methodology.
  • Involved in the implementation of design using vital phases of the Software development life cycle (SDLC) that includes Development, Testing, Implementation and Maintenance Support.
  • Applied OOAD principles for the analysis and design of the system.
  • Used WebSphere Application Server to deploy the build.
  • Developed front-end screens using AJAX, JSP, HTML, JQuery, Java Web Services and CSS.
  • Used Spring Framework for developing business objects.
  • Performed data validation in Struts Form beans and Action Classes.
  • Used Eclipse for the Development, Testing and Debugging of the application.
  • Used DOM Parser to parse the xml files.
  • Log4j framework has been used for logging debug, info & error data.
  • Used Oracle 10g Database for data persistence.
  • SQL Developer was used as a database client.
  • Used WinSCP to transfer file from local system to other system.
  • Performed Test Driven Development (TDD) using JUnit.
  • Used Ant script for build automation.
  • Used Rational ClearQuest for defect logging and issue tracking.

Environment: Windows XP, UNIX, Java5.0, Design Patterns, Web sphere, AJAX, J2EE (Servlets, JSP), HTML, JSON, JavaScript, Java Web Services, CSS, Eclipse, SQL Server, JUnit.

Confidential

Java/J2EE Developer

Responsibilities:

  • Developed activity, sequence and class diagrams using Unified Modeling Language and RationalRose.
  • Developed HTML and JSP pages using Struts.
  • Developed Controller Servlets, Action and Form objects for process of interacting with Oracle databaseand retrieving dynamic data.
  • Responsible for coding SQL Statements and Stored procedures for back end communication using JDBC.
  • Used Apache Log4J logging API to log errors and messages.
  • Developed XML parser for File parsing.
  • Involved in writing Detail Design Documents with UML Specifications.
  • Involved in unit testing and system testing and also responsible for preparing test scripts for the system testing.
  • Responsible for packaging and deploying components in to the WebSphere.
  • Developed backend components, DB Scripts for the backend communication.
  • Responsible for performance tuning of the product and eliminating memory leakages in the product.

Environment: Java, Java Beans, JSP, Servlets, JDBC, LOG4J, IBM DB2, XML, HTML, Struts 1.2, Web Sphere, CVS

We'd love your feedback!