We provide IT Staff Augmentation Services!

Sr. Big Data/hadoop Developer Resume

Little Rock, AR

SUMMARY:

  • A dynamic professional with around 8 years of diversified experience in the field of Information Technology with an emphasis on Big Data/Hadoop EcoSystem tools using industry accepted methodologies and procedures.
  • Around 5 years of experience as Hadoop Developer with sound knowledge in Hadoop ecosystem technologies.
  • Hands on experience in Hadoop eco system components such as HDFS, MapReduce, Yarn, Pig, Hive, Hbase, Oozie, Sqoop, Kafka and Apache.
  • Good understanding in processing of real - time data using Spark.
  • Hands on experience in Importing and exporting data from different databases like MySQL, Oracle, Teradata into HDFS using Sqoop
  • Strong experience working with real time streaming applications and batch style large scale distributed computing applications using tools like Spark Streaming, Kafka, Flume, MapReduce, Hive.
  • Experience working with Cassandra and NoSQL database including MongoDB and Hbase.
  • Managing and scheduling batch Jobs on a Hadoop Cluster using Oozie.
  • Experience in managing and reviewing Hadoop Log files.
  • Used Zookeeper to provide coordination services to the cluster.
  • Experienced using Sqoop to import data into HDFS from RDBMS and vice-versa.
  • Experience and understanding in Spark and Storm.
  • Experience in analyzing data using Hive, Pig Latin, and custom MR programs in Java.
  • Experience in coding web applications using Java Web Technologies- JSP, Servlets and JDBC, full Understanding of utilizing J2EE technology Stack, including Java related frameworks like Spring, ORM Frameworks(Hibernate).
  • Experience in designing the User Interfaces using HTML, CSS, JavaScript and JSP.
  • Developed web application in open source java framework Spring.
  • Developed RESTful Web Services using Spring Rest and Jersey framework.

TECHNICAL SKILLS:

­­­­­­­­­­Hadoop/BigData Ecosystems: HDFS, MapReduce, Pig, Hive, Impala, Sqoop, Oozie, Zookeeper, Flume, Spark and Scala, Hadoop (Cloudera), Hortonworks and Pivotal)

Database: Oracle, MYSQL, SQL Server, NoSQL, Hbase

IDE’s: Eclipse, Netbeans

Languages: C, Java, PIG LATIN, UNIX Shell Scripting, Python

Scripting Languages: HTML, CSS, Java Script, DHTML, XML, JQuery

Web Technologies: HTML, XML, Java Script, JQuery

Web/Application Servers: Apache Tomcat, Web Logic

WORK EXPERIENCE:

Confidential, Little Rock, AR

Sr. Big Data/Hadoop Developer

Responsibilities:

  • Worked on requirement gathering, analysis and translated business requirements into technical design with Hadoop Ecosystem.
  • Involved in preparation of docs like Deployment Instruction documents and QA testing document.
  • Documented the systems processes and procedures for future references.
  • Provide batch processing solution to certain unstructured and large volume of data by using HadoopMap Reduce framework.
  • Used Hortonworks data platform 2.4 to implement our project.
  • Installed and worked on Apache to monitor hadoop jobs.
  • Used HUE to view HDFS directory structure, monitor jobs, Query editors (Hive)
  • Created various Sqoop commands which imports data from Oracle source into Hadoop Distributed file system.
  • Participated in Rapid Application Development and Agile processes to deliver new cloud platform services.
  • Used Java MapReduce programming to format the data for analysis.
  • Created hive UDFs and used them to achieve the proper output.
  • Used Oozie workflows to schedule the Hadoop jobs.
  • Involved in story-driven agile development methodology and actively participated in daily scrum meetings.
  • Involved in SDLC Requirements gathering, Analysis, Development and Testing of application developed using AGILE methodology.
  • Actively participated in software development lifecycle (scope, design, implement, deploy, test), including design and code reviews, test development, test automation.

Environment: Hortonworks Data Platform 2.5, HDFS, YARN, Hive, Sqoop, Oracle 11g, HUE, Linux Shell Scripting, Windows and Unix

Confidential, Seattle, WA

Big data/Hadoop Engineer

Responsibilities:

  • Worked on importing data from various sources and performed transformations using MapReduce, Pig to load data into HDFS.
  • Worked with cross functional teams to develop a Big Data platform.
  • Loaded the data from the different Data sources like (Teradata and DB2) into HDFS using SQOOP and load into Hive tables, which are partitioned.
  • Developed and maintained Big Data streaming and batch applications using Storm.
  • Configured Sqoop jobs to import data from RDBMS into HDFS using Oozie workflows.
  • Developed multiple scripts for analyzing data using Hive and Pig and integrating with HBase.
  • Created HBase tables to store variable data formats coming from different portfolios.
  • Worked with QA and DevOps teams to troubleshoot any issues that may arise during production
  • Created Cassandra tables to load large sets of structured, semi-structured and unstructured data coming from Linux, NoSQL and a variety of portfolios.
  • Developed pig scripts to transform the data into structured format and it are automated through Oozie coordinators.
  • Used Splunk to captures, indexes and correlates real-time data in a searchable repository from which it can generate reports and alerts.

Environment: Hadoop, HDFS, Spark, Strom, Kafka, Map Reduce, Hive, Pig, Sqoop, Oozie, DB2, Java, Python, Splunk, UNIX Shell Scripting.

Confidential, CA

Hadoop Developer

RESPONSIBILITIES:

  • Developed data pipeline using Flume, Sqoop, Pig and MapReduce to ingest customer behavioral data and financial histories into HDFS for analysis.
  • Worked on importing and exporting data from Oracle and DB2 into HDFS and HIVE using Sqoop for analysis, visualization and to generate reports.
  • Installed, Configured and Maintained Apache Hadoop clusters for application development and Hadoop Ecosystem Components like Hive, Hbase, Zookeeper and Sqoop.
  • Developed multiple MapReduce jobs in java for data cleaning.
  • Schedule the jobs with workflow engine like Oozie. Actions can be performed both sequentially and parallely using Oozie.
  • Built wrapper shell scripts to hold this Oozie workflow.
  • Involved in collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
  • Involved in creating Hadoop streaming jobs using Python.
  • Provided ad-hoc queries and data metrics to the Business Users using Hive, Pig.
  • Used Pig as ETL tool to do transformations, event joins and some pre-aggregations before storing the data onto HDFS.
  • Worked on MapReduce Joins in querying multiple semi-structured data as per analytic needs.
  • Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
  • Used Hive and created Hive tables and involved in data loading and writing Hive UDFs.
  • Worked on various performance optimizations like using distributed cache for small datasets, partition and bucketing in hive, doing map side joins etc. .
  • Worked on executors on each assigned cluster nodes-work processes to run computations and store data using spark context
  • Created concurrent access for hive tables with shared and exclusive locking that can be enabled in hive with the help of Zookeeper implementation in the cluster.
  • Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
  • Familiarity with NoSQL databases including HBase, MongoDB.
  • Wrote shell scripts for rolling day-to-day processes and it is automated.
  • Manage and analyze Apache Hadoop log files.
  • Involved in story-driven agile development methodology and actively participated in daily scrum meetings.

ENVIRONMENT: Hadoop, MapReduce, Hive, HDFS, PIG, Sqoop, Oozie, Cloudera, Flume, HBase, Zookeeper, MongoDB, Cassandra, Oracle, NoSQL and Unix/Linux, Kafka, AWS

Confidential

Analyst

  • Analyze large datasets to provide strategic direction to the company.
  • Perform quantitative analysis of product sales trends to recommend pricing decisions
  • Conduct cost and benefit analysis on new ideas.
  • Scrutinize and track customer behavior to identify trends and unmet needs.
  • Develop statistical models to forecast inventory and procurement cycles.
  • Assist in developing internal tools for data analysis.

PROGRAMMER

Confidential
  • Coded, tested, debugged, implemented and documented apps using Java and Ruby.
  • Developed eCommerce solutions and social networking functionality.
  • Designed, developed and maintained eCommerce and social networking applications.
  • Built report interfaces and data feeds.
  • Gathered and collected information from various programs, analyzed time requirements and prepared documentation to change existing programs.

Hire Now