We provide IT Staff Augmentation Services!

Hadoop Developer Resume

Jersey City, NJ


  • Around 10 years of professional IT experience including 2.5 years in Hadoop/Big data ecosystem. Domain worked include: Li fe a n d P ens i o n in s u rance, Financia l a n d B a n ki n g appli ca ti ons .
  • Comm i tt ed tea m p la y er wi th s tr ong a n al ytica l a n d p robl e m s ol vin g s kill s, will i n g n e ss to q u i ckl y adap t to n ew e n viro n m e n t & tech n ol ogi es, d edi cat ed to su cce s sf ul p r oj ect comp l eti on a nd ex cel l e n t commu n i ca ti on a n d i n terper s on a l s kill s .
  • 2.5 years of hands on experience in Hadoop Eco system technologies such as in Pig, Hive, HBase, Map Reduce, Oozie, Flume, Kafka and Sqoop.
  • Good Understanding of Hadoop architecture and Hands - on experience with Hadoop components such as Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce concepts in HDFS Framework.
  • Handled importing and exporting data using Sqoop from HDFS to RDBMS and vice-versa.
  • Hands on experience in writing map-reduce programs in java.
  • Experience in Apache Spark using pyspark and scala to process real time data.
  • Experience in analytics with Apache Spark (RDD, DataFrames and Streaming API).
  • Used Spark Streaming to divide streaming data into batches as an input to Spark engine for batch processing.
  • Performed Importing and exporting data into HDFS, Hive and HBase using Sqoop.
  • Hands-on experience on using Cloudera Platform.
  • Hands on experience in Installation of Hadoop Clusters.
  • Extracted data from databases with SQL and used Python for analytics and data cleaning.
  • Analyzing the data using statistical features in Tableau to develop trend analysis.
  • Involved in consuming data from RESTful web services.
  • Excellent knowledge with Unit Testing, Regression Testing, Integration Testing, User Acceptance Testing, Production implementation and Maintenance.
  • Ha nd s on exp eri ence wi th OS /390, Z / OS, ISP F, C O BO L, P L 1, D B2, C ICS, JC L, VSA M, S YNC S O RT, Ea s y t rie ve, REXX, Fil e Aid, Xp edi tor, Inter Te s t, IB MD eb u gg er, E n d e vor, O P C, Sp u fi, IB M Utili ti es, RD z a n d Ch a n g e co nfig urat ion s y s tem .
  • Worked on annual statistics reporting which is used for generating detailed information about the pension capitals in Confidential .
  • Br i ef exp osur e i n Ora cl e10g, TOAD, Inf orma ti ca P owe r, Python.
  • Brief exposure in ETL T ran s fo rma ti ons su c h a s S our ce Quali fi er, L oo k up, Fi l ter, Ex p r es s i on, Ro u ter, Joi n er, Upda te S tr a teg y, Ran k, Agg rega tor, S orter, S eq u e n ce G en erato r a n d Norm ali zer.
  • Experience in development using J2EE technologies like JDBC, JSP.
  • Experience working with Waterfall model and Agile Methodology.
  • P erf orm i mpa ct a na ly si s a nd p r ov i d e sol ut ion s to user ’s cha ng e r equ ests.
  • Pr odu cti on i mp l em enta ti on up on successfu l user a cce p ta nce te sti ng .
  • G ood exp eri ence i n tr oub l e sh ooti ng a nd sy stem p erf orma nce tu ni ng .
  • Ex p eri ence i n a na l yz in g th e en ti r e sy stem a nd th e i mpa ct w i th oth er ba ck end a nd fr ont - end sy stem s.


Hadoop & Spark\ Programming languages: HDFS, Mapreduce v2.6.x, YARN, HBase \ COBOL, PL1, JCL, REXX, Java, Python, \ v0.98.0, Pig 0.14.0, Pig Latin, Hive 0.14.0, \ Linux shell scripts.\ Sqoop 1.4.4, Flume 1.4.0, Kafka 0.8.0, Impala, Oozie 4.0.1, Hue, Zookeeper 3.4.6, Spark v2.0, Python API, Scala API.

Java & J2EE Technologies\ IDE\: Core Java, JSP, JDBC\ Eclipse

Frameworks\ Databases\: MVC\ MySql 5.0.x, DB2 v10

ETL tools\ BI tool \: Informatica\ Tableau 9.0

Mainframe middleware \ Mainframe Utilities\: VSAM\ TSO, ISPF, Spufi, QMF, DataManager

Mainframe tools\: Endeavor, Expeditor, Intertest, FileMaster, IBM Debugger tool, RDz


Confidential, Jersey City, NJ

Hadoop Developer


  • Developed program to list HBase tables and its corresponding regions to identify the empty regions and then merged it with the adjacent non-empty regions. This was done as part of performance improvement.
  • Loaded Apache log data into Hive table and created hive queries which helped to spot the long running agent queries.
  • Worked on HBase upgrade from 0.94 to 0.96 version.
  • Worked on Cloudera upgrade from CDH 4.3 to CDH 5.7.
  • Written spark-scala script to find the most popular movie.
  • Used Spark RDD to identify the top rated items.
  • Self-Join and filter functions are used to identify the duplicate data.
  • Generated structured formatted data and loaded that into Spark cluster using Spark SQL and DataFrame API.
  • Implemented logic using spark-scala to calculate the average, maximum, minimum value for a given item per year.

Technologies: Hadoop, HDFS, MR 2.0.x, HIVE 0.10.0, Pig, HBase 0.96, MySql, Putty, Zookeeper 3.4.6, Linux and shell scripting, CDH 4.3, CDH 5.9, JSP, Servlet, Scala 2.11.8, Spark 2.0.0.

Confidential, Piscataway, NJ

Software Engineer (Hadoop Developer)


  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce and loaded data into HDFS.
  • Implemented Partitioning, Dynamic Partitioning, Buckets in Hive.
  • Involved in creating Hive tables, then applied HQL on those tables for data validation.
  • Used Impala to pull the data from Hive tables.
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Hands on extracting data from different databases and to copy into HDFS file system using Sqoop.
  • Written Sqoop incremental import job to move new / updated info from Database to HDFS.
  • Created Oozie coordinated workflow to execute Sqoop incremental job daily.
  • Load data from various data sources into HDFS using Kafka.
  • Hands on experience in joining raw data with the reference data using Pig scripting.
  • Used different file formats like Text files, Sequence Files, Avro.
  • Implemented Spark RDD transformations, actions to migrate Map reduce algorithms using scala.
  • Created various Parser programs using scala to extract data from unstructured data.
  • Experience in Zookeeper to coordinate the servers in clusters to maintain the data consistency and Monitored services.
  • Used Oozie workflow engine to run multiple Hive and Pig jobs.
  • Used Tableau to generate dashboards for product trend analysis.
  • Working with clients on requirements based on their business needs.

Technologies: Hadoop, HDFS, MR 2.5.x, HIVE 0.14.0, Pig, Sqoop 1.4.4, HBase 0.98, OOzie 4.0.1, MySql, Putty, Spark v1.4, Scala, Flume 1.4.0, Impala, Zookeeper 3.4.6, Linux and shell scripting, Tableau 9.0

Confidential, Denmark

Software Engineer


  • Worked closely with the business analysts to convert the Business Requirements into Technical requirements and prepared low and high level documentation.
  • Hands on using log files and to copy them into HDFS using flume .
  • Hands on writing Map Reduce code to make unstructured data as structured data and for inserting data into HBase from HDFS.
  • Experience in creating integration between Hive and HBase.
  • Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce and loaded data into HDFS.
  • Involved in loading data from Linux file system to HDFS.
  • Hands on experience in joining raw data with the reference data using Pig scripting.
  • Hands on extracting data from different databases and to copy into HDFS file system using Sqoop.
  • Written Sqoop incremental import job to move new / updated info from Database to HDFS.
  • Created Tableau reports and dashboards for business users to show the number of policies falling under a particular category of products.
  • Involved in review process and as a senior member in the team, helped new team member to get involved in the project assignments.
  • Established custom Map Reduces programs in order to analyze data and used Pig Latin to clean unwanted data.

Technologies: Hadoop, HDFS, MR 2.3.x, HIVE 0.12.0, Pig, Sqoop 1.4.1, HBase 0.98.0, DB2 v8, Putty, Zookeeper 3.4.5, UNIX and shell scripting, Tableau 8.0


Software Engineer


  • Involved in analyzing a nd und ersta nd i ng th e r equ i r em ent s & f un cti ona l spec i f i cat i ons from client.
  • Pr epared techni cal spec i f i cat i ons ba s ed on th e ex i st i ng f un cti ona li ty a nd r equ i r em ent s. Care was taken to r e - use m ost of th e ex i st i ng comp onent s /m odu l es.
  • Involved in estimating the task once after the high level solution is being approved by the client.
  • Performed analysis for requirement changes to find out the affected list.
  • Implemented services using Core Java.
  • Developed and deployed UI layer logics of sites using JSP.
  • Business logic was implemented using COBOL and PL1 language. DB2 was used for data storage and retrieval.
  • Work ed on scre en ch a ng es u si ng G em i n i a nd Ne tsy rtool s in EDI and DLIPS Web applications. Gemini and Netsyr tools are built on top of HTML and Javascript.
  • Worked on CICS screen maintenance for implementing business changes.
  • Performed debugging using IBM Debugger tool for understanding and fixing the bugs.
  • System testing was performed using QC tool to keep track of the defects.
  • Change configuration management tool was used for version control.
  • FTPJobs were used for sending the report to the client mailbox.
  • Involved in review process and as a senior member in the team, helped new team member to get involved in the project assignments.

Technologies: Java, Eclipse, Web Sevices, DB2, COBOL, PL1, JCL, CICS

Confidential, Franklin Lakes, NJ

System Engineer


  • Involved in analyzing f un ct i o na l specification, finding a ff ected li st of pr ogram s a nd ho m ogeneous i m p l em enta t i on.
  • Prepared Technical specification based on the existing functionality and requirement.
  • Developed pr ogram s a nd j obs usi ng JCL, CO BO L, DB2, CICS, and REXX.
  • Used Xp edi tor tool for debugging and to understand the program flow.
  • Created detailed technical design specification for enhancing the batch programs,
  • Care was taken to re-use most of the existing components/modules.
  • Responsible for correct versioning of code by creating and moving the package using E n d e vor.
  • Involved in preparing test plans for unit and system testing.
  • Followed coding standards to ensure code consistency.

Technologies: COBOL, REXX, JCL, DB2, CICS

Confidential, Mellon, NYC

Mainframe Programmer


  • Coord i na ted w i th m a nag em ent to deliver the task within the time limit and good quality.
  • Involved in development/enhancement of applications using COBOL, JCL, VSAM, DB2
  • Inv ol v ed i n pr odu ct i o n supp ort a cti v i t i es. E nsur ing th e ba tch cy cle g ets comp l eted in time.
  • Also, fixed the issues within the time mentioned i n Serv i ce Level Agr ee m ent (SLA).
  • Inv o l v ed in f ix i ng a b end s such a s Spa cea bend s, Fil e cont ent i on err ors, VSA M spa ce ab end s a nd DB2 ab end s.
  • Inv o l v ed i n m on i t ori ng th e Ba tch C yc l es
  • As par t of v a l ue add, tools were crea ted using REXX to mak e t he r out i ne ta sk easier a nd fa ster .

Technologies: COBOL, REXX, JCL, DB2, VSAM

Hire Now