We provide IT Staff Augmentation Services!

Hadoop Developer Resume

Jersey City, NJ

Hire Now

SUMMARY:

  • Around 10 years of professional IT experience including 2.5 years in Hadoop/Big data ecosystem. Domain worked include: Li fe a n d P ens i o n in s u rance, Financia l a n d B a n ki n g appli ca ti ons .
  • Comm i tt ed tea m p la y er wi th s tr ong a n al ytica l a n d p robl e m s ol vin g s kill s, will i n g n e ss to q u i ckl y adap t to n ew e n viro n m e n t & tech n ol ogi es, d edi cat ed to su cce s sf ul p r oj ect comp l eti on a nd ex cel l e n t commu n i ca ti on a n d i n terper s on a l s kill s .
  • 2.5 years of hands on experience in Hadoop Eco system technologies such as in Pig, Hive, HBase, Map Reduce, Oozie, Flume, Kafka and Sqoop.
  • Good Understanding of Hadoop architecture and Hands - on experience with Hadoop components such as Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce concepts in HDFS Framework.
  • Handled importing and exporting data using Sqoop from HDFS to RDBMS and vice-versa.
  • Hands on experience in writing map-reduce programs in java.
  • Experience in Apache Spark using pyspark and scala to process real time data.
  • Experience in analytics with Apache Spark (RDD, DataFrames and Streaming API).
  • Used Spark Streaming to divide streaming data into batches as an input to Spark engine for batch processing.
  • Performed Importing and exporting data into HDFS, Hive and HBase using Sqoop.
  • Hands-on experience on using Cloudera Platform.
  • Hands on experience in Installation of Hadoop Clusters.
  • Extracted data from databases with SQL and used Python for analytics and data cleaning.
  • Analyzing the data using statistical features in Tableau to develop trend analysis.
  • Involved in consuming data from RESTful web services.
  • Excellent knowledge with Unit Testing, Regression Testing, Integration Testing, User Acceptance Testing, Production implementation and Maintenance.
  • Ha nd s on exp eri ence wi th OS /390, Z / OS, ISP F, C O BO L, P L 1, D B2, C ICS, JC L, VSA M, S YNC S O RT, Ea s y t rie ve, REXX, Fil e Aid, Xp edi tor, Inter Te s t, IB MD eb u gg er, E n d e vor, O P C, Sp u fi, IB M Utili ti es, RD z a n d Ch a n g e co nfig urat ion s y s tem .
  • Worked on annual statistics reporting which is used for generating detailed information about the pension capitals in Confidential .
  • Br i ef exp osur e i n Ora cl e10g, TOAD, Inf orma ti ca P owe r, Python.
  • Brief exposure in ETL T ran s fo rma ti ons su c h a s S our ce Quali fi er, L oo k up, Fi l ter, Ex p r es s i on, Ro u ter, Joi n er, Upda te S tr a teg y, Ran k, Agg rega tor, S orter, S eq u e n ce G en erato r a n d Norm ali zer.
  • Experience in development using J2EE technologies like JDBC, JSP.
  • Experience working with Waterfall model and Agile Methodology.
  • P erf orm i mpa ct a na ly si s a nd p r ov i d e sol ut ion s to user ’s cha ng e r equ ests.
  • Pr odu cti on i mp l em enta ti on up on successfu l user a cce p ta nce te sti ng .
  • G ood exp eri ence i n tr oub l e sh ooti ng a nd sy stem p erf orma nce tu ni ng .
  • Ex p eri ence i n a na l yz in g th e en ti r e sy stem a nd th e i mpa ct w i th oth er ba ck end a nd fr ont - end sy stem s.

TECHNICAL SKILLS:

Hadoop & Spark\ Programming languages: HDFS, Mapreduce v2.6.x, YARN, HBase \ COBOL, PL1, JCL, REXX, Java, Python, \ v0.98.0, Pig 0.14.0, Pig Latin, Hive 0.14.0, \ Linux shell scripts.\ Sqoop 1.4.4, Flume 1.4.0, Kafka 0.8.0, Impala, Oozie 4.0.1, Hue, Zookeeper 3.4.6, Spark v2.0, Python API, Scala API.

Java & J2EE Technologies\ IDE\: Core Java, JSP, JDBC\ Eclipse

Frameworks\ Databases\: MVC\ MySql 5.0.x, DB2 v10

ETL tools\ BI tool \: Informatica\ Tableau 9.0

Mainframe middleware \ Mainframe Utilities\: VSAM\ TSO, ISPF, Spufi, QMF, DataManager

Mainframe tools\: Endeavor, Expeditor, Intertest, FileMaster, IBM Debugger tool, RDz

PROFESSIONAL EXPERIENCE:

Confidential, Jersey City, NJ

Hadoop Developer

Responsibilities:

  • Developed program to list HBase tables and its corresponding regions to identify the empty regions and then merged it with the adjacent non-empty regions. This was done as part of performance improvement.
  • Loaded Apache log data into Hive table and created hive queries which helped to spot the long running agent queries.
  • Worked on HBase upgrade from 0.94 to 0.96 version.
  • Worked on Cloudera upgrade from CDH 4.3 to CDH 5.7.
  • Written spark-scala script to find the most popular movie.
  • Used Spark RDD to identify the top rated items.
  • Self-Join and filter functions are used to identify the duplicate data.
  • Generated structured formatted data and loaded that into Spark cluster using Spark SQL and DataFrame API.
  • Implemented logic using spark-scala to calculate the average, maximum, minimum value for a given item per year.

Technologies: Hadoop, HDFS, MR 2.0.x, HIVE 0.10.0, Pig, HBase 0.96, MySql, Putty, Zookeeper 3.4.6, Linux and shell scripting, CDH 4.3, CDH 5.9, JSP, Servlet, Scala 2.11.8, Spark 2.0.0.

Confidential, Piscataway, NJ

Software Engineer (Hadoop Developer)

Responsibilities:

  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce and loaded data into HDFS.
  • Implemented Partitioning, Dynamic Partitioning, Buckets in Hive.
  • Involved in creating Hive tables, then applied HQL on those tables for data validation.
  • Used Impala to pull the data from Hive tables.
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Hands on extracting data from different databases and to copy into HDFS file system using Sqoop.
  • Written Sqoop incremental import job to move new / updated info from Database to HDFS.
  • Created Oozie coordinated workflow to execute Sqoop incremental job daily.
  • Load data from various data sources into HDFS using Kafka.
  • Hands on experience in joining raw data with the reference data using Pig scripting.
  • Used different file formats like Text files, Sequence Files, Avro.
  • Implemented Spark RDD transformations, actions to migrate Map reduce algorithms using scala.
  • Created various Parser programs using scala to extract data from unstructured data.
  • Experience in Zookeeper to coordinate the servers in clusters to maintain the data consistency and Monitored services.
  • Used Oozie workflow engine to run multiple Hive and Pig jobs.
  • Used Tableau to generate dashboards for product trend analysis.
  • Working with clients on requirements based on their business needs.

Technologies: Hadoop, HDFS, MR 2.5.x, HIVE 0.14.0, Pig, Sqoop 1.4.4, HBase 0.98, OOzie 4.0.1, MySql, Putty, Spark v1.4, Scala, Flume 1.4.0, Impala, Zookeeper 3.4.6, Linux and shell scripting, Tableau 9.0

Confidential, Denmark

Software Engineer

Responsibilities:

  • Worked closely with the business analysts to convert the Business Requirements into Technical requirements and prepared low and high level documentation.
  • Hands on using log files and to copy them into HDFS using flume .
  • Hands on writing Map Reduce code to make unstructured data as structured data and for inserting data into HBase from HDFS.
  • Experience in creating integration between Hive and HBase.
  • Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce and loaded data into HDFS.
  • Involved in loading data from Linux file system to HDFS.
  • Hands on experience in joining raw data with the reference data using Pig scripting.
  • Hands on extracting data from different databases and to copy into HDFS file system using Sqoop.
  • Written Sqoop incremental import job to move new / updated info from Database to HDFS.
  • Created Tableau reports and dashboards for business users to show the number of policies falling under a particular category of products.
  • Involved in review process and as a senior member in the team, helped new team member to get involved in the project assignments.
  • Established custom Map Reduces programs in order to analyze data and used Pig Latin to clean unwanted data.

Technologies: Hadoop, HDFS, MR 2.3.x, HIVE 0.12.0, Pig, Sqoop 1.4.1, HBase 0.98.0, DB2 v8, Putty, Zookeeper 3.4.5, UNIX and shell scripting, Tableau 8.0

Confidential

Software Engineer

Responsibilities:

  • Involved in analyzing a nd und ersta nd i ng th e r equ i r em ent s & f un cti ona l spec i f i cat i ons from client.
  • Pr epared techni cal spec i f i cat i ons ba s ed on th e ex i st i ng f un cti ona li ty a nd r equ i r em ent s. Care was taken to r e - use m ost of th e ex i st i ng comp onent s /m odu l es.
  • Involved in estimating the task once after the high level solution is being approved by the client.
  • Performed analysis for requirement changes to find out the affected list.
  • Implemented services using Core Java.
  • Developed and deployed UI layer logics of sites using JSP.
  • Business logic was implemented using COBOL and PL1 language. DB2 was used for data storage and retrieval.
  • Work ed on scre en ch a ng es u si ng G em i n i a nd Ne tsy rtool s in EDI and DLIPS Web applications. Gemini and Netsyr tools are built on top of HTML and Javascript.
  • Worked on CICS screen maintenance for implementing business changes.
  • Performed debugging using IBM Debugger tool for understanding and fixing the bugs.
  • System testing was performed using QC tool to keep track of the defects.
  • Change configuration management tool was used for version control.
  • FTPJobs were used for sending the report to the client mailbox.
  • Involved in review process and as a senior member in the team, helped new team member to get involved in the project assignments.

Technologies: Java, Eclipse, Web Sevices, DB2, COBOL, PL1, JCL, CICS

Confidential, Franklin Lakes, NJ

System Engineer

Responsibilities:

  • Involved in analyzing f un ct i o na l specification, finding a ff ected li st of pr ogram s a nd ho m ogeneous i m p l em enta t i on.
  • Prepared Technical specification based on the existing functionality and requirement.
  • Developed pr ogram s a nd j obs usi ng JCL, CO BO L, DB2, CICS, and REXX.
  • Used Xp edi tor tool for debugging and to understand the program flow.
  • Created detailed technical design specification for enhancing the batch programs,
  • Care was taken to re-use most of the existing components/modules.
  • Responsible for correct versioning of code by creating and moving the package using E n d e vor.
  • Involved in preparing test plans for unit and system testing.
  • Followed coding standards to ensure code consistency.

Technologies: COBOL, REXX, JCL, DB2, CICS

Confidential, Mellon, NYC

Mainframe Programmer

Responsibilities:

  • Coord i na ted w i th m a nag em ent to deliver the task within the time limit and good quality.
  • Involved in development/enhancement of applications using COBOL, JCL, VSAM, DB2
  • Inv ol v ed i n pr odu ct i o n supp ort a cti v i t i es. E nsur ing th e ba tch cy cle g ets comp l eted in time.
  • Also, fixed the issues within the time mentioned i n Serv i ce Level Agr ee m ent (SLA).
  • Inv o l v ed in f ix i ng a b end s such a s Spa cea bend s, Fil e cont ent i on err ors, VSA M spa ce ab end s a nd DB2 ab end s.
  • Inv o l v ed i n m on i t ori ng th e Ba tch C yc l es
  • As par t of v a l ue add, tools were crea ted using REXX to mak e t he r out i ne ta sk easier a nd fa ster .

Technologies: COBOL, REXX, JCL, DB2, VSAM

Hire Now
Client Services
Job Seekers
Visa Sponsorship