SUMMARY

• Well - organized, Target oriented and dedicated professional with more than 10+ years of experience in IT Industry, having nearly 3+ years of experience in Hadoop full Stack Development and Big Data Analytics in Cloudera, Hortonworks Hadoop Environments and Agile, Scrum software development methodologies.
• Experienced in working with Cloudera CDH and Hortonworks HDP environments.
• Worked on scalable distributed computing systems, software architecture, data structures and algorithms using Hadoop, Apache Spark etc.
• Experienced in performing Importing and exporting the data between HDFS and RDBMS
• Hands on in writing Map Reduce jobs in Hive, Pig and Impala.
• Experience on Storage and Processing data using Hue
• Efficient in analyzing data using HiveQL, Pig Latin, partitioning an existing data set with static and dynamic partition, tuning data for optimal query performance.
• Experience with various performance optimizations like distributed cache for small datasets, Partitions, Bucketing in Hive and Map side joins.
• Experience in working with different data sources like Text files, XML files, JSON files and Databases.
• Good Experience in Working with different file formats such as Text, Sequence files, Avro, ORC and Parquet.
• Knowledge on ingesting data into Hive for ETL operations.
• Proficient in using Big data ingestion tools like a Sqoop and Flume
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs.
• Good knowledge in writing Spark application using Spark-Scala and Spark-SQL
• Extensive Knowledge on Tuning Spark jobs Running on Production Cluster
• Extensive experience with relational databases and hands on experience on Linux systems.
• Rich experience in writing Ad-hoc Programs.
• Expertise in Product requirement analysis, high/detail level designing, developing and testing various Mainframes applications including COBOL, JCL, CICS,VSAM, Z/OS Utility, DB2 SQL Utility, QMF, SPUFI, File manager, IBM MQ series, Endeavor, changeman, Remedy,CA7,Smartest,Xpediter,IBM Debug, Inter test, File Aid, Smart test, ISPF Utility, FTP,NDM, AbendAid and MVS.
• Well versed with software life cycle with inclination to Quality, instrumental in achieving and running the projects run at CMM level 5.
• Quick learner of new technologies and positive attitude towards the changes in the Industry.

TECHNICAL SKILLS

Big Data Technologies: Hadoop, YARN, MapReduce, Sqoop, Flume, Oozie, Hive, Impala, Tez, Pig, Spark, SparkSQL, Spark Streaming, HBase, MongoDB, Cassandra, Kafka, Zookeeper, ElasticSearch.

Programming languages: Core Java, Scala, Python, Pig Latin, COBOL, CICS, JCL, EZTRIEVE, MQSeries, CWS-CICS Web Support and REXX

Databases: Oracle, SQL Server, DB2, IMS, VSAM, SQL, and IDMS.

NoSQL Databases: HBase, MongoDB, Cassandra.

Well-versed in different OS: Windows, Linux, Mac

Cloud Environments: AWS, Azure.

Network protocols: HTTP, FTP, TCP/IP

Methodologies: Agile, Waterfall

Version Tools: Git, SVN and CVS.

Tools: & Utilities: Putty, WinSCP, FileZilla, CompuSet, Xpeditor, File-Aid, Endevor, Job Track, File Manager, FTP, NDM, Changeman, Intra test, PANVALET, CA-7/CA-11, ISPF edit Macros, MICRO Focus Workbench and Micro Focus Mainframe Express, Net Express, Autosys,VC4U,BMC Utility (load, unload) and IBM utility(IDCAMS,IEBCOPY,IEBCOMPR,IEBGENER,IEFBR14,SORT,DFSORT,SYCSORT.

Cluster Management tools: Cloudera Manager, Ambari

OLTP: CICS, IMS DC

PROFESSIONAL EXPERIENCE

Confidential - Wayne, PA

Sr. Developer

Responsibilities:

• Extensively involved in Design phase and delivered Design documents in Hadoop eco system with HDFS, HIVE, PIG, SQOOP and SPARK with SCALA.
• Collected the logs from the physical machines and the Open Stack controller and integrated into HDFS using Kafka.
• Involved in the high-level design of the Hadoop architecture for the existing data structure and Business process
• Worked with clients to better understand their reporting and dash boarding needs and present solutions using structured Agile project methodology approach.
• Worked on analyzing Hadoop cluster and different Big Data Components including Pig, Hive, Spark, HBase, Kafka, Elastic Search, database and SQOOP.
• Involved in loading disparate datasets into Hadoop Data Lake, this would be available to the data science team to predict the future.
• Implemented Partitioning, Dynamic Partitions and Buckets in HIVE for increasing performance benefit and helping in organizing data in a logical fashion.
• Installed Hadoop, Map Reduce, HDFS, and developed multiple Map-Reduce jobs in PIG and Hive for data cleaning and pre-processing.
• Created tables in HBase to store variable data formats of PII data coming from different portfolios
• Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement.
• Experienced in pulling the data from Amazon S3 bucket to Data Lake and built Hive tables on top of it and created data frames in Spark to perform further analysis.
• Developed data pipeline using Flume, Sqoop, Pig and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.
• Involved in loading data from UNIX file system to HDFS using Flume and HDFS API.
• Developed RDD's/Data Frames in Spark using Scala and Python and applied several transformation logics to load data from Hadoop Data Lake to Hbase
• Used various HBase commands and generated different Datasets as per requirements and provided access to the data when required using grant and Revoke
• Created Hive tables as per requirement as internal or external tables, intended for efficiency.
• Created and maintained Technical documentation for launching Hadoop Clusters and for executing Hive queries and Pig Scripts
• Worked in tuning Hive & Pig to improve performance and solved performance issues in both scripts
• Involved unit testing, interface testing, system testing and user acceptance testing of the workflow tool.
• Used JIRA for bug tracking and GIT for version control.
• Involved in story-driven agile development methodology and actively participated in daily scrum meetings.

Confidential - Wayne, PA

Responsibilities:

• Involved in loading data from UNIX file system to HDFS.
• Worked on importing and exporting data from MySQL and DB2 into HDFS and HIVE using Sqoop.
• Involved in loading data from local file system (LINUX) to HDFS
• Involved in running Ad-Hoc query through PIG Latin language, Hive.
• Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.
• Created Hive External tables and loaded the data into tables and query data using HQL.
• Handled importing of data from various data sources, performed transformations using Hive, MapReduce and loaded data into HDFS.
• Developed Simple to complex Map Reduce Jobs using Hive and Pig
• Analyzed large data sets by running Hive queries and Pig scripts
• Involved in creating Hive tables, loading with data and writing hive queries that will run internally in MapReduce way.
• Experienced in defining job flows using Oozie
• Experienced in collecting, aggregating, and moving large amounts of streaming data into HDFS using Flume.
• Load and transform large sets of structured, semi structured and unstructured data
• Responsible to manage data coming from different sources and application
• Working Knowledge in NoSQL Databases like HBase.
• Good Knowledge of analyzing data in HBase using Hive and Pig.
• Involved in Unit level and Integration level testing.
• Prepared design documents and functional documents.
• Involved in running Hadoop jobs for processing millions of records of text data
• Experienced in running Hadoop streaming jobs to process terabytes of xml format data.

Confidential

Sr. Developer

Responsibilities:

• Wrote letter process COBOL, DB2 programs and process. Modified letter process COBOL, DB2 programs to address defects found during Systems Integration and User Acceptance testing. Extracted sample test data from test and production DB2 tables using SQL and SPUFI.
• Maintained Online and Batch COBOL, DB2, CICS, DB2 Stored procedure programs and jobs. Coded and tested COBOL, DB2, CICS and DB2 stored procedure batch and online screen programs for the MFS System from specification.

Confidential - Paramus, NJ

Sr. Developer

Responsibilities:

• Wrote specification for online programs coded and tested COBOL, CICS (CWS-CICS Web Support), DB2 stored procedure program from specification.
• Developed eTT-Damage Notification System COBOL, CICS, DB2, online and batch COBOL Program.
• CWS-CICS Web Support is used as an interface between Front-End & Back-End

Confidential - Princeton, NJ

Sr. Developer

Responsibilities:

• Interacting with Client and various functional users for requirement gathering, business process understanding, system analysis and finalization of technical & functional specifications.
• Coding new modules and enhancing the existing modules like MCH-CA ( Corporate Action) and MCH-Reporting.
• Developed batch reporting programs for MCH-Reporting system COBOL, CICS online programs.

Confidential, NJ

Responsibilities:

• Conversion from Eztrieve Program to Cobol Program to support in Microfocus.
• Conversion from DB2 database to VSAM database according to the client requirement.
• Develop new MICRO FOCUS COBOL screens
• Good Experience in Micro Focus COBOL and Net Express
• Quality Center (QC 9.2) as management tool - Used quality center for requirement maintenance, execution of cases and logging defects.

Environment: COBOL, JCL, VSAM, DB2, DB2 stored procedures, SQL, EZTRIEVE, ENDEVOR, MQSeries, INTERTEST,CICS, FTP, NDM, Micro Focus Mainframe Express, Net Express, AUTOSYS, BMC Utility - load and unload, IBM Utility and FILE AID.

Confidential

Responsibilities:

• Responsible for various duties like designing, developing, debugging, coding, writing specifications and other assigned duties related to programming and Modification of Shandi screens.
• Involved in designing new screen and coding using COBOL with CICS program and also involved in modification of existing screens using CICS.
• Performance tuned production and test jobs and NDM transmission. Reduced run times from 20% to 90%.

Environment: Z/OS Utility, DB2,SQL,DB2 stored procedure, MS DB, CICS,MQ Series, JCL, EZTRIEVE, SHANDI, AbendAid, FTP,NDM, IBM Utility, UNIX and COBOL.

Confidential

Responsibilities:

• Writing new programs as per the Client request, reviewing the Ad-hoc Programs and Timely Deliver the Deliverables etc. Audit trail of changes.
• Responsible for various duties like designing, developing, debugging, coding, writing specifications and other assigned duties related to programming.
• Having experience with Omni 5.2 version.

Environment: Z/OS Utility, COBOL, JCL, VSAM, DB2, REXX, XPEDITOR, FILE-AID, CHANGEMAN, PANVALET and IBM Utility.