SUMMARY:

• 14 years of professional IT/Software development/Consulting/Management experience.
• Around 5 years of hands - on experience in developing and delivering real world Big Data solutions like device performance analysis, Portfolio analysis, Click Stream analysis and Sentiment Analysis using Horton Works/Cloudera Hadoop Distribution, Pig, Hive, Sqoop, Flume, Scala, Spark, Java, Linux, Python, Yarn, Tez, Hbase, Oozie and MongoDB.
• Excellent Experience in Hadoop architecture and various components such as Confidential, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce/Spark programming paradigm.
• Strong experience in collecting and storing log data in Confidential using Apache Flume.
• Capable of processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture.
• Familiar with data architecture including data ingestion pipeline design, Hadoop information architecture, data modeling and data mining, machine learning and advanced data processing.
• Extensively involved in design and development, tuning, deployment and maintenance of NoSQL databases.
• Experience in Importing and exporting data from different databases like DB2, MySQL, Oracle and Netezza into Confidential /Hbase/Mongodb.
• Strong Hands on experience in Multi-Channel Data Integration and Master Data Management.
• Experience includes project estimates, cross functional analysis across various domains, systems & design, development, customization, testing, implementation of various application systems.
• Extensively involved in Data compression, Performance tuning, Scheduling, trouble shooting and Cluster maintenance activities.
• Good experience to provide technical oversight for large complex projects and achieve desired customer satisfaction from inception to deployment in a consulting environment.
• Advanced analytical, problem solving, negotiation and organizational skills with demonstrated ability to multi-task, organize, prioritize and meet deadline.
• Expertise in extending Hive and Pig core functionalities by writing custom UDFs.
• Strong knowledge in Software Development Life Cycle (“SDLC”) processes.
• Good Knowledge on Banking, Finance, Capital Market, Telecom and Health Care Industries.
• Ability to multitask and work multiple projects concurrently.
• Ability to work independently and as part of a team.
• Excellent communication, presentation and organizational skills.

TECHNICAL SKILLS:

Operating Systems: Windows 7/Vista/XP, DOS, Unix, Linux, CentOS, OS390, Z/OS, MVS/ESA

Languages: Java, Scala, COBOL, Confidential, C

Scripting Languages: Linux, Python, Java script, HTML, XML, JSP

Database: Netezza, Oracle, SQL-Server, DB2, MySQL

Big Data Ecosystem: Hadoop MapReduce, Confidential, Hive, PIG, Scala, Spark, Hbase, Mongo DB, Sqoop, Flume, Impala, Kafka, Avro, Parquet, Oozie, Ganglia, Nagios and Zookeeper

Special Software/ Tools: HUE, Ambari, Talend, informatica, Control-M, Cloudera Manager, Tableau, Mongify, Aginity, SQL Work bench, RoboMongo, Adobe Confidential, Eclipse, JDeveloper, HP Quality Centre, Remedy, SVN, JIRA, JEDIT, Notepad++, iNotepad, PAC2000/Remedy, CA7, Endevor, Change man, FILE-AID, Xpeditor, QMF, SPUFI, BMC, VSAM

PROFESSIONAL EXPERIENCE:

Hadoop Architect/Lead Hadoop Developer

Confidential, Baltimore, MD/New York, NY

Software: Hadoop MapReduce, Hive, Impala, Sqoop, Spark, Linux, Scala, Mongo DB, Cloudera Manager Hbase, Hue, Zoo keeper, Sql Server, Informatica, Control-M, IntelliJ Idea, Python, JIRA, SVN, Tableau and Alteryx.

Responsibilities:

• Analyzed the existing Confidential system and prepared estimation, detail design document to modernize the sql-server based systems to big data systems
• Interacted and coordinated with multiple teams to understand the project requirements and task ownership
• Involved in requirement analysis, strategy development, project planning and implementation.
• Imported and exported the data using Sqoop from Relational Database systems to Confidential and vice-versa.
• Converted complex sql-server stored procedures to Hive queries and spark jobs.
• Used different types of hive tables and different compression techniques for various types of data and functionality
• Designed and developed Spark batch jobs for complex business requirements.
• Used impala for various ad hoc requests
• Involved in project automation and various performance tuning activities.
• Prepared automation test scripts to support unit, integration and system testing.
• Provided quick response to internal and external client's ad hoc requests.
• Imported refined data from Hive to Tableau/alteryx for data visualization and report.
• Created standards and guide lines for the design and development, tuning, deployment and maintenance of hadoop projects.
• Guided and managed the team to achieve project goals and milestones.

Senior Hadoop Engineer/Hadoop Architect

Confidential, Hicksville, NY

Software: Confidential, Map Reduce, Pig, Hive, Sqoop, Spark, Scala, Mongo DB, Hbase, Linux, Java, python, Hue, Flume, Avro, Parquet, Yarn, Tez, Talend, Oozie, Ambari, Zoo keeper, Ganglia, Nagios, Tableau, Kafka, Robomongo, Aginity, SQL work bench, SVN, JIRA,JEDIT and Adobe Confidential .

Responsibilities:

• Involved in requirement analysis, design, strategy development, planning, coding, unit testing, deployment and support.
• Designed, developed and implemented map reduce jobs to support end to end solutions using Hive, Pig, Linux, Java and Python.
• Responsible for load, aggregate and move large amounts of log data using Flume.
• Data Import and export using Sqoop from Relational Database systems to Confidential and vice-versa.
• Analyzed large data sets by running Hive queries, Pig scripts and Spark process.
• Created Hive tables, partitioned tables, loaded data to hive tables from various sources and analyzed data using hive queries.
• Developed end to end real time processing of Wi-Fi traps using Flume, Spark, Scala and Python.
• Designed HBase tables and loaded data using Bulk loading.
• Prepared complex Hive queries to perform various ad-hoc reports.
• Extensively used different compression technique (Snappy, Lzo, Gzip, Bzip2) for different requirements to use the cluster wisely and considered the decompression time for critical applications.
• Used Apache Parquet for compression and columnar storage.
• Prepared hourly and daily cycle scripts/jobs and scheduled it in test and production environment using Talend to run hourly/daily/weekly
• Involved in various performance tuning activities, trouble shooting and resolving operational issues.
• Good experience on using Confidential ’s to preprocess Confidential log data.
• Successfully configured the capacity scheduler for resource sharing between different applications/jobs in Hadoop cluster.
• Experienced in Data loading/unloading and responsible to manage data coming from different sources.
• Implemented test scripts to support test driven development and continuous integration.
• Created standards and guide lines for the design and development, tuning, deployment and maintenance of Hadoop projects.
• Provided quick response to ad hoc internal and external client requests for data and experienced in creating ad hoc reports.
• Responsible for data archival process and restore when necessary.
• Imported refined data from Confidential into Tableau for data visualization and report.
• Good Knowledge on alerting mechanism tools like Ganglia and Nagios.
• Experienced in agile software development process and development best practices

Senior Hadoop Engineer

Confidential, Minneapolis, MN

Software: Hadoop MapReduce, Pig, Hive, Sqoop, Talend, Linux, Mongo DB, Hbase, Java, Hue, Ambari, Zoo keeper, Oracle, Tableau, Eclipse, CentOS, Python, JIRA and SVN.

Responsibilities:

• Imported and exported the data using Sqoop from Relational Database systems (MySQL and Oracle) to Confidential and vice-versa.
• The data that are stored on Confidential were preprocessed/validated using Hive then the processed data is stored into Hive warehouse which enabled business analysts and R analysts to get the required data from Hive
• Prepared complex Hive queries and involved in data loading and performed various ad-hoc reports.
• Involved in requirement analysis, strategy development, project planning and implementation.
• Designed and developed multiple MapReduce jobs in Java for complex analysis.
• Converted the Oracle stored procedure into HiveQL.
• Worked on loading and transformation of large datasets of structured and semi structured data into Hadoop ecosystem.
• Created standards and guide lines for the design and development, tuning, deployment and maintenance of Hadoop projects.
• Implemented test scripts to support test driven development and continuous integration.
• Involved in various performance tuning activities.
• Provided quick response to ad hoc internal and external client requests for data and experienced in creating ad hoc reports.
• Imported refined data from Confidential into Tableau for data visualization and report.

Programmer Analyst

Confidential, Charleston, WV

Software: COBOL, Confidential, VSAM, DB2, CICS, SPUFI, QMF, Platinum, File-aid, Mainframe Express, Remedy

Responsibilities:

• Involved in SDLC activities such as Analysis, Design, Coding, Review, Unit testing, Integration testing, Implementation and Support.
• Converted the business requirement into technical design document.
• Prepared unit test cases and integration test cases.
• Actively involved in bug fixes and production support activities.
• Responsible for mass change of reference data and user data.
• Good experience on change/release management activities.
• Worked on POC to move data from mainframes to Hadoop and Import/export the data using Sqoop from Relational Database systems to Confidential .

Module Lead

Confidential, Minneapolis, MN

Software: DB2, COBOL, Confidential, CICS, SPUFI, JAVA, J2EE, Eclipse IDE, JSP, HTML, XML, CSS, JIRA, ORACLE, Hibernate, Web services, Pac2000/Remedy and JDBC

Responsibilities:

• Participated in project planning sessions with business analysts and team members to analyze business IT Requirements and translated business requirements into detailed design.
• Involved in work estimates, program changes, coding new programs and code review.
• Working on integration project with both bi-directional and unidirectional integration between two different systems.
• Extracted business logic from CICS/COBOL online programs and prepared detail design document.
• Involved in developing application using JAVA/J2EE.
• Developed user interfaces using JSP, HTML and CSS.
• Used Hibernate framework for the backend persistence.
• Created Unit Test Plan document with relevant test cases also prepared UAT turnover document and test plan for system integration testing (SIT).
• Co-ordinate with the testing team and the users for UAT/ SIT/ MIT testing.
• Prepared Change Request/ Work Request in Remedy tool to install/deploy the components in production.

Module Lead

Confidential, San Francisco, CA

Software: DB2, COBOL, CICS, Confidential, Java, J2EE, Eclipse IDE, HTML, XML, Java Script, Web services, SPUFI, JDBC, Pac2000/Remedy

Responsibilities:

• Analyzed the business requirement and converted to technical design document.
• Extracted business logic from CICS/COBOL online programs and prepared detail design document.
• Coded several mainframe programs from scratch and maintained the application.
• Involved in bug fixes, project implementation and production support activities.
• Involved in development/conversion of applications using Java, J2EE.
• Prepared Project Estimates/Project plan.
• Developed user interfaces using JSP, HTML and CSS.
• Involved in Test plan creation, coding, unit testing and reviews.
• Maintain the Activity status report, Weekly status report to client, Supplier metrics preparation and Monthly metrics preparation.
• Deployed the application on the Web Sphere Application Server.
• Worked as onsite coordinator for 5 member team.

Mainframe Developer

Confidential

Software: COBOL, Confidential, DB2, VSAM, CICS, Panvalet, MAX, INTERTEST, REMEDY, TSO/ISPF, SAS, MYSQL

Responsibilities:

• Responsible for requirement analysis and detail design preparation.
• Involved and software programming and debugging.
• Investigated, recreated and fixed the problems raised by customers.
• Prepared/Modified procedures and Confidential ’s
• Involved in unit testing and regression testing.
• Responsible for running the hourly batch jobs in test environment.
• Involved in code reviews, test design reviews and test result reviews.
• Prepared unit test plan and regression test plan.
• Responsible for project implementation and support.