PROFESSIONAL SUMMARY:

• Around 8 years of professional experience in IT, including 4+ years of work experience in Big Data, Hadoop Development and Ecosystem Analytics in Insurance,
• Health Care & Retail Industry Project sectors wif multiprogramming language expertise like Java, Python and Scala.
• Hadoop Developer wif 4 years of working experience in designing and implementing complete end - to-end Hadoop Infrastructure using MapReduce, Spark, Kafka, PIG, HIVE, Sqoop, Oozie, Flume.
• Java Programmer wif 4+ years of Extensive programming experience in developing web based applications and Client-Server technologies.
• Expert Hands-on in Installing, Configuring, Testing Hadoop Ecosystem components.
• Good knowledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce concepts
• Experience in working wif MapReduce programs using Hadoop for working wif Big Data.
• Experience in analyzing data using Hive QL, Pig Latin and custom MapReduce programs in Java
• Experience in importing and exporting data using Sqoop from Relational Database Systems to HDFS and vice-versa.
• Collecting and aggregating a large amount of Log Data using Apache Flume and storing data in HDFS for further analysis.
• Built a dashboard to show the statistics on transaction status using Datameer and Platfora.
• Job/workflow scheduling and monitoring tools like Oozie.
• Experience in designing both time driven and data driven automated workflows using Oozie
• Worked in complete Software Development Life Cycle (analysis, design, development, testing, implementation and support) using Agile Methodologies.
• Transforming some existing programs into lambda architecture.
• Experience in setting up monitoring infrastructure for Hadoop cluster using Nagios and Ganglia.
• Experience on Hadoop clusters using major Hadoop Distributions - Cloudera(CDH4, CDH5), Hortonworks(HDP)
• Experience in different layers of Hadoop Framework - Storage (HDFS), Analysis (Pig and Hive), Engineering (Jobs and Workflows).
• Experienced in using Integrated Development environments like Eclipse, NetBeans, Kate and gEdit.
• Migration from different databases (i.e. Oracle, DB2, Cassandra, MongoDB) to Hadoop.
• Worked and migrated RDMBS databases into different NoSQL database.
• Experience in designing and coding web applications using Core Java & web Technologies- JSP, Servlets and JDBC.
• Excellent knowledge in Java and SQL in application development and deployment.
• Familiar wif data warehousing "fact" and "dim" table and star schema and combined wif Google Fusion tables for visualization.
• Good working experience in PySpark and Spark Sql.
• Familiar wif Scala, closures, higher order functions, monads.
• Hands on experience in creating various database objects like tables, views, functions, and triggers using SQL.
• Excellent technical, communication, analytical and problem-solving skills and ability to get on well wif people including cross-cultural backgrounds and troubleshooting capabilities.

TECHNICAL SKILLS:

Big Data Ecosystems: Hadoop, MapReduce, HDFS, HBase, Hive, Pig, Sqoop, Spark, Storm, Kafka, Oozie, MongoDB, Cassandra

Languages: C, Core Java, Unix, SQL, Python, R, C#, Haskell, Scala

J2EE Technologies: Servlets, JSP, JDBC, Java Beans.

Methodologies: Agile, UML, Design Patterns (Core Java and J2EE).

Monitoring and Reporting: Ganglia, Nagios, Custom Shell scripts.

NoSQL Technologies: Cassandra, MongoDB, Neo4j, HBase

Frameworks: MVC, Struts, Hibernate, Spring.

Database: Oracle 11g, MySQL, MS-SQL Server, Teradata. PostgreSQL, IBM DB2

Operating Systems: Windows XP/Vista/7, UNIX.

Software Package: MS Office 2010.

Tools: & Utilities: Eclipse, Net Beans, My Eclipse, SVN, Git, Maven, SOAP UI, JMX explorer, XML Spy, QC, QTP, Jira

Web Servers: WebLogic, WebSphere, Apache Tomcat.

Web Technologies: HTML, XML, JavaScript, jQuery, AJAX, SOAP, and WSDL.

PROFESSIONAL EXPERIENCE:

Confidential

Senior Hadoop Developer

Responsibilities:

• Devised and lead the implementation of the next generation architecture for more efficient data ingestion and processing.
• Extensive experience wif Hadoop and HBase, including multiple public presentations about these technologies.
• Created Hive queries that halped market analysts spot emerging trends by comparing fresh data wif EDW tables and historical metrics.
• Enabled speedy reviews and first-mover advantages by using Oozie to automate data loading into the Hadoop Distributed File System and PIG to pre-process the data.
• Provided design recommendations and thought leadership to sponsors/stakeholders that improved review processes and resolved technical problems and suggested some solution translation via lambda architecture.
• Used Splunk for HadoopOps for Managing, Monitoring and reviewing the whole infrastructure lives operations & activity. Also Managed MapReduce job to rapidly sort, filter and report on performance metrics, time, status, user or resource usage.
• Real time streaming the data using Spark wif Kafka.
• Implemented Spark using Python (pySpark) and Spark SQL for faster testing and processing of data.
• Worked on migrating MapReduce programs into Spark transformations using Spark and Scala.
• Identify concurrent job workloads that may impact or be impacted by failures or bottlenecks.
• Created a definitive record of user activity across the cluster and wif role-based access to Splunk searches same.
• Developed some utility halper classes to get data from HBase tables.
• Professional experience wif NoSQL Hbase solutions to solve real world scaling problems.
• Attending daily status calls to follow scrum process to complete each user story wifin the timeline.
• Also be the part of triage call to handle defect reported by tester team or QA team.
• Coordinating wif EM to resolve any configuration related issues.
• Implemented Cluster for NoSQL tools Cassandra, MongoDB as a part of POC to address HBase limitations.
• Worked on Implementation of a toolkit that abstracted Solr & ElasticSearch.
• Worked on Spark wif Python and Scala.
• Data is loaded back to the Teradata for the BASEL reporting and for the business users to analyze and visualize the data using Datameer
• Viewing various aspect of a cluster using Cloudera Manager.

Confidential, Hartford, CT

Hadoop Developer

Responsibilities:

• Loading the data from the different Data sources like (Teradata, DB2, Oracle and flat files) into HDFS using Sqoop and load into Hive tables, which are partitioned.
• Created different pig scripts & converted them as a shell command to provide aliases for common operation for project business flow.
• Implemented various Hive queries for Analysis and call then from java client engine to run on different nodes.
• Created few Hive UDF's to as well to hide or abstract complex repetitive rules.
• Developed Oozie Workflows for daily incremental loads, which gets data from Teradata and then imported into hive tables.
• Involved in End to End implementation of ETL logic.
• Reviewing ETL application use cases before on boarding to Hadoop.
• Developed bash scripts to bring the log files from FTP server and then processing it to load into Hive tables.
• All the bash scripts are scheduled using Resource Manager Scheduler.
• Moved data from HDFS to Cassandra using Map Reduce and BulkOutputFormat class.
• Developed Map Reduce programs for applying business rules to the data.
• Did Implementation using Apache Kafka replacement for a more traditional message broker (JMS Solace) to reduce licensing and decouple processing from data producers, to buffer unprocessed messages.
• Implemented receiver based approach, here I worked on Spark streaming for linking wif StreamingContext using java API and handle proper closing & waiting for stages as well.
• Maintaining Authentication module to support Kerberos.
• Experience in Implementing Rack Topology scripts to the Hadoop Cluster.
• Implemented the part to resolve issues related wif old Hazelcast API EntryProcessor.
• Participated wif the admin team in designing and upgrading CDH 3 to HDP 4.
• Developed Some Helper class for abstracting Cassandra cluster connection act as core toolkit.
• Enhanced existing module written in python scripts.
• Used dashboard tools like Tableau.

Confidential, San Francisco, CA

Hadoop Developer

Responsibilities:

• Understand the exact requirement of a report from the Business groups and users.
• Frequent interactions wif Business partners.
• Designed and developed a Medicare-Medicaid claims system using Model-driven architecture on a customized framework built on Spring.
• Moved data from HDFS to Cassandra using Map Reduce and BulkOutputFormat class.
• Imported trading and derivatives data in Hadoop Distributed File System and Eco System (MapReduce, Pig, Hive, Sqoop).
• Was part of an activity to setup Hadoop ecosystem at dev & QA Environment.
• Managed and reviewed Hadoop Log files.
• Responsible writing PIG Script and Hive queries for data processing
• Running Sqoop for importing data from Oracle & Other Database.
• Creation of shell script to collect raw logs from different machines.
• Created Partition in a hive as static and dynamic.
• Implemented Pig Latin scripts using operators such as LOAD, STORE, DUMP, FILTER, DISTINCT, FOREACH, GENERATE, GROUP, COGROUP, ORDER, LIMIT, AND UNION.
• Defined some PIG UDF for some financial functions such as swap, hedging, Speculation and arbitrage
• Coded many MapReduce program to process unstructured logs file.
• Worked on Import and export data into HDFS and Hive using Sqoop
• Used parameterize pig script and optimized script using illustrate and explain.
• Involved in the process of configuring HA, Kerberos security issues and name node failure restoration activity time to time as a part of zero downtime.
• Implemented FAIR Scheduler as well.

Confidential - Boston, MA

Java/J2EE Developer

Responsibilities:

• Write design document based on requirements from MMSEA user guide.
• Performed requirement gathering, design, coding, testing, implementation and deployment.
• Worked on modeling of Dialog process, Business Processes and coding Business Objects, QueryMapper and
• JUnit files.
• Created the Business Objects methods using Java and integrating the activity diagrams.
• Worked in web services using SOAP, WSDL.
• Wrote Query Mappers and MQ Experience in JUnit Test Cases.
• Developed the UI using XSL and JavaScript.
• Managed software configuration using ClearCase and SVN.
• Design, develop and test features and enhancements.
• Perform error rate analysis of production issues and technical errors. Provide production support. Fix production defects.
• Analyze user requirement document and develop test plan, which includes test objectives, test strategies, test environment, and test priorities.
• Perform Functional testing, Performance testing, Integration testing, Regression testing, Smoke testing and User Acceptance Testing (UAT).
• Converted Complex SQL queries running at mainframes into pig and Hive as a part of a migration from mainframes into Hadoop cluster.

Confidential

Java Developer

Responsibilities:

• Involved in various SDLC phases like Design, Development and Testing.
• Developed front end using Struts and JSP. Developed web pages using HTML,JavaScript, JQuery and CSS.
• Used various CoreJavaconcepts such as Exception Handling, Collection APIs to implement various features and enhancements.
• Developed server side components servlets for the application.
• Involved in coding, maintaining, and administering Servlets and JSP components to be deployed on a Web Sphere application server.
• Implemented Hibernate ORM to Map relational data directly tojavaobjects
• Worked wif Complex SQL queries, Functions and Stored Procedures.
• Involved in developing spring web MVC framework for portals application.
• Implemented the logging mechanism using log4j framework.
• Developed REST API, Web Services.
• Wrote test cases in JUnit for unit testing of classes.
• Used Maven to build the J2EE application.
• Used SVN to track and maintain the different version of the application.
• Involved in maintenance of different applications wif onshore team.
• Good working experience in Tepestry processing claims.
• Working experience wif professional billing claims.