PROFESSIONAL SUMMARY:

• 8+ years of Professional Experience in IT industry, which includes 4 years of experience with Big Data and Hadoop Eco Systems and strong programming experience using Java, Scala, PHP and SQL.
• Solid hands - on experience with Hadoop ecosystem components like Spark, Hive, Impala, Mapreduce, Pig, HBase, Sqoop, Kafka, Yarn and Oozie.
• Strong fundamental understanding of Distributed Systems Architecture and parallel processing frameworks.
• Strong experience designing and implementing end to end data pipelines running on terabytes of data.
• Used Spark extensively to perform data transformations, data validations and data aggregations.
• Hands on Experience on Data Ingestion tools like Apache Sqoop for importing and exporting data to Relational data base systems (RDBMS) and vice-versa.
• Experience developing Kafka producers and Kafka Consumers for streaming millions of events per second on streaming data.
• Good knowledge and development experience with using Mapreduce framework.
• Proficient in creating Hive DDL's, writing Hive custom UDF’s.
• Experience designing Oozie workflows to schedule and manage data flow.
• Good experience is designing and implementing end to end Data Security and Governance within Hadoop Platform using Kerberos.
• Experience in working with NoSQL database like HBase, Cassandra and Mongo DB.
• Experience in ETL process consisting of data transformation, data sourcing, mapping, conversion and loading.
• Experience working with various Hadoop Distributions like Cloudera, Amazon AWS and Horton works distributions.
• Experience in Apache Spark Core, Spark SQL, Spark Streaming, Spark ML.
• Experience in using different columnar file formats like Avro, ORC and Parquet formats.
• Experience in working with the integration of Hadoop with Amazon s3 Redshift.
• Good experience in Object Oriented Programming, using Java & J2EE (Servlets, JSP, Java Beans, EJB, JDBC, RMI, XML, JMS, Web Services, AJAX).
• Proficiency in frameworks like Struts, Spring, Hibernate.
• Expertise in working with RDBMS databases like Oracle and DB2.
• Experience in Database design, Database analysis, Entity relationships, Programming SQL.
• Strong expertise in creating Shell-Scripts, Regular Expressions and Cron Job Automation.
• Worked with geographically distributed and culturally diverse team, including roles that involve interaction with clients and team members.

TECHNICAL SKILLS:

Big Data Eco System: Hadoop, HDFS, MapReduce, Hive, Pig, Impala, HBase, Sqoop, NoSQL (HBase, Cassandra), Spark, Spark SQL, Spark Streaming, Zookeeper, Oozie, Kafka, Flume, Hue, Cloudera Manager, Amazon AWS, Horton work clusters

Java/J2EE & Web Technologies: J2EE, JMS, JSF, Servlets, HTML, CSS, XML, XHTML, AJAX, Angular JS, JSP, JSTL

Languages: C, C++, Core Java, Shell Scripting, PL/SQL, Python, Pig Latin, Scala

Scripting Languages: JavaScript and UNIX Shell Scripting, Python

Operating system: Windows, MacOS, Linux and Unix

Design: UML, Rational Rose, Microsoft Visio, E-R Modelling

DBMS / RDBMS: Oracle 11g/10g/9i, Microsoft SQL Server 2012/2008, MySQL, DB2 and NoSQL, Teradata SQL, RDBMS, MongoDB, Cassandra, HBase

IDE and Build Tools: Eclipse, NetBeans, Microsoft Visual Studio, Ant, Maven, JIRA, Confluence

Version Control: SVN, CVS, GITHUB

Security: Kerberos

Web Services: SOAP, RESTful, JAX-WS

Web Servers: Web Logic, Web Sphere, Apache Tomcat, Jetty

PROFESSIONAL SUMMARY:

Confidential, Richfield, MN

Sr. Hadoop Developer

Responsibilities:

• Developed new platform using Hadoop for performing user behavioral analytics.
• Ingested customer profile information from data warehouse into HDFS using Sqoop
• Developed custom connectors for pulling marketing and campaign data feeds from FTP
• servers into HDFS.
• Performed Data Ingestion from multiple internal clients exposed as Rest calls using Apache Kafka.
• Created Kafka producers for streaming real time click stream events from adobe Rest services into our topics.
• Developed Spark code using Scala and Spark-SQL/Streaming for faster processing of data. 
• Analyzed the data using Spark Data Frames and series of Hive Scripts to produce summarized results from Hadoop to downstream systems.
• Used spark SQL to load the metrics from the summarized results to hive tables in parquet format.
• Worked with Spark Data Frames, Spark SQL and Spark Mlib extensively.
• Worked with a team to improve the performance and optimization of the existing algorithms in Hadoop using Spark, Spark -SQL, Data Frame.
• Implemented business logic in Hive and written UDF’s to process the data for analysis.
• Implemented security on Hadoop Cluster using with Kerberos by working with operations team to move from a non-secured cluster to secured cluster.
• Created Hive external tables on top of the HDFS data.
• Used Cloudera Manager to manage and monitor Hadoop Stack.
• Used Oozie to define a workflow to coordinate the execution of Spark, Hive and Sqoop jobs.
• Used Impala to perform interactive querying.
• Developed interactive Dashboards using tableau connecting to Impala.
• Worked with Data Science team in developing Spark Mlib applications to develop various predictive models.
• Used Jira as an issue tracking tool for design and documentation of run time problems and procedures.
• Expertise on interacting with the project team to organize timelines, responsibilities and deliverables to provide all aspects of technical support.
• Coordinated effectively with offshore team and managed project deliverable on time.

Environment: Hadoop 2.x, Spark, Scala, Hive, Sqoop, Oozie, Kafka, Cloudera Manager, Storm, ZooKeeper, HBase, Impala, YARN, Cassandra, JIRA, Kerberos, Shell Scripting, SBT, GITHUB, Maven.

Confidential, Peoria, AZ

Hadoop Developer

Responsibilities:

• Involved in requirement analysis, design, coding and implementation phases of the project.
• Used Sqoop to load structured data from relational databases into HDFS.
• Loaded transactional data from Teradata using Sqoop and created Hive Tables.
• Worked on automation of delta feeds from Teradata using Sqoop and from FTP Servers to Hive.
• Performed Transformations like De-normalizing, Cleansing of data sets, Date Transformations, parsing some complex columns.
• Worked with different compression codecs like GZIP, SNAPPY and BZIP2 in MapReduce, Pig and Hive for better performance.
• Handled Avro, JSON and Apache Log data in Hive using custom Hive SerDes.
• Worked on batch processing and scheduled workflows using Oozie.
• Implemented installation and configuration of multi-node cluster on the cloud using Amazon Web Services (AWS) on EC2.
• Used cloud computing on the multi-node cluster and deployed Hadoop application on cloud S3 and used Elastic Map Reduce (EMR) to run Map-reduce.
• Used Hive-QL to create partitioned RC, ORC tables, used compression techniques to optimize data process and faster retrieval. 
• Implemented Partitioning, Dynamic Partitioning and Buckets in Hive for efficient data access.
• Written Pig UDF’s to cleanse the incoming huge data.

Environment: HDFS, Hadoop, Pig, Hive, HBase, Sqoop, Teradata, Flume, Map Reduce, Oozie, Java 6/7, Oracle 10g, YARN, UNIX Shell Scripting, Maven, Agile Methodology, JIRA, Linux.

Confidential, Fairfax, VA

Hadoop Developer

Responsibilities:

• Developed complex MapReduce jobs in Java to perform data extraction, aggregation and transformation
• Load the data into HDFS from different Data sources like Oracle, DB2 using Sqoop and load into Hive tables.
• Analyzed big data sets by running Hive queries and Pig scripts.
• Integrated the hive warehouse with HBase for information sharing among teams.
• Developed the Sqoop scripts for the interaction between Pig and MySQL Database.
• Worked on Static and Dynamic partitioning and Bucketing in Hive.
• Scripted complex Hive QL queries on Hive tables for analytical functions.
• Developed complex Hive UDFs to work with sequence files.
• Designed and developed Pig Latin scripts and Pig command line transformations for data joins and custom processing of Map Reduce outputs.
• Created dashboards in Tableau to create meaningful metrics for decision making.
• Performed rule checks on multiple file formats like XML, JSON, CSV and compressed file formats.
• Monitored System health and logs and respond accordingly to any warning or failure conditions.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required
• Used storage format like AVRO to access multiple columnar data quickly in complex queries.
• Implemented Counters for diagnosing problem in queries and for quality control and application-level statistics.
• Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms
• Implemented  Log4j  to trace logs and to track information.
• Developed some helper class for abstracting Cassandra cluster connection act as core toolkit.
• Installed Oozie workflow engine and scheduled it to run data/time dependent Hive and Pig jobs
• Involved in Agile methodologies, daily Scrum meetings, Sprint planning.

Environment: HDFS, MapReduce, Cassandra, Hive, Pig, Sqoop, Tableau, NoSQL, Shell Scripting, Maven, Git, HDP Distribution, Eclipse, Log4j, JUnit, Linux.

Confidential 

Java Developer

Responsibilities:

• Involved in designing use-case diagrams, class diagram, interaction using UML model with Rational Rose. 
• Developed design patterns using MVC 2 Web Framework .
• Implemented views using Struts tags, JSTL and Expression Language .
• Used Spring for dependency injection plugging in the Hibernate DAO objects for the business layer. 
• Created Spring Interceptors to validate web service requests and enables notifications. 
• Integrated Hibernate ORM framework with Spring framework for data persistence and transaction management.
• Designed REST APIs that allows sophisticated, effective and low-cost application Integration.
• Worked with java core   concepts like JVM internals, multithreading, garbage collection .
• Implemented Java   Message Services (JMS) using JMS API. 
• Adopted J2EE design patterns like Singleton, Service Locator and Business Facade
• Developed POJO classes and used annotations to map with database tables
• Used the features of Spring Core layer (IOC), Spring MVC, Spring AOP, Spring ORM layer and Spring DAO support layer to develop the application.
• Involved in the configuration of Struts Framework, Spring Framework and Hibernate mapping tool
• Used Jasper Reports for designing multiple reports.
• Implemented web service client program to access Affiliates web service using SOAP/REST Web Services.
• Involved in production support, resolving the production issues and maintaining the application server.
• Utilized Agile Methodology/Scrum (SDLC) to manage projects and team.
• Unit tested all the classes using JUNIT at various class level and methods level.
• Worked with all the test cases with testing team and created test cases with use cases.

Environment: J2EE, Hibernate 3.0, JSF, Rational Rose, Spring1.2, JSP 2.0, Servlet 2.3, XML, JDBC, JNDI, JUnit, IBM WAS 6.0, RAD 7.0, Oracle 9i, PLSQL, Log4j, Linux, RESTful Webservices.

Confidential

Java Developer

Responsibilities:

• Involved in Requirement Analysis, Design, Development and Testing of the risk workflow system.
• Involved in the implementation of the Software development life cycle (SDLC) that includes Development, Testing, Implementation and Maintenance Support.
• Developed front-end screens using Struts, JSP, HTML, AJAX, jQuery, JavaScript, JSON and CSS.
• Implemented XSLT’s for transformations of the xml’s in the spring web flow.
• Developed POJO based programming model using spring framework.
• Used IOC (Inversion of Control) Pattern and Dependency Injection of Spring framework for wiring and managing business objects.
• Used Hibernate framework for Entity Relational Mapping.
• Used Web Services to connect to mainframe for the validation of the data.
• Created and maintained the configuration of Spring Application Framework(IOC) and implemented business logic using EJB3.
• Developed Web Services utilizing HTTP, XML, XSL and SOAP.
• SOAP has been used as a protocol to send request and response in the form of XML messages.
• WSDL has been used to expose the Web Services.
• Developed stored procedures, Triggers and functions to process the data using PL/SQL and mapped it to Hibernate Configuration File and established data integrity among all tables.
• Involved in the up gradation of WebSphere and SQL Servers.
• Participated in Code Reviews of other modules, documents, test cases.
• Performed unit testing using JUnit and performance and volume testing.

Environment: Java1.5/J2EE, JDK, JSP, HTML, CSS, Struts, EJB, JMS, Spring, Hibernate, Eclipse, WebSphere Application Server, Web services (SOAP, REST), JavaScript, PL/SQL, CVS, RAD and Oracle10g .