SUMMARY:

• 8 + years of experience with 4+ years in Hadoop and Spark Development along with 4 years of experience in Java/J2EE enterprise application design, development, and maintenance.
• Extensive experience implementing Big Data solutions using various distributions of Hadoop and its ecosystem tools.
• Hands - on experience in installing, configuring, and monitoring Hadoop clusters (on premise & cloud AWS).
• Experience in designing and implementing complete end-to-end Hadoop based data analytical solutions using HDFS, MapReduce, Spark, Yarn, Kafka, PIG, HIVE, Sqoop, Storm, Flume, Oozie, Impala, HBase etc.
• Good experience in creating data ingestion pipelines, data transformations, data management, data governance and real time streaming at an enterprise level.
• Profound experience in creating real time data streaming solutions using Apache Spark/Spark Streaming, Kafka.
• Hands on experience using Spark framework with Scala.
• Worked with various formats of files like delimited text files, click stream log files, Apache log files, Avro files, JSON files, XML Files.
• Has good understanding of various compression techniques used in Hadoop processing like Gzip, SNAPPY, LZO etc.
• Good experience on NoSQL: HBase, MongoDB and Cassandra.
• Worked on maintaining Hadoop cluster such as Troubleshooting, Managing and reviewing the performance related configuration fine tuning.
• Good knowledge on AWS data services including Kinesis, Athena, Redshift and EMR.
• Worked in supporting and managing Cloudera’s Hadoop distributions platform.
• Experience developing PigLatin and HiveQL scripts for Data Analysis and ETL purposes and also extended the default functionality by writing User Defined Functions (UDFs), (UDAFs) for custom data specific processing.
• Hands-on experience in using Flume to transfer log data files to Hadoop Distributed File System.
• Mastered in using the using the different columnar file formats like RCFile, ORC and Parquet formats.
• Familiar with installation of Oozie workflow engine to run multiple Hive and Pig jobs that run independently with time and data availability.
• Good experience in optimizing Map-Reduce algorithms by using Combiners and Custom practitioners.
• Strong front end user development skills using HTML, CSS, JavaScript, JQuery, Angular-Js.
• Very good understanding and Working knowledge of Object Oriented Programing concepts.
• Experience includes application development in Java (client/server), JSP, Servlet programming, Enterprise Java Beans, Struts, JSF, JDBC, spring, Spring Integration, Hibernate.
• Very good understanding in AGILE scrum process.
• Experience in using version control tools like Bit-Bucket, SVN etc.
• Good experience in working with cloud environment like Amazon Web Services () EC2 and S3 . 
• Hands on experience on working with Amazon EMR framework transferring data to EC2  server
• Possess working knowledge on SQL, Stored Procedures, Functions, packages, DB Triggers, and Indexes.
• Strong working knowledge in using IDE tools like Eclipse, Intellij.

TECHNICAL SKILLS:

Big Data Ecosystems: Hadoop, Teradata, Map Reduce, Spark, HDFS, HBase, Pig, Hive, Sqoop, Oozie, Storm, Kafka and Flume, AWS.

Spark Streaming Technologies: Spark Streaming, Storm

Scripting Languages: Python, Bash, Java Scripting, HTML5, CSS3

Programming Languages: Java, Scala, SQL, PL/SQL

Databases: RDBMS, NoSQL, Oracle.

Java/J2EE Technologies: Servlets, JSP (EL, JSTL, Custom Tags), JSF, Apache Struts, Junit, Hibernate 3.x, log4J Java Beans, EJB 2.0/3.0, JDBC, RMI, JMS, JNDI.

Tools: Eclipse, Maven, Ant, MS Visual Studio, Net Beans

Methodologies: Agile, Waterfall

PROFESSIONAL EXPERIENCE:

Confidential, White Plains, NY

Hadoop Developer

Responsibilities

• Developed simple to complex Map Reduce jobs using Java language for processing and validating the data.
• Developed data pipeline using Sqoop, Spark, Map Reduce, and Hive to ingest, transform and analyze operational data.
• Developed Map Reduce and Spark jobs to summarize and transform raw data.
• Implemented Spark using Scala and utilizing Data frames and Spark SQL API for faster processing of data.
• Portioned, bucketing and perform joins on hive tables and utilizing hive SerDes like REGEX, JSON and AVRO.
• Used Spark for interactive queries, processing of streaming data and integration with popular NoSQL database for huge volume of data.
• Wrote and tested Python scripts to create new data files for Linux sever configuration using a Python templet tool.
• Real time streaming the data using Spark with Kafka.
• Setup Oozie work flow /sub work flow jobs for Hive/SQOOP/HDFS actions.
• Ingested data from RDBMS and performed data transformations, and then export the transformed data to Cassandra as per the business requirement.
• Handled importing data from different data sources into HDFS using Sqoop and also performing transformations using Hive, Map Reduce and then loading data into HDFS.
• Exported the analyzed data to the relational databases using Sqoop, to further visualize and generate reports for the BI team.
• Collecting and aggregating large amounts of log data using Flume and staging data in HDFS for further analysis
• Analyzed the data by performing Hive queries (Hive QL) and running Pig scripts (Pig Latin) to study customer behavior.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Developed Hive scripts in Hive QL to de-normalize and aggregate the data.
• Created HBase tables and column families to store the user event data.
• Scheduled and executed workflows in Oozie to run Pig jobs.
• Used Impala to read, write and query the Hadoop data in Hive.
• Setup Hadoop cluster on Amazon EC2 for POC .

Environment: Hadoop, HDFS, HBase, Pig, Hive, Scala, MapReduce, Sqoop, Flume, ETL, REST, Java, Python, PL/SQL, Oracle 11g, Unix/Linux, Amazon Ec2 server, Cassandra.

Hadoop Developer

Responsibilities

• Migrating data from Oracle to HDFS and vice-versa using SQOOP.
• Collection and Aggregation of large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
• Worked with different file formats and compression techniques to determine standards.
• Written Hive queries to analyze/transform the data in HDFS.
• Experience in writing Spark applications for Data validation, cleansing, transformations and custom aggregations.
• Designed and implemented Multi-level partitioning, Buckets in Hive.
• Analyzing/Transforming data with Hive and Pig.
• Developed Spark applications for the entire batch processing by using Scala.
• Effective coordination with offshore team and managed project deliverables on time.
• Responsible for creating Hive tables, loading the structured data resulted from MapReduce jobs into the tables and writing Hive queries to further analyze the logs to identify issues and behavioral patterns.
• Involved in running MapReduce jobs for processing millions of records.
• Used Hive to analyze data ingested in to the Hbase by using Hive-Hbase integration and computes various metrics for reporting on the dashboard.
• Involved in requirement and design phase to implement Streaming Lambda Architecture to use real time streaming using Spark and Kafka . 
• Developed job flows to automate the workflow for pig and hive jobs.
• Loaded the aggregated data onto the oracle from Hadoop environment using Sqoop for reporting on the dashboard.
• Worked on QA support activities, test data creation and unit testing activities.
• Experience in installing, configuring, and using Hadoop components like Hadoop MapReduce, YARN, HDFS, HBase, Hive, Pig, Flume, Sqoop and Cassandra.
• Implemented installation and configuration of multi-node cluster on the cloud using Amazon Web Services (AWS) on EC2.
• Used cloud computing on the multi-node cluster and deployed Hadoop application on cloud S3 and used Elastic Map Reduce(EMR) to run a Map-reduce.
• Experience in installing, configuring, and maintaining the Hadoop cluster including YARN configuration using Cloudera, Hortonworks.
• Extracted the data onto HDFS using Flume.
• Created the table definition and made the contents available as a schema backed RDD.

Environment: JDK 1.6, CDH4.x, CDH5.x, Hive, Pig, MapReduce, Scala, Flume, Oracle, Python, Sqoop, Flume, Cassandra, Yarn, Hadoop, AWS, S3, AWS Redshift, Hbase.

Hadoop Developer/Hadoop Admin

Responsibilities

• Responsible for developing efficient MapReduce on AWS cloud programs for more than 20 years’ worth of claim data to detect and separate fraudulent claims.
• Uploaded and processed more than 30 terabytes of data from various structured and unstructured sources into HDFS (AWS cloud) using Sqoop and Flume.
• Played a key-role is setting up a 40 node Hadoop cluster utilizing Apache MapReduce by working closely with the Hadoop Administration team.
• Creating end to end Spark applications using Scala to perform various data cleansing, validation, transformation and summarization activities on user behavioral data.
• Worked with the advanced analytics team to design fraud detection algorithms and then developed MapReduce programs to efficiently run the algorithm on the huge datasets.
• Developed Java programs to perform data scrubbing for unstructured data.
• Involved in converting Cassandra/Hive/SQL queries into Spark transformations using Spark RDD's, and Scala Python .
• Responsible for designing and managing the Sqoop jobs that uploaded the data from Oracle to HDFS and Hive.
• Used Flume to collect the logs data with error messages across the cluster.
• Designed and Maintained Oozie workflows to manage the flow of jobs in the cluster.
• Played a key role in installation and configuration of the various Hadoop ecosystem tools such as, Hive, Pig, and HBase.
• Actively updated the upper management with daily updates on the progress of project that include the classification levels that were achieved on the data.

Environment: Java, Hadoop, Hive, Pig, Sqoop, Flume, HBase, Scala, Oracle 10g, Teradata, Python, Cassandra, AWS, SQL.

Java Developer

Responsibilities

• Developed web application using Struts Framework.
• Developed user interfaces using JSP, HTML, CSS.
• Used Eclipse as an IDE tool to develop the application.
• Worked heavily with the Struts tags - used struts as the front controller to the web application.
• Implemented Struts Framework according to the MVC design pattern.
• Implemented validation framework for creation of validation.xml and used validation-rules.xml.
• Worked on bug fixing and enhancements on change requests.
• Analyzed the requirements of the client and developed detailed design documents.
• Used message driven beans for asynchronous processing alerts to the customer.
• Used Struts Framework to generate forms and actions for validating the user request data.
• Developed server-side validation checks using struts validators and java script validations.
• With JSP’s and Struts custom tags, developed and implemented validations of data.
• Developed applications which access the database with JDBC to execute queries, prepared statements, and procedures.
• Developed programs to manipulate the data and perform CRUD operations on request to the database.
• Worked on developing Use Cases, Class Diagrams, Sequence diagrams, and Data Models.
• Coding of SQL, PL/SQL, and Views using IBM DB2 for the database.
• Working on issues while converting JAVA to AJAX.
• Supported in developing business tier using the stateless session bean.
• Using the GWT to build screens and make remote procedure calls to middleware.
• Using Clear case for source code control and JUNIT testing tool for unit testing.
• Reviewing the code and perform integrated module testing.

Environment: Windows XP, Java/J2ee, Struts, JUNIT, Java, Servlets, JavaScript, SQL, HTML, XML, Eclipse.

Java/J2EE Developer

Responsibilities

• Maintained the UI screens using web technologies like HTML, Java Script, JQuery, and CSS.
• Involved in requirements analysis, design, and development and testing.
• Designed, deployed, and tested Multi-tier application using the Java technologies.
• Involved in front end development using JSP,HTML and CSS.
• Documented the changes for future development projects.
• Responsible to write the different service classes and utility API which will be used across the frame work.
• Used Axis to implementing Web Services for integration of different systems.
• Used MYSQL database to store data and execute SQL queries on the backend.
• Used Apache Ant for the build process.
• Exposed various capabilities as Web Services using SOAP/WSDL.
• Used SOAP UI for testing the Restful Webservices by sending an SOAP request.
• Used Hibernate as Persistence framework mapping the ORM objects to table using Hibernate annotations
• Involved in developing JSP for client data presentation and data validation on the client side with in the forms.
• Used JDBC conntections to store and retrieve the data from the database.
• Developed Web services component using XML, WSDL, and SOAP with DOM parser to transfer and transform data between applications.
• Involved in production support, monitoring server and error logs and Foreseeing the Potential Issues, and escalating to the higher levels.

Environment: Java, J2EE, JSP, Servlets, Spring, Servlets, Custom Tags, Java Beans, JMS, Hibernate, IBM MQ Series, Ajax, JUnit, JNDI, Oracle, XML, SAX, Rational Rose, UML.

Java/J2EE Developer

Responsibilities

• Involved in designing the Project Structure, System Design, and every phase in the project.
• Responsible for developing platform related logic and resource classes, controller classes to access the domain and service classes.
• Involved in Technical Discussions, Design, and Workflow.
• Participate in the Requirement Gathering and Analysis.
• Employed JAXB to un marshal XML into Java Objects.
• Developed Unit Testing cases using JUnit Framework.
• Implemented the data access using Hibernate and wrote the domain classes to generate the Database Tables.
• Involved in implementation of view pages based on XML attributes using normal Java classes.
• Involved in integration of App Builder and UI modules with the platform.

Environment: Hibernate, Java, JAXB, JUnit, XML, UML, Oracle11g, Eclipse, Windows XP.