SUMMARY:

• 8+ years of experience in IT in fields of software design, implementation, and development and also support of business applications for health, insurance and telecom industries.
• 4+Years of experience in Linux and Big data Hadoop, Hadoop Ecosystem components like MapReduce, Sqoop, Flume, Kafka, Pig, Hive, Spark, Storm, HBase, Oozie, and Zookeeper
• Having good experience in Hadoop framework and related technologies like HDFS , MapReduce , Pig, Hive, HBase, Sqoop and Oozie
• Hands of experience on data extraction, transformation and load in Hive, Pig and HBase
• Experience in the successful implementation of ETL solution between an OLTP and OLAP database in support of Decision Support Systems with expertise in all phases of SDLC .
• Experience in creating DStreams from sources like Flume , Kafka and performed different Spark transformations and actions on it.
• Experience in integrating Apache Kafka with Apache Storm and created Storm data pipelines for real time processing.
• BI tools (Spotfire, Crsytal Reports, Lumira, Tableau) integration with Hadoop
• Worked on improving the performance and optimization of the existing algorithms in Hadoop using Spark context, Spark - SQL, Data Frames, RDD's, Spark YARN.
• Delivery experience on major Hadoop ecosystem Components such as Pig, Hive, Spark, Kafka, Elastic Search & HBase and monitoring with Cloudera Manager . Extensive working experience using Sqoop to import data into HDFS from RDBMS and vice-versa.
• Procedural knowledge in cleansing and analyzing data using HiveQL , Pig Latin and custom MapReduce programs in Java .
• Experience in transferring the data using Informatica tool from AWS S3 to AWS Redshift
• Experienced data pipelines using Kafka and Akka for handling large terabytes of data.
• Hands on experience on Solr to Index the files directly from HDFS for both Structured and Semi Structured data.
• Strong experience in RDBMS technologies like MySQL , Oracle , Postgres and DB2 .
• Training and Knowledge in Mahout , Spark MLlib for use in data classification, regression analysis, recommendation engines and anomaly detection.
• Experienced in Developing Spark application using Spark Core, Spark SQL and Spark Streaming API's.
• Experience on predictive intelligence and smooth maintenance in spark streaming is done using Conviva and MLlib from Spark.
• Experience in configuring deployment environment to handle the application using Jetty server and Web Logic 10 and Postgres database at the back-end.
• Involved in installing cloudera distribution of Hadoop on amazonEC2 Instances .
• Expertise on SPARK engine creating batch jobs with incremental load through HDFS/S3, KINESIS, Sockets, AWS etc.,
• Hands on experience in using BI tools like Splunk/Hunk, Tableau .
• Worked on Amazon AWS concepts like EMR and EC2 web services for fast and efficient processing of Big Data .
• Experience in deployment of Big Data solutions and the underlying infrastructure of Hadoop Cluster using Cloudera, MapR and Hortonworks distributions.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa
• Analyzed the SQL scripts and designed the solution to implement using PYSPARK.
• Experience of MPP databases such as HP Vertica and Impala.
• Hands on experience in the SVN and GitHub.
• Experience in extensive usage of Struts, HTML, CSS, JSP, JQuery, AJAX and JavaScript for interactive pages.
• Involved in configuring and working with Flume to load the data from multiple sources directly into HDFS .
• Hand on experience and knowledge in Lumira , Regex , Sed, Maven, Log4j, Junit and Ant .
• Worked with the Apache Nifi flow to perform the conversion of Raw XML data into JSON , AVRO .
• Experience in understanding security requirements for Hadoop and integrate with Kerberos authentication and authorization infrastructure.
• Involved in converting Hive / SQL queries into Spark transformations using Spark RDDs , Python and Scala .
• Hands on experience on implementation projects like Agile and Waterfall methodologies .
• Experienced in integration of Hadoop cluster with Spark engine to perform BATCH and GRAPHX operations.
• Performed analytics in Hive using various files format like JSON , Avro , ORC , and Parquet.
• Working knowledge of database such as Oracle 8i/9i/10g, Microsoft SQL Server , DB2 , Netezza .
• Experience in NoSQL databases like HBase , Cassandra , Redis and MongoDB .
• Experience and hands-on knowledge in Akka and LIFT Framework .
• Experience in Object Oriented Analysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.

TECHNICAL SKILLS:

Big Data Ecosystem: Hadoop, MapReduce, Pig, Hive, YARN, Kafka, Flume, Sqoop, Impala, Oozie, Zookeeper, Spark, Ambari, Mahout, MongoDB, Cassandra, Avro, Storm, Parquet and Snappy.Hadoop Distributions Cloudera (CDH3, CDH4, and CDH5), Hortonworks, MapR and Apache

Languages: Java, Python, Jruby, SQL, HTML, DHTML, Scala, JavaScript, XML and C/C++

No SQL Databases: Cassandra, MongoDB and HBase

Java Technologies: Servlets, JavaBeans, JSP, JDBC, JNDI, EJB and struts

XML Technologies: XML, XSD, DTD, JAXP (SAX, DOM), JAXB

Development Methodology: Agile, waterfall

Web Design Tools: HTML, DHTML, AJAX, JavaScript, JQuery and CSS, AngularJs, ExtJS and JSON

Development / Build Tools: Eclipse, Ant, Maven, IntelliJ, JUNIT and log4J

Frameworks: Struts, spring and Hibernate

App/Web servers: WebSphere, WebLogic, JBoss and Tomcat

DB Languages: MySQL, PL/SQL, PostgreSQL and Oracle

RDBMS: Teradata, Oracle 9i,10g,11i, MS SQL Server, MySQL and DB2

Operating systems: UNIX, LINUX, Mac os and Windows Variants

Data analytical tools: R and MATLAB

ETL Tools: Talend, Informatica, Pentaho

PROFESSIONAL EXPERIENCE:

Sr. Hadoop/Spark Developer

Confidential, Oklahoma

Responsibilities:

• Experienced in designing and deployment of Hadoop cluster and various Big Data components including HDFS, MapReduce, Hive, Sqoop, Pig, Oozie, Zookeeper in Cloudera distribution
• Involved in loading and transforming large Datasets from relational databases into HDFS and vice-versa using Sqoop imports and export.
• Responsible for loading Data pipelines from webservers and Teradata using Sqoop with Kafka and Spark Streaming API .
• Hands-on experience with Amazon EC2 , Amazon S3 for the computing and storage of data.
• Excellent Understanding of from Amazon Web Services (AWS) services like EC2, S3, EBS, RDS and VPC.
• Data validation between Hive target tables and Redshift source tables using automation scripts.
• Designed and published visually rich and intuitive Tableau dashboards and Crystal Reports for executive decision making.
• Involvement in creating custom UDFs for Pig and Hive to consolidate strategies and usefulness of Python into Pig Latin and HQL ( HiveQL ).
• Extensively worked on Text , ORC , Avro and Parquet file formats and compression techniques like Snappy , Gzip and Zlib .
• Wrote ETL jobs to read from web APIs using REST and HTTP calls and loaded into HDFS using java and Talend.
• Worked on Apache Nifi as ETL tool for batch processing and real time processing.
• Experienced on creating multiple kind of Report in Power BI and present it using Story Points.
• Very Good understanding and Working Knowledge of Object Oriented Programming(OOPS), Python and Scala.
• Handled importing of data from various data sources, performed transformations using Hive MapReduce , loaded data into HDFS and extracted data from MYSQL into HDFS vice-versa using Sqoop .
• Sed Hive Queries in Spark - SQL for analysis and processing the data. Used Scala programming to perform transformations and applying business logic.
• Worked on Sequence files, RC files, Map side joins, bucketing , partitioning for Hive performance enhancement and storage improvement and utilizing Hive SerDes like REGEX, JSON and AVRO .
• Implemented advanced procedures like text analytics and processing using the in-memory computing capabilities like Apache Spark written in Scala.
• Developing Scripts and Batch Job to schedule various Hadoop Program.
• Hands on experience in using Map reduce programming model for Batch processing of data stored in HDFS
• Good knowledge in using Splunk for data monitoring and visualization.
• Ingested data from RDBMS and performed data transformations, and then export the transformed data to Cassandra as per the business requirement and also used Cassandra through Java services.
• Experience in NoSQL Column-Oriented Databases like Cassandra and its Integration with Hadoopcluster
• Built re-usable Hive UDF libraries which enabled various business analysts to use these UDF's in Hive querying .
• Developed Solr web apps to query and visualize and solr indexed data from HDFS .
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop . Cluster co-ordination through Zookeeper .
• Used a LAMBDA EXPRESSION to improve SackEmployees further and avoid the need for a separate class.
• Written shell scripts that run multiple Hive jobs which helps to automate different hive tables incrementally which are used to generate different reports using Tableau for the Business use.
• Experienced in Apache Spark for implementing advanced procedures like text analytics and processing using the in-memory computing capabilities written in Scala.
• Used OOZIE Operational Services for batch processing and scheduling workflows dynamically.
• Worked totally in agile methodology and also developed Spark scripts by using Scala shell.

Environment: Hadoop, Hive, Map Reduce, Sqoop, Kafka, Spark, Yarn, Pig, Cassandra, Oozie, shell Scripting, Scala, Maven, Java, JUnit, agile methodologies, MySQL, Tableau, AWS, EC2, S3, cloudera, power BI, Solr.

Hadoop Developer

Confidential, Lawrenceville

Responsibilities:

• Responsible for planning, organizing, and implementation of complex business solutions, producing deliverables within stipulated time.
• Worked with Linux systems and RDBMS database on a regular basis in order to ingest data using Sqoop.
• Experience in Cloud based services(AWS) to retrieve the data.
• Using Kafka and Kafka brokers we initiated spark context and processed live streaming information with the help of RDD as is.
• Experience in supporting multi-region AWS cloud and Created placement groups to maintain cluster of instances.
• Installed, Configured Talend ETL on single and multi-server environments.
• Experience in creating tables, dropping and altered at run time without blocking updates and queries using HBase and Hive .
• Developed Pyspark code to read data from Hive , group the fields and generate XML files Enhanced the Pyspark code to write the generated XML files to a directory to zip them to CDAs
• Developed ETL test scripts based on technical specifications/Data design documents and Source to Target mappings.
• Used Spark API over Hortonwork Hadoop YARN to perform analytics on data in Hive.
• Hands-on experience with Hortonwork tools like Tez and Ambari .
• Extracted files from MongoDB through Sqoop and placed in HDFS and processed.
• Worked on Spark using Python and Spark SQL for faster testing and processing of data.
• Involved in enabling Amazon Kinesis firehose to capture streaming data directly on to S3 and also Red Shift . It automatically scales to match the throughput of your data and requires no ongoing administration.
• Developed and maintained the continuous integration and deployment systems using Jenkins , ANT , Akka and MAVEN .
• Used Akka as a framework to create reactive, distributed, parallel and resilient concurrent applications in Scala .
• Involved in building a scalable email system using amazon simple email service , s3 and Akka actors for handling heavy loads of emails.
• Writing user console page in lift along with the snippets in Scala . The product is responsible to give access to the user to all their credentials and privileges within the system
• Used Oozie workflow engine to create the workflows and automate the MapReduce , Hive , Pig jobs.
• Developed Unix shell scripts to load large number of files into HDFS from Linux File System.
• Using HIVE join queries to join multiple tables of a source system and load them into Elastic Search Tables.

Environment: Hadoop ecosystem components, ETL, Spark, Kafka, Python, Shell Scripting, SQL Talend, Elastic search, Linux- Ubuntu, AWS, Hortonworks, MongoDB, VPC, Lambda, Hive, Zookeeper, Pig, Sqoop, Oozie, Tez, Ambari, YARN, Akka, Jenkins, Kinesis, ANT, Map Reduce.

Hadoop Developer

Confidential, Austin

Responsibilities:

• Hands on experience in loading data from UNIX file system and Teradata to HDFS
• Installed and configured Flume , Hive , Pig , Sqoop and Oozie on the Hadoop cluster .
• Developed PIG scripts for the analysis of semi structured data.
• Developed Java MapReduce programs on log data to transform into structured way to find user location, age group, spending time.
• Collected and aggregated large amounts of web log data from different sources such as webservers , mobile and network devices using Apache Flume and stored the data into HDFS for analysis
• Create a complete processing engine, based on Cloudera's distribution, enhanced to performance.
• Working with Eclipse using Maven plugin for Eclipse IDE .
• Created HBase tables to store variable data formats coming from different portfolios Performed real time analytics on HBase using Java API and RestAPI.
• Extracted files from Couch DB, MongoDB through Sqoop and placed in HDFS for processed
• Integrated Oozie with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as Map - Reduce , Pig , Hive , and Sqoop ) as well as system specific jobs (such as Java programs and shell scripts ).
• Developed ETL using Hive , Oozie , shell scripts and Sqoop . Used Scala for coding the components, & Utilized Scala pattern matching in coding
• Supported Data Analysts in running Map Reduce Programs.
• Implemented Name Node backup using NFS. This was done for High availability
• Analyzed the weblog data using the HiveQL, integrated Oozie with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as MapReduce, Pig, Hive, and Sqoop) as well as system specific jobs (such as Java programs and shell scripts).
• Experience in Pig scripts for sorting, joining and grouping the data
• Experienced with working on Avro Data files using Avro Serialization system.

Environment: HDFS, Map Reduce, Pig, Hive, Sqoop, Flume, HBase, Java, Maven, Avro, Cloudera, Eclipse and Unix Shell Scripting, Oozie, ETL, Scala.

Java Developer

Confidential

Responsibilities:

• Developed Use Case diagrams, business flow diagrams, Activity / State diagrams.
• Designed and implemented the training and reports modules of the application using Servlets , JSP and ajax
• Developed XML Web Services using SOAP , WSDL , and UDDI .
• Interact with Business Users and Develop Custom Reports based on the criteria defined.
• Requirement gathering and information collection. Analysis of gathered information so as to prepare a detail work plan and task breakdown structure
• Experience in develop of SDLC life cycle and undergo in all the phases in it.
• Implemented applications using Java, J2EE , JSP , Servlets , JDBC , RAD , XML , HTML , XHTML , Hibernate Struts , spring and JavaScript on Windows environments.
• Developed action Servlets and JSPs for presentation in Struts MVC framework.
• Developed PL / SQL View function in Oracle 9i database for get available date module.
• Used Oracle SQL 4.0 as the database and wrote SQL queries in the DAO Layer.
• Used RESTFUL Services to interact with the Client by providing the RESTFUL URL mapping.
• Implementing project using Agile SCRUM methodology, involved in daily stand up meetings and sprint showcase and sprint retrospective.
• Used SVN and GitHub as version control tool.
• Experienced in developing web-based applications using Python, Django , PHP , XML , CSS , HTML , Java Script and JQuery .
• Developed presentation layer using HTML , JSP , Ajax , CSS and JQuery .
• Worked with Struts MVC objects like Action Servlet, Controllers, validators , Web Application Context, Handler Mapping , Message Resource Bundles, Form Controller, and JNDI for look-up for J2EE components.
• Experience in JIRA and tracked the test results and interacted with the developers to resolve issue.
• Created the UI tool - using Java, XML, XSLT, DHTML and JavaScript
• Used XSLT to transform my XML data structure into HTML pages.
• Deployed EJB Components on Tomcat . Used JDBC API for interaction with Oracle DB .
• Wrote build & deployment scripts using shell , Perl and ANT scripts
• Extensively used Java multi-threading to implement batch Jobs with JDK 1.5 features
• Experience in application using Core Java, JDBC, JSP, Servlets, spring, Hibernate, Web Services, SOAP, and WSD
• Implemented Hibernate in the data access object layer to access and update information in the Oracle10gDatabase

Environment: HTML, Java Script, Ajax, Servlets, JSP, SOAP, SDLC life cycle, Java, Hibernate, Scrum, JIRA, Git Hub, JQuery, CSS, XML, ANT, Tomcat Server, Jasper Reports.

Java Developer

Confidential

Responsibilities:

• Involved in the design, development and deployment of the Application using Java /J2EE Technologies.
• Developed web components using JSP Servlets , JDBC and Coded JavaScript for AJAX and client side data validation.
• Designed and Developed mappings using different transformations like Source Qualifier , Expression, Lookup (Connected & Unconnected), Aggregator , Router , Rank , Filter and Sequence Generator .
• Imported data from various Sources transformed and loaded into Data Warehouse Targets using Informatica Power Center.
• Made substantial contributions in simplifying the development and maintenance of ETL by creating re-usable Source, Target , Mapplets , and Transformation objects.
• Developed JSP pages using Custom tags and Tiles framework and Struts framework .
• Used different user interface technologies JSP , HTML, CSS, and JavaScript for developing the GUI of the application.
• Skills gained on web - based REST API , SOAP API , Apache for real-time data streaming
• Programmed Oracle SQL , T- SQL Stored Procedures, Functions , Triggers and Packages as back-end processes to create and update staging tables, log and audit tables, and creating primary keys.
• Extensively used Transformations like Aggregator, Router, Joiner, Expression, Lookup, Update Strategy, and Sequence Generator.
• Developed mappings, sessions and workflows using Informatica Designer and Workflow Manager based on source to target mapping documents to transform and load data into dimension tables.
• Used FTP services to retrieve Flat Files from the external sources.

Environment: Java, Ajax, Informatica Power Center 8.x/9.x, REST API, SOAP API, Apache, Oracle10/11g, SQL Loader, MS SQL SERVER, Flat Files, Targets, Aggregator, Router, Sequence Generator,