SUMMARY

• Around 8 years of experience in IT industry in Big Data Technologies this includes 3 plus years of experience in Big Data, Hadoop stack, 2 Years of experience in SQL and 3+ year of experience as Java developer.
• Well versed in designing and implementing MapReduce jobs using JAVA on Eclipse to solve real world scaling problems.
• Development and maintenance of Scala applications that are executed on the Cloudera platform.
• Extensive experience with wiring SQL queries using HiveQL to perform analytics on structured data.
• Expertise in Data load management, importing & exporting data using SQOOP and FLUME.
• Experienced in Developing Spark application using Spark Core, Spark SQL and Spark Streaming API's.
• Implemented business logic using Pig scripts. Wrote custom Pig UDF to analyse data.
• Performed PIG operations, joining operations and transformations on data to aggregate and analyse data.
• Experience in Managing scalable Hadoop clusters including Cluster designing, provisioning, custom configurations, monitoring and maintaining using Hadoop distributions: Cloudera CDH, Horton Works HDP.
• Good experience using Apache Spark, Storm, kinesis and Kafka.
• Good Knowledge on Spark framework on both batch and real time data processing.
• Good Knowledge and experience inSparkusing Python and Scala.
• Hands on experience in creating Hive UDF's for the requirements and to handle Json and xml files.
• Delivering projects (full SDLC) using big data technologies like Hadoop, Oozie and NoSQL.
• Good understanding of NiFi workflow on picking up files from different locations and moving to HDFS or sending to Kafka brokers.
• Have knowledge on injecting data from multiple data sources to HDFS and Hive using NiFi and importing data using Nifi tool from Linux servers.
• Extensive working knowledge in setting up and running Clusters, monitoring, Data analytics, Sentiment analysis, Predictive analysis, Data presentation with big data world.
• Excellent understanding of NoSQL databases like HBase.
• Set up clusters in Amazon EC2 and S3 including the automation of setting & extending the clusters in AWS.
• Developed MapReduce/Spark Python modules for machine learning & predictive analytics in Hadoop on AWS. Implemented a Python - based distributed random forest via Python streaming.
• Created EBS volumes for storing application files for use with EC2 instansces whenever they are mounted to them.
• Excellent interpersonal and communication skills, creative, research-minded, technically competent and result-oriented with problem solving and leadership skills.

TECHNICAL SKILLS

Big Data Ecosystems: MapReduce, Hive, Sqoop, Spark, Kafka, Pig, Flume, HBase, Oozie

Streaming Technologies: Spark Streaming, Storm

Scripting Languages: Python, Bash, Java Scripting, HTML5, CSS3

Programming Languages: Java, Scala, SQL, PL/SQL

Java/J2EE Technologies: Servlets, JSP, JSF, JUnit, Hibernate, Log4J, EJB, JDBC, JMS, JNDI

Databases: Oracle, RDBMS, NoSQL

IDEs / Tools: Eclipse, JUnit, Maven, Ant, MS Visual Studio, Net Beans

Methodologies: Agile, Waterfall

PROFESSIONAL EXPERIENCE

Confidential, Irving, Tx

Responsibilities:

• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs,
• Developed the code for Importing and exporting data into HDFS and Hive using Sqoop.
• Developed Spark scripts to perform as per client requirement.
• Scheduling Jobs by using Autosys.
• Deployed code using Udeploy.
• Experienced in writing complex SQL queries and scheduled tasks using cron jobs.
• Used Spark andSpark-SQLto read the parquet data and create the tables in hive using the Scala API.
• ImplementedSparkusing Scala and Spark SQL for faster testing and processing of data.
• Implemented Spark using Scala and utilizingData framesand Spark SQL API for faster processing of data.
• Responsible for writing Hive Queries for analyzing data in Hive warehouse using HQL.
• Good Knowledge on Gemfire Cache servers and making necessary modifications.
• Compaction of tables in Hive.
• Developed several advanced Map Reduce programs in Java as part of functional requirements for Big Data.
• Developing Hive User Defined Functions in java, compiling them into jars and adding them to the HDFS and executing them with Hive Queries.
• Experienced in managing and reviewing Hadoop log files.
• Extensively using kafka & Kinesis.
• Installed Hadoop ecosystems (Hive, Pig, Sqoop, HBase, Oozie) on top of Hadoop cluster.
• Experienced in writing complex SQL queries and scheduled tasks using cron jobs.
• Involved in importing data from SQL to HDFS and Hive for analytical purpose.

Environment: Spark, Autosys, Hadoop, Hue, IMPALA, Hive, Java, SQL.

Confidential, Middletown, NJ

Responsibilities:

• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Python and Scala.
• Developed the code for Importing and exporting data into HDFS and Hive using Sqoop.
• Developed Spark scripts to perform as per client requirement in python.
• Involved in developing alinear regressionmodel to predict a continuous measurement for improving the observation on wind turbine data developed using spark with Scala API.
• Used Spark andSpark-SQLto read the parquet data and create the tables in hive using the Scala API.
• ImplementedSparkusing Scala and Spark SQL for faster testing and processing of data.
• Implemented Spark using Scala and utilizingData framesand Spark SQL API for faster processing of data.
• Responsible for writing Hive Queries for analyzing data in Hive warehouse using HQL.
• Developed python and shell scripts to schedule the processes running on a regular basis.
• Developed several advanced Map Reduce programs in Java as part of functional requirements for Big Data.
• Developing Hive User Defined Functions in java, compiling them into jars and adding them to the HDFS and executing them with Hive Queries.
• Experienced in managing and reviewing Hadoop log files.
• Extensively using kafka & Kinesis.
• Tested and reported defects in an Agile Methodology perspective.
• Developed MapReduce/Spark Python modules for machine learning & predictive analytics in Hadoop on AWS. Implemented a Python-based distributed random forest via Python streaming.
• Set up clusters in Amazon EC2 and S3 including the automation of setting & extending the clusters in AWS.
• Experienced with event-driven and scheduledAWSLambda functions to trigger variousAWSresources.
• Experienced with installation ofAWSCLI to control variousAWSservices through SHELL/BASH scripting.
• Installed Hadoop ecosystems (Hive, Pig, Sqoop, HBase, Oozie) on top of Hadoop cluster.
• Experienced in writing complex SQL queries and scheduled tasks using cron jobs.
• Proficient inAWSservices like VPC, EC2, S3, ELB, AutoScalingGroups(ASG), EBS, RDS, IAM, CloudFormation, Route 53, CloudWatch, CloudFront, CloudTrail.
• Involved in importing data from SQL to HDFS and Hive for analytical purpose.

Environment: Spark, Python, Java, SQL, MongoDB, Scala, AWS

Confidential - Houston TX

Hadoop/Spark Developer

Responsibilities:

• Developed spark scripts by using Scala shell as per requirements.
• Worked with spark core, Spark Streaming and spark SQL modules of Spark.
• Developed multiple POCs using Spark and deployed on the Yarn cluster, compared the performance of Spark, with Hive and SQL/Teradata.
• Developed Kafka producer and consumers, Cassandra clients and Spark along with components on HDFS, Hive.
• Development and maintenance of Scala applications that are executed on the Cloudera platform.
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Python and Scala.
• Developed the code for Importing and exporting data into HDFS and Hive using Sqoop.
• Automated Sqoop incremental imports by using Sqoop jobs and automated the jobs using Oozie.
• Responsible for writing Hive Queries for analyzing data in Hive warehouse using HQL.
• Involved in defining job flows using Oozie for scheduling jobs to manage apache Hadoop jobs.
• Developed python and shell scripts to schedule the processes running on a regular basis.
• Developed several advanced Map Reduce programs in Java as part of functional requirements for Big Data.
• Developing Hive User Defined Functions in java, compiling them into jars and adding them to the HDFS and executing them with Hive Queries.
• Experienced in managing and reviewing Hadoop log files.
• Tested and reported defects in an Agile Methodology perspective.
• Installed Hadoop ecosystems (Hive, Pig, Sqoop, HBase, Oozie) on top of Hadoop cluster.
• Involved in importing data from SQL to HDFS and Hive for analytical purpose.
• Implemented the workflows using Oozie framework to automate tasks.

Environment: Hadoop, Hue, HDFS, Spark, MapReduce, Hive, Oozie, Java, Python, NoSQL, Cloudera, Linux, MySQL, SQL.

Confidential

Hadoop/Scala Developer

Responsibilities:

• Developing parser and loader map reduce application to retrieve data from HDFS and store to HBase and Hive.
• Importing the unstructured data into the data lake HDFS using Flume.
• Experienced in working with spark eco system using Spark SQL and Scala queries on different formats like Text file, CSV file.
• Expertized in implementing Spark usingScalaandSpark SQLfor faster testing and processing of data responsible to manage data from different sources.
• Develop new and existing modules in Scala while working with developers across the globe.
• Used Oozie to orchestrate the map reduce jobs that extract the data on a timely manner.
• Written Map Reduce java programs to analyse the log data for large-scale data sets.
• Involved in using HBase Java API on Java application.
• Automated Sqoop jobs for extracting the data from different Data Sources like MySQL to pushing the result set data to Hadoop Distributed File System.
• Implemented Map Reduce jobs using Java API and PIG Latin as well HIVEQL.
• Participated in the setup and deployment of Hadoop cluster.
• Hands on design and development of an application using Hive (UDF).
• Importing and exporting Data from MySQL/Oracle to HiveQL Using SQOOP.
• Designed and built many applications to deal with vast amounts of data flowing through multiple Hadoop clusters, using Pig Latin and Java-based map-reduce.
• Developed UDFs in Java when necessary, to use in PIG and HIVE queries
• Specifying the cluster size, allocating Resource pool, Distribution of Hadoop by writing the specification texts in JSON File format.
• Responsible for defining the data flow within Hadoop eco system and direct the team in implement them.
• Extensive knowledge of Scala/Java development
• Development and maintenance of Scala applications that are executed on the Cloudera platform.
• Performed data validation on target, working with ETL and DQ teams to resolve defects and making any mapping changes if required. Worked with the BI team as part of dashboards, scorecards development.
• Implemented Informatica (ETL) mappings/workflows.
• Used existing ETL standards to develop these mappings.
• Coordinate with ETL team to implement all ETL procedures for all new projects.
• Develop and perform tests on all ETL codes for system data and analyze all data and design all data mapping techniques for all data models in systems.
• Provide support to all ETL schedule and maintain compliance to same and develop and maintain various standards to perform ETL codes and maintain an effective project life cycle on all ETL processes.
• Expertise in design and development of various web and enterprise applications using Type safe technologies like Scala, Akka and Play framework.
• Extensively used Akka actors architecture for scalable & hassle free multi-threading. Millions of activity messages per second were handled very easily by the actors by propagating the messages to appropriate child actors in asynchronous or synchronous fashion.
• Hands on experience on Java8, Scala and Play/Akka framework.
• Worked with Play framework and Akka parallel process.

Environment: Hadoop, HDP (Horton works), Hive, Ambari, Zookeeper, Map Reduce, Sqoop, Pig, UNIX, Java, Eclipse, Oracle, SQL Server, MySQL., ETL, Akka framework.

Confidential

Java Developer

Responsibilities:

• Designed the user interfaces using JSP.
• Developed Custom tags, JSTL to support custom User Interfaces.
• Developed the application using Struts (MVC) Framework.
• Implemented Business processes such as user authentication, Account Transfer using Session EJBs.
• Used Eclipse to write the code for JSP, Servlets, Struts and EJBs.
• Deployed the applications on Web Logic Application Server.
• Used Java Messaging Services (JMS) and Backend messaging for reliable and asynchronous exchange of important information such as payment status report.
• Developed the entire Application(s) through Eclipse.
• Worked with Web Logic Application Server to deploy the Application(s).
• Developed the Ant scripts for preparing WAR files used to deploy J2EE components.
• Used JDBC for database connectivity to Oracle.
• Worked with Oracle Database to create tables, procedures, functions and select statements.
• Used JUnit Testing, debugging, and bug fixing.
• Used Log4J to capture the log that includes runtime exceptions and developed WAR framework to alert the client and production support in case of application failures.
• Performed Data driven testing using Selenium and TestNG functions which reads data from property and XML files. Involved in CICD process using GIT, Jenkins job creation, Maven build and publish.
• Used Maven to build and run the Selenium automation framework. Involved in building and deploying scripts using Maven to generate WAR, EAR and JAR files.

Environment: Eclipse, Web Sphere Application Server, JSP, Servlet, HTML, JUnit, JavaScript, CSS, EJB, Hibernate, Struts, XML, JAXP, CVS, JAX-RPC, AXIS, SOAP, TOAD, AJAX, Jenkins, Maven, Log4J, UNIX, Linux, Java, J2EE, JSP, Struts, JNDI, Oracle 10g, HTML, XML, Web Logic 8.1, Ant, CVS, Log4J, JUnit, JMS, JDBC, JavaScript, Eclipse IDE, UNIX Shell Scripting, Rational Unified Process (RUP).