SUMMARY

• 8+ years of experience in the field of IT including five years of experience in Hadoop ecosystem and four years of experience as a Java developer wif good object - oriented programming skills.
• Expertise in design and development of various web and enterprise applications using Type safe technologies like Scala, Akka, Play framework, Slick.
• Experienced in using Scala, Java tools like Intelli J, Eclipse.
• Good noledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, MapReduce concepts responsible for writing MapReduce programs and setting up standards and processes for Hadoop-based application design and implementation.
• Expertise wif different tools in Hadoop Environment including Pig, Hive, HDFS, MapReduce, Sqoop, Spark, Kafka, Yarn, Oozie, and Zookeeper.
• Extensive work in ETL process consisting of data transformation, data sourcing, mapping, conversion and loading using Informatica.
• Expertise in developing data driven applications usingPython2.7,Python3.0 on Pycharm and Anaconda Spyder IDE's.
• Expertise wif data visualization and analysis tools like R, WEKA and Scikit-learn usingPython.
• Experience in installation, configuration, and deployment of Big Data solutions.
• Experience in using Cloudera Manager for installation and management of single-node and multi-node Hadoop cluster.
• Worked on Multi Clustered environment and setting up Hortonworks Hadoop echo -System.
• Hands on experience in configuring and working wif Flume to load the data from multiple sources directly into HDFS.
• Expertise in MapReduce programs in HIVE and PIG to validate and cleanse the data in HDFS, obtained from heterogeneous data sources, to make it suitable for analysis.
• Analyzed or transformed stored data by writing MapReduce jobs based on business requirements.
• Good noledge about YARN configuration.
• Experience in developing Pig scripts and Hive Query Language.
• Managing and scheduling batch Jobs on a Hadoop Cluster using Oozie.
• Hands on experience working wif NoSQL database including MongoDB and HBase.
• Experience in optimizing MapReduce jobs to use HDFS efficiently by using various compression mechanisms.
• Participated in multiple big data POCs to evaluate different architectures, tools and vendor products.
• Experience in developing Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Hands on experience in using Sqoop to import data into HDFS from RDBMS and vice-versa.
• Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
• Expertise in Web technologies using Core Java, J2EE, Servlets, EJB, JSP, JDBC, Java Beans, and Design Patterns.
• Detailed understanding of Software Development Life Cycle (SDLC) and sound noledge of project implementation methodologies including Waterfall and Agile.
• Ability to adapt to evolving technology, a strong sense of responsibility and . Willing to relocate: Anywhere.

TECHNICAL SKILLS

Operating Systems: Windows 8/7/XP, Unix, Ubuntu 13.X, Mac OSX

Hadoop Eco System: Hadoop 1.x/2.x(Yarn), HDFS, Map Reduce, Mongo, HBase, Hive, PIG, Apache Spark, Zookeeper, Sqoop, Oozie, Flume, Storm, HDP, AWS, Eclipse, Cloudera-desktop and SVN.

Java Tools: Java, Map Reduce, J2EE (JSP, Servlets, EJB, JDBC, JMS, JNDI, RMI), Struts, Hibernate, AJAX, XSLT, HTML, JavaScript, CSS, JUnit, JAVA, JSP, JSON, J2EE, Web Services, DHTML, Javascript, DOM, SAX, Jquery, XML, XSLT.

API’s: Servlets, EJB, Java Naming, and Directory Interface(JNDI), MapReduce

Development Tools: Eclipse, RAD/RSA (Rational Software Architect), IBM DB2 Command Editor, SQL Developer, Microsoft Suite (Word, Excel, PowerPoint, Access), Open Office Suite (Editor, Calc etc..), VM Ware

Languages: Scala, Java, Java EE, JSP, Python

No SQL Databases: HBase, Cassandra, Monod

Servers: Web sphere (WAS) 6.x/7.0, Web Logic 10-12c

PROFESSIONAL EXPERIENCE

Confidential - Reston, VA

Sr. Big Data Developer

Responsibilities:

• Supported the design, development and implementation of Big Data solutions for the Informatics Team.
• Determined the Assess requirements and best technologies and design patterns to utilize for data-centric solutions.
• Design, build and launch extremely efficient & reliable data pipelines for real time streaming, search and Indexing solutions.
• Worked closely wif Cloudera Administrators to optimize the usage of our clusters and plan for future expansion and usage.
• Worked in an agile environment, collaborate wif Solution Architects, Scrum Masters, Developers and Testers.
• Developed MapReduce/Spark Python modules formachinelearning & predictive analytics inHadoop.
• Utilized Spark, Scala,Hadoop, HBase, Kafka, Spark Streaming, MLLib, Python, a broad variety ofmachinelearningmethods including classifications, regressions, dimensionally reduction etc.
• Used Python and Spark to implement differentmachinelearningalgorithms including Generalized Linear Model.
• Implemented HBase/SOLR for real time search/indexing at production scale.
• Designed ControlIM jobs by domain area, which resulting into longer wait times for other jobs that are not dependent.
• Created event-based job execution across various application projects.
• Evaluated the top 10 long running jobs for optimization and performance efficiencies.
• Allocated maximum server resources to the processes enabling parallel runs for all jobs wif maximum threads per job.
• Worked wif Sqoop in Importing and exportingdatafrom different databases like DB2, AQDB into HDFS and Hive.
• Create queues and allocated the clusters resources to provide the priority for jobs.
• Captured thedatalogs from web server into HDFS using Flume for analysis.

Environment: Hadoop, HDFS, Hive, IBM Q-rep, Sqoop, Parquet, HiveQL, Spark, Ab Initio, MQ, Flume, Kafka, ControlIM, DB2, AQDB, Cloudera, Python, Scala.

NCR Corporation - Atlanta, GA Sr. Hadoop Developer

Description: Today, NCR works wif more than 28,000 businesses around the world. I was a part of Big Data Processing Team to take advantage of available user data to make better decisions that significantly enhanced organizational success. I was involved in setting up Hadoop Cluster and wrote MapReduce jobs, Hive queries and Pig Latin scripts to explore through the data to find significant information for trend analysis.

Responsibilities:

• Developed Spark code using Scala and Spark-SQL for faster testing and processing of data.
• Used Spark API over Cloudera Hadoop YARN to perform analytics on data in Hive.
• Exploring wif the Spark improving the performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, Pair RDD's, Spark YARN.
• Experienced wif batch processing of data sources using Apache Spark, Elastic search.
• Involved in converting Hive/SQL queries intoSparktransformations usingSparkRDDs, Python.
• Developed analytical components using Scala, Spark, Apache Mesos and Spark Stream.
• Experienced wif NoSQL databases like HBase, MongoDB and Cassandra.
• Experience in migration of data across cloud environment to Amazon EC2 clusters.
• Developed multiple POCs using PySpark and deployed on the Yarn cluster, compared the performance ofSpark, wif Hive and SQL/Teradata.
• Compiled SQL queries to native HBase scans by using Apache Phoenix.
• Used Phoenix to perform the dynamic and ad-hoc queries to build machine learning models that represent normal activity, in order to detect abnormal, anomalous and risky behaviors.
• Analyzed the SQL scripts and designed the solution to implement using PySpark
• Uploaded data to Hadoop Hive and combined new tables wif existing databases.
• Very good Experience wif IBM WebSphere Application.
• Deployed the Cassandra cluster in cloud (Amazon AWS) environment wif scalable nodes as per the business requirement.
• Generated the data cubes using hive, Pig, JAVA Map-Reducing on provisioning Hadoop cluster in AWS.
• Understanding of data storage and retrieval techniques, ETL, and databases, to include graph stores, relational databases, tuple stores

Environment: Hadoop, MapReduce, HDFS, Hive, Apache Spark, Apache Kafka, Apache Cassandra, Amazon EC2, MongoDB, Hbase, Java, SQL, HDP, Sqoop, Flume, Oozie, Java (jdk 1.6), Eclipse.

Confidential - Pittsburgh, PA

Sr. Hadoop Developer

Responsibilities:

• Worked wif the advanced analytics team to design fraud detection algorithms and tan developed MapReduce programs to efficiently run the algorithm on the huge datasets.
• Proficient at usingSparkAPIs to cleanse, explore, aggregate, transform, and store machine sensor data.
• Ran data formatting scripts inpythonand created terabyte csv files to be consumed byHadoop MapReduce jobs.
• Experience wifPysparkfor using Spark libiries by using python scripting for data analysis.
• DevelopedPySparkcode to mimic the transformations performed in the on-premise environment.
• Performed data analysis, feature selection, feature extraction using Apache Spark Machine Learning streaming libraries inPython.
• Played a key-role is setting up a 50 node Hadoop cluster utilizing Apache Spark by working closely wif the Hadoop Administration team.
• Created Hive tables to store data into HDFS, loading data and writing hive queries that will run internally in map-reduce way.
• Involved in Cluster coordination services through Zookeeper.
• Working wif technologies such asApacheStorm, Coherence Cache, SAP Event Stream Processor (CEP), JMS (Java Message Service).
• UsedApacheStormand associated topologies to ingest, process and store stream data fromApache Kafka into HBase.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Played a key role in installation and configuration of the various Hadoop ecosystem tools such as Solr, Kafka, Pig, HBase and Cassandra.
• Implemented various hive optimization techniques like Dynamic Partitions, Buckets, Map Joins, Parallel executions in Hive.
• Involved in scheduling Airflow workflow engine to run multiple Hive and pig jobs using python.
• Designed and Maintained Oozie workflows to manage the flow of jobs in the cluster.
• Parse Json files through Spark core to extract schema for the production data using SparkSQL and Scala.
• Actively updated the upper management wif daily updates on the progress of project that include the classification levels that were achieved on the data.

Environment: Hadoop, HDFS, Pig, Hive, MapReduce, Sqoop, Kafka, CDH3, Cassandra, Python, Oozie, Java collection, Scala, AWS cloud, Storm, SQL, NoSQL, Bitbucket, Jenkins, HBase, Flume, Apache spark, Solr, Zookeeper, ETL, Centos, Eclipse.

Confidential - Overland Park, KS

Hadoop Developer

Responsibilities:

• Involved in Installing, Configuring Hadoop community, and Hortonworks platforms utilizing HDP2.3 Distribution.
• Involved in making Hive tables, stacking the information and composing Hive queries that will run inside in MapReduce.
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDsand Scala.
• WrotePythonmodules to view and connect the Apache Cassandra instance
• Involved in writing MapReduce jobs.
• Real-time streaming the information utilizing Spark wif Kafka.
• Responsible for creating information pipeline utilizing flume, Sqoop, and pig to remove the information from weblogs and store in HDFS.
• Involved in emitting processed information from Hadoop to relational databases or external frameworks utilizing Sqoop, HDFS GET or CopyToLocal.
• Developed data pipeline utilizing Flume, Sqoop, Pig and Java MapReduce to ingest client behavioral information and money related histories into HDFS for analysis.
• Experienced in managing and assessing Hadoop log records.
• Used Pig to do changes like event joins, filter boot traffic and some pre-aggregations before storing the information onto HDFS.
• Written Hive inquiries for data to meet the business requirements.
• Importing and sending out information into HDFS and Hive utilizing Sqoop and Kafka.
• Developed an information pipeline utilizing Kafka and Storm to store data into HDFS.
• Created various Parser programs to extract data from Autosys, Tibco Business Objects, XML, Informatica, Java, and database views usingScala.
• Loading spilling data using Kafka, Flume and real time Using Spark and Storm.
• Involved in creating Hive Tables, loading wif data and writing Hive queries which will invoke and run MapReduce jobs in the backend.
• Implemented the workflows using Apache Oozie framework to automate tasks.
• Developed scripts and automated data management from end to end and sync up between all the clusters.

Environment: Hadoop, Scala, MapReduce, Mongo, Yarn, Hive, Pig, Storm, HBase, Oozie, Sqoop, Flume, Core Java, HDP, HDFS, Eclipse, Kafka.

Confidential - Boston, MA

Hadoop Developer

Responsibilities:

• Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
• Involved in installing and updating and managing AWS Environment.
• Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
• Involved in running Hadoop streaming jobs to process terabytes of XML format data.
• Participated in requirement gathering and analysis phase of the project in documenting the business requirements by conducting workshops/meetings wif various business users.
• Involved in Sqoop, HDFS Put or CopyFromLocal to ingest data.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Created and maintained technical documentation for launching Hadoop clusters and for executing Hive queries and Pig Scripts.
• Implemented test scripts to support test-driven development and continuous integration.
• Implemented SQL, PL/SQL Stored Procedures.
• Involved in developing Shell scripts to orchestrate the execution of all other scripts (Pig, Hive, and MapReduce) and move the data files wifin and outside of HDFS.
• Involved in developing Hive UDFs for the needed functionality that is not out of the box available from Apache Hive.
• Actively updated the upper management wif daily updates on the progress of a project that include the classification levels that were achieved on the data.

Environment: Hadoop, MapReduce, NoSQL, Hive, AWS, Pig, Sqoop, Core Java, HDP, HDFS, Eclipse.

Confidential

Java Developer

Responsibilities:

• Involved in Analysis, design, and coding on J2EE Environment.
• Implemented MVC architecture using Struts, JSP, and EJB's.
• Used Core Java concepts in an application such as multithreaded programming, synchronization of threads used thread to wait, notify, join methods etc.
• Presentation layer design and programming on HTML, XML, XSL, JSP, JSTL and Ajax.
• Creating cross-browser compatible and standards-compliant CSS-based page layouts.
• Worked on Hibernate object/relational mapping related to the database schema.
• Designed, developed and implemented the business logic required for Security presentation controller.
• Used JSP, Servlet coding under J2EE Environment.
• Designed XML files to implement most of the wiring need for Hibernate annotations and Struts configurations.
• Responsible for developing the forms, which contains the details of the employees, and generating the reports and bills.
• Developed Web Services for data transfer from client to server and vice versa using Apache Axis, SOAP, and WSDL.
• Involved in designing of class and data flow diagrams using UML Rational Rose.
• Created and modified Stored Procedures, Functions, Triggers and Complex SQL Commands using PL/SQL.
• Involved in the Design of ERD (Entity Relationship Diagrams) for the Relational database.
• Developed Shell scripts in UNIX and procedures using SQL and PL/SQL to process the data from the input file and load into the database.
• Used CVS for maintaining the Source Code Designed, developed and deployed on WebLogic Server.
• Performed Unit Testing on the applications that are developed.

Environment: Java (JDK 1.6), J2EE, JSP, Servlet, Hibernate, JavaScript, JDBC, Oracle 10g, UML, Rational Rose, SOAP, Web Logic Server, JUnit, PL/SQL, CSS, HTML, XML, Eclipse

Confidential

Java Developer

Responsibilities:

• Worked on both WebLogic Portal 9.2 for Portal development and WebLogic 8.1 for Data Services Programming. Developed the presentation layer using JSP, HTML, CSS and client validations using JavaScript.
• Developed user interface using JSP, Struts Tag Libraries to simplify the complexities of the application.
• Developed business logic using Stateless session beans for calculating asset depreciation on Straight line and written down value approaches.
• Database Modification using SQL, PL/SQL, Stored procedures, triggers, Views in Oracle.
• Created java classes to communicate wif the database using JDBC.
• Responsible for design and implementation of various modules of the application using Struts-Spring-Hibernate architecture.
• Developed the Web Interface using Servlets, Java Server Pages, HTML, and CSS.
• Extensively used the JDBC Prepared Statement to embed the SQL queries into the java code.
• Developed DAO (Data Access Objects) using Spring Framework 3.
• Developed Web applications wif Rich Internet applications using Java applets, Silverlight, Java.
• Deployed dis application, which uses J2EE architecture model and Struts Framework first on WebLogic and halped in migrating to JBoss Application server.
• Used JavaScript to perform client-side validations and Struts-Validator Framework for server-side validation.
• Designed and developed the application using various design patterns, such as session facade, business delegate and service locator.
• Involved in designing use-case diagrams, class diagrams, interaction using UML model wif Rational Rose.

Environment: Java, Servlets, JSP, EJB, J2EE, STRUTS, XML, XSLT, JavaScript, HTML, CSS, Spring 3.2, SQL, PL/SQL, MS Visio, Eclipse, JDBC, Windows XP, WebLogic portal, JBoss.