SUMMARY

• Over 8+ years of professional IT experience in requirement gathering, design, development, testing, implementation and maintenance. Progressive experience in all phases of the iterative Software Development Life Cycle (SDLC)
• Good Knowledge onHadoopCluster architecture and monitoring the cluster.
• Focus on Hive querying and Hive optimization.
• Experienced with writing Hive UDF and GenericUDF.
• Experienced with data cleansing in writing Map Reduce jobs and Spark jobs.
• Proficient in Java, Scala and Python, experienced with C and C#.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice versa.
• Good Knowledge on Spark, Storm and HBase to do real time streaming.
• Experienced with real time streaming analytic systems.
• In - depth knowledge of Statistics, Machine Learning, Data mining.
• Experienced with big data machine learning in Mahout and Spark MLlib.
• Experienced with statistic tools Matlab, R and SAS.
• Experienced supervised learning techniques like Multi-Linear Regression, Nonlinear Regression, Logistic Regression, Artificial Neural Networks, Support Vector Machine, Decision tree, Random Forest. Experienced with main unsupervised learning techniques.
• Experienced recommender system.
• In-depth understanding of Data Structure and Algorithms.
• Work with colleagues to transfer from Hadoop System to Spark System.
• Experience in managing and reviewingHadooplog files.
• Excellent understanding and knowledge of NOSQL databases like HBase, Cassandra.
• Experience in implementing in setting up standards and processes forHadoopbased application design and implementation.
• Knowledge of Databases and data warehousing platforms
• Experience in Object Oriented Analysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.
• Excellent understanding / knowledge ofHadooparchitecture and various components such as HDFS, Job Tracker, Task Tracker, NameNode, Data Node and Map Reduce programming paradigm.
• Hands on experience in installing, configuring, and usingHadoopecosystem components like HadoopMap Reduce, HDFS, HBase, Hive, Sqoop, Pig, Zookeeper and Flume.
• Good Exposure on ApacheHadoopMap Reduce programming, PIG Scripting and Distribute Application and HDFS.
• Experience in coordinating .NET application development from onsite. Version control using TFS and code publish/build to QA and Production environments using MS Visual Studio.
• Experience in managingHadoopclusters using Cloudera Manager tool.
• Very good experience in complete project life cycle (design, development, testing and implementation) of Client Server and Web applications.
• Experience in Administering, Installation, configuration, troubleshooting, Security, Backup, Performance Monitoring and Finetuning of Linux Redhat.
• Extensive experience working in Oracle, DB2, SQL Server and My SQL database.
• Hands on experience in VPN, Putty, winSCP, VNCviewer, etc.
• Scripting to deploy monitors, checks and critical system admin functions automation.
• Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
• Ability to adapt to evolving technology, strong sense of responsibility and accomplishment.

TECHNICAL SKILLS

Ecosystem: Big Data HDFS, HBase, Impala,HadoopMap Reduce, Zookeeper, Hive, Pig, Sqoop,Spark

Data Bases: Oracle … (SQL & PL/SQL), Sybase ASE 12.5, DB2, MS SQL Server, MySQL

Programming Languages and Scripting: SQL, PL/SQL, C, C++, PHP, Python, Core Java, JavaScript, Shell Script

Web Technologies: HTML, XML, AJAX, SOAP, ODBC, JDBC, Java Beans, EJB, MVC, JSP, Servlets, Java Mail, Struts, Junit Frameworks MVC, Spring, Struts, Hibernate, .NET

Configuration Management Tools: TFS, CVS IDE / Testing Tools Eclipse.

Data warehousing and NoSQL Databases: Netezza, Hbase.

Methodologies: Agile, V-model

Operating System: Windows, UNIX, Linux

Software Products: Putty, Eclipse, Toad 9.1, DB Visualizer, Comptel's AMD 6.0.3 & 4.0.3, InterConnecT v7.1 & 6.0.7, MS Project 2003, HP Quality Center, MS Management studio, MS SharePoint

PROFESSIONAL EXPERIENCE

Confidential, Foster City, CA

Hadoop Developer

Responsibilities:

• Installed and configuredHadoopMapreduce, HDFS, Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
• Developed workflows using custom Map Reduce, Pig, Hive, Sqoop, Spark
• Work with colleagues to transfer from Hadoop System to Spark System
• Cooperate with Data Scientists to implement Statistic tools, Machine Learning and Recommender System
• Cooperate with Data Scientists to implement Network Analysis algorithms
• Transfer formal Hive queries to Spark SQL.
• Transfer machine learning jobs from Mahout to Spark MLlib
• Wrote Mapreduce jobs and PIG to do ETL in Hadoop System
• Tuned the cluster for optimal performance to process these large data sets
• Built reusable Hive UDF libraries for business requirements which enabled users to use these UDF's in Hive querying
• The logs and semi structured content that are stored on HDFS were preprocessed using PIG and the processed data is imported into Hive warehouse which enabled business analysts to write Hive queries
• Configured big data workflows to run on the top ofHadoopusing Control M and these workflows comprises of heterogeneous jobs like Pig, Hive, Sqoop and MapReduce
• Developed suit of Unit Test Cases for Mapper, Reducer and Driver classes using MR Testing library
• Developed workflow in Control M to automate tasks of loading data into HDFS and preprocessing with PIG
• Used Maven extensively for building jar files of Map Reduce programs and deployed to Cluster
• Bug fixing and 247production support

Environment: CDH3, PIG(0.8.1), HIVE(0.7.1), Sqoop (V1), Oozie (V2.3.2), Core Java, Oracle 11g, SQL Server 2008, Hbase, ClouderaHadoopDistribution, Map Reduce, DataStax, IBM DataStage 8.1, Oracle 11g / 10g, PL/SQL, SQL*PLUS, Toad 9.6, Windows NT, LINUX, UNIX Shell Scripting.

Confidential, New York, NY

Hadoop Developer

Responsibilities:

• Involved in review of functional and nonfunctional requirements.
• Facilitated knowledge transfer sessions.
• Installed and configuredHadoopMap reduce, HDFS, Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
• Working with Storm and Hbase to do real time steaming.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Experience in managing and reviewingHadooplog files.
• Extracted files from RDBMS through Sqoop and placed in HDFS and processed.
• Experience in runningHadoopstreaming jobs to process terabytes of xml format data.
• Got good experience with NOSQL database.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Replaced default Derby metadata storage system for Hive with MySQL system.
• Executed queries using Hive and developed Map Reduce jobs to analyze data.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Developed the Pig UDF's to preprocess the data for analysis.
• Developed Hive queries for the analysts.
• Involved in loading data from LINUX and UNIX file system to HDFS.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig.
• Developed a custom File System plug in forHadoopso it can access files on Data Platform. This plugin allowsHadoopMapReduce programs, HBase, Pig and Hive to work unmodified and access files directly.
• Designed and implemented Map reduce based large scale parallel relation learning system
• Extracted feeds form social media sites such as Facebook, Twitter using Python scripts.
• Setup and benchmarkedHadoop/HBase clusters for internal use
• SetupHadoopcluster on Amazon EC2 using whirr for POC.
• Wrote recommendation engine using mahout.

Environment: Java, Eclipse, Oracle 10g, Sub Version,Hadoop, Hive, HBase, Map Reduce, HDFS, Pig Hive, Cassandra, Java (JDK 1.6),HadoopDistribution of Cloudera, Map Reduce, IBM Data Stage 8.1, Oracle 11g / 10g, PL/SQL, SQL*PLUS, Toad 9.6, Windows NT, LINUX, UNIX Shell Scripting.

Confidential, Albany, NY

JAVA Developer

Responsibilities:

• Involved in business requirements analysis.
• Built the application using Struts framework with JSP as view part.
• Developed Dispatch Actions, Action Forms and Custom taglibs in Struts framework.
• Designed JSP pages as view in Struts for frontend templates.
• Developed Session Beans for handling the back business requirements.
• Used the RSD IDE for development and Clear Case for the versioning.
• Involved in configuring the resources and administering the Web sphere application server 6.
• Built and deployed the application on Web sphere application server.
• Written stored procedures in DB2.
• Developed code to handle web requests involving Request Handlers, Business Objects, and Data Access
• Objects. Has coded different package structures based on the purpose and security issues handled by that particular package which assists developers in future enhancements or modifications of code.
• Involved in making the client side validations with JavaScript.
• Involved in code reviews, system integration and testing. Developed unit test cases using JUnit framework.
• Involved in deploying the application on UNIX (DEV, QA and Prod Environments) box.
• Used Change management tool Service Center for promoting the War file from one environment to other.
• Involved in user acceptance testing, fixing bugs and Production support.

Environment: Core Java, J2EE, Apache Struts, Websphere 5 & 6, JNDI, JDBC, JSP, UNIX and Windows NT, DB2 and SQL Server.

Jr. JAVA Developer

Responsibilities:

• As Team member I involved mainly in implementation and customization of tasks related to module
• Creating of the database tables, writing the queries and stored procedures.
• Coding Java, JSP, and Servlets using the extended Cantata Struts framework.
• Used JNI for calling the libraries and other implemented functionality in C language.
• Involved in writing the programs for the XA transaction management on multiple databases of the application.
• Writing stored procedures & functions (TSQL equal to PL/SQL) in the Sql server DB.
• Used the Stax API / JAXP to read / manipulate the xml properties files.
• Review, Deploying.
• Junit Testing.

Environment: Java, Extended Sturs Framework, JSP, Servlets, SQL, LDAP, Sql Server 2000, JBoss Server 4.0, Eclipse, Ant, CVS, Windows NT and Unix.