SUMMARY

• 7 years of professional experience in IT in Analysis, Design, Development, Testing, Documentation, Deployment, Integration, and Maintenance of web based and Client/Server applications using Java and Big Data technologies.
• 2+ years of experience on Big data technologies such as HDFS, MapReduce, YARN, Pig, Hive, Sqoop, HBase, Flume, Kafka, Oozie, Zookeeper.
• 1+ years hands on experience on Spark (Spark SQL, Spark Streaming).
• Expertise in components of Hadoop ecosystem like HDFS, MapReduce, YARN, Pig, Hive, HBase, Sqoop, Oozie, Zookeeper, Impala, Spark.
• Good Understanding of Hadoop architecture and experience with Hadoop components such as JobTracker, TaskTracker, NameNode, DataNode and MapReduce concepts and HDFS Framework.
• Experience in writing Spark SQL scripts and implementing Spark RDD transformations and actions using Scala/ Python.
• Experience working on Tableau and Zeppelin for Data Visualization and Analytics.
• Experience working on Sqoop, Impala, Kafka, Flume and HBase.
• Experience in developing customized UDFs to extend Hive functionality.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice - versa.
• Worked on Scala, Python and Java to create frameworks for processing data pipelines throughSpark.
• Good knowledge on Spark GraphX and Spark MLlib.
• Experience in developing MapReduce programs Hadoopfor analyzing the Big Data.
• Experience with data analytics using R and Python.
• Experience in CoreJavawith strong understanding and working knowledge of Object Oriented Concepts like Collections, Multi-Threading, Exception Handling and Polymorphism.
• Involved in pre-sales activities like requirement gathering & effort estimation.

TECHNICAL SKILLS

Big Data Technologies: HDFS, MapReduce, YARN, Pig, Hive, Sqoop, HBase, Oozie, Impala, Zookeeper, Flume, Kafka

Spark: Spark SQL, Spark Streaming

AWS Cloud Services: S3, EBS, EC2, VPC, EMR, DYNAMODB

Languages: Scala, Python, Java

Databases: Oracle 8.0 & 9i,10g, SQL, NoSQL

Operating Systems: Linux, UNIX, Ubuntu, Windows 2008/2012

Design Tools: Eclipse, ER-Diagram, Sequence Diagrams, Business Process Models, Flow Charts etc.

Data Visualization / Analytics: Tableau, Zeppelin

Environment: CDH 4 & 5, Horton Works

PROFESSIONAL EXPERIENCE

Confidential

Big data Engineer

Responsibilities:

• Gathering the requirements from the business users and performing the requirement analysis.
• Experience in leading a team technically and assigning the tasks to the team and updating the development and the testing status to the manager in a day-to-day basis.
• Worked on 120 nodes, 1PB cluster, with 10TB of monthly incremental data.
• Worked on analyzing Hadoop cluster and different big data tools including Pig, HBase and Sqoop.
• Used Sqoop to import and export data from HDFS to RDBMS and vice-versa.
• Created HBase tables to store various data formats of data coming from different portfolios.
• Used Impala for performance upgrade for hive scripts.
• Used Cloudera Manager for installation and management of Hadoop Cluster.
• Wrote Pig scripts to run ETL jobs on the data in HDFS.
• Involved in scheduling Oozie workflow engine to run multiple Hive and pig jobs.
• Extensively used Spark for optimizing of existing process in Hadoop using Spark Context, Spark SQL, Data Frames and pair RDDs.
• Developed Spark core and Spark SQL scripts using Scala for faster data processing.
• Developed Spark scripts to import large files from Amazon S3 buckets.
• DevelopedSparkapplications in python (PySpark) on distributed environment.
• Configured Spark Streaming to receive real time data from Kafka and store the stream data to HDFS.
• Extensively worked on Spark by using notebook-style Zeppelin for data visualization.
• Worked with BI teams in generating the reports in Tableau.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Environment: Hadoop, MapReduce, Hive, HDFS, Pig, Sqoop, Oozie, Spark, Flume, Kafka, HBase, Impala, ZooKeeper, Tableau, Zeppelin, CDH 5, Oracle, NoSQL and Unix/Linux.

Confidential, Princeton, NJ

Hadoop Consultant

Responsibilities:

• Worked on 35 nodes, installed Cloudera CDH 5.4.0 clusters for application development and Hadoop tools like Hive, Pig, HBase, Zookeeper and Sqoop.
• Internally trained team members on Hadoop Ecosystem.
• Developed data pipeline using Sqoop, Pig and Hive to structure customer behavioral data and financial histories in HDFS for analysis.
• Did optimization of Hive by converting input data from Text file to ORC achieving 78% compression and faster execution
• Involved in collecting and aggregating large amounts of log data using Flume and staging data in HDFS for further analysis.
• Involved in planning infrastructure and design to move from Flume to Kafka in future incorporating fault tolerance and multiple data sources.
• Used Pig as ETL tool to do transformations, event joins and some pre-aggregations and storing the data onto HDFS.
• Installed Oozie workflow engine to run multiple Sqoop, Hive and Pig Jobs.
• Used Sqoop to import and export data from HDFS to RDBMS and vice-versa.
• Created Hive tables and involved in data loading and writing Hive UDFs.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.
• Environment: Hadoop, MapReduce, Hive, HDFS, Pig, Sqoop, Oozie, Flume, Kafka, ZooKeeper, CDH 5, Oracle, NoSQL and Unix/Linux.

Confidential

Java Consultant

Responsibilities:

• Worked on both WebLogic Portal 9.2 for Portal development and WebLogic 8.1 for Data Services Programming.
• Developed the presentation layer using JSP, HTML, CSS and client validations using JavaScript.
• Developed user interface using JSP, Struts Tag Libraries to simplify the complexities of the application.
• Developed business logic using Stateless session beans for calculating asset depreciation on Straight line and written down value approaches.
• Database Modification using SQL, PL/SQL, Stored procedures, triggers, Views in Oracle.
• Created java classes to communicate with the database using JDBC.
• Responsible for design and implementation of various modules of the application using Struts-Spring-Hibernate architecture.
• Developed the Web Interface using Servlets, Java Server Pages, HTML, and CSS.
• Extensively used the JDBC Prepared Statement to embed the SQL queries into the java code.
• Developed DAO (Data Access Objects) using Spring Framework 3.
• Developed Web applications with Rich Internet applications using Java applets, Silverlight, Java.
• Deployed this application, which uses J2EE architecture model and Struts Framework first on WebLogic and helped in migrating to JBoss Application server.
• Used JavaScript to perform client-side validations and Struts-Validator Framework for server-side validation.
• Designed and developed the application using various design patterns, such as session facade, business delegate and service locator.
• Involved in designing use-case diagrams, class diagrams, interaction using UML model with Rational Rose.
• Environment: Java, Servlets, JSP, EJB, J2EE, STRUTS, XML, XSLT, JavaScript, HTML, CSS, Spring 3.2, SQL, PL/SQL, MS Visio, Eclipse, JDBC, Windows XP, WebLogic portal, JBoss.

Confidential

Sr Software Engineer

Responsibilities:

• Installed and configured RHEL 2.1/3/4, Solaris 10 and Red hat on Intel and AMD hardware
• Configured hands-free installation using Kick start and PXE
• Involved in installing, configuration and administration of SQL Server 2000 & 2005
• Monitored Linux and Solaris servers using tools like vmstat, SAR, top, free etc.
• Installed, updated and erased the required packages to provide services using rpm and up2date.
• Experience writing SQL queries, Stored Procedures, Triggers, Functions, creating Tables, Views and Indexes.
• Partitioned disks using Disk Druid, fdisk with raid options and multipathing with power path on SAN devices
• Configured/scheduled tasks using cron.
• Created thedatabase, tables, table spaces, data objects, setting privileges, indexes, user logins, and passwords (security)
• Resolve and troubleshootdatabaseissues, and perform upgrades, conversions and deployments.
• Create, deploy, configure and monitor complex ETL/SSIS packages
• Configured SSIS services,DatabaseMail, and Alerts on cluster.
• Manage and resolve trouble tickets fordatabase-related issues submitted by customers.
• Implement and maintaindatabasesecurity, replication, SQL Clustering, DTS, Log shipping, SQL Mail.
• Resolve and troubleshootdatabaseissues, and perform upgrades, conversions and deployments.
• Worked on creating new file system, mounting file system, monitoring free space, disk usage, locating files and checking & clearing log files.
• Worked with NOC (Network Operations Center).
• Performed administrative tasks such as System Startup/shutdown, Backup strategy, Printing, Documentation, User Management, Security and Network management
• Provided responsive off-hour support in a 24/7 environment and ensured maximum availability of all servers and applications
• Environment: Linux, Networking, Security, MS Sql Server 2008.