SUMMARY

• 8 years of overall IT experience in a variety of industries, which includes hands on experience of 3+ years in Big Data Analytics and development.
• Experience with the tools in Hadoop Ecosystem including Pig, Hive, HDFS, MapReduce, Sqoop, Storm, Spark, Kafka, Yarn, Oozie, and Zookeeper.
• Excellent knowledge on Hadoop ecosystems such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming paradigm.
• Experience in manipulating/analyzing large datasets and finding patterns and insights within structured and unstructured data.
• Strong experience on Hadoop distributions like Cloudera, MapR and HortonWorks.
• Good understanding of NoSQL databases and hands on work experience in writing applications on NoSQL databases like HBase, Cassandra and MongoDB.
• Experienced in writing complex MapReduce programs that work with different file formats like Text, Sequence, Xml, parquet and Avro.
• Experience in Oozie and workflow scheduler to manage Hadoop jobs by Direct Acyclic Graph (DAG) of actions with control flows.
• Experience in migrating the data using Sqoop from HDFS to Relational Database System and vice - versa according to client's requirement.
• Extensive Experience on importing and exporting data using stream processing platforms like Flume and Kafka.
• Experience in complete project life cycle (design, development, testing and implementation) of Client Server and Web applications.
• Excellent Java development skills using J2EE, J2SE, Servlets, JSP, EJB, JDBC, SOAP and RESTful web services.
• Strong Experience of Data Warehousing ETL concepts using Informatica Power Center, OLAP, OLTP and AutoSys.
• Experience in database design using PL/SQL to write Stored Procedures, Functions, Triggers and strong experience in writing complex queries for Oracle.
• Experienced in working with Amazon Web Services (AWS) using EC2 for computing and S3 as storage mechanism.
• Strong experience in Object-Oriented Design, Analysis, Development, Testing and Maintenance.
• Excellent implementation knowledge of Enterprise/Web/Client Server using Java, J2EE.
• Experienced in using agile approaches, including Extreme Programming, Test-Driven Development and Agile Scrum.
• Worked in large and small teams for systems requirement, design & development.
• Key participant in all phases of software development life cycle with Analysis, Design, Development, Integration, Implementation, Debugging, and Testing of Software Applications in client server environment, Object Oriented Technology and Web based applications.
• Experience in using various IDEs Eclipse, IntelliJ and repositories SVN and Git.
• Experience of using build tools Ant, Maven.
• Preparation of Standard Code guidelines, analysis and testing documentations.

TECHNICAL SKILLS

BigData/Hadoop Technologies: HDFS, YARN, MapReduce, Hive, Pig, Impala, Sqoop, Flume, Spark, Kafka, Storm, Drill, Zookeeper and Oozie

NO SQL Databases: HBase, Cassandra, MongoDB

Languages: C, Java, Scala, Python, SQL, PL/SQL, Pig Latin, HiveQL, Java Script, Shell Scripting

Java & J2EE Technologies: Core Java, Servlets, Hibernate, Spring, Struts, JMS, EJB, RESTful

Application Servers: Web Logic, Web Sphere, JBoss, Tomcat.

Cloud Computing Tools: Amazon AWS.

Databases: Microsoft SQL Server, MySQL, Oracle, DB2

Operating Systems: UNIX, Windows, LINUX

Build Tools: Jenkins, Maven, ANT

Business Intelligence Tools: Tableau, Splunk, Qlik View

Development Tools: Microsoft SQL Studio, Eclipse, NetBeans, IntelliJ

Development Methodologies: Agile/Scrum, Waterfall

Version Control Tools: Git, SVN

PROFESSIONAL EXPERIENCE

Confidential, NJ

Sr. Hadoop/Spark Developer

Responsibilities:

• Responsible for building scalable distributed data solutions using Hadoop.
• Experience in Job management using Fair scheduler and Developed job processing scripts using Oozie workflow.
• Configured deployed and maintained multi-node Dev and Test Kafka Clusters.
• DevelopedSparkscripts by using Scala shell commands as per the requirement.
• UsedSparkAPI over Cloudera Hadoop YARN to perform analytics on data in Hive.
• Developed UDFFs using both Data frames/SQL/Data sets and RDD/MapReduce inSpark1.6 for Data Aggregation, queries and writing data back into OLTP system through Sqoop.
• Experienced in performance tuning ofhive for setting right Batch Interval time, correct level of Parallelism and memory tuning.
• Implemented ELK (Elastic Search, Log stash, Kibana) stack to collect and analyse the logs produced by the spark cluster.
• Performed advanced procedures like text analytics and processing, using the in-memory computing capabilities ofSpark.
• Experienced in handling large datasets using Partitions in hive.
• Designed, developed and did maintenance of data integration programs in a Hadoop and RDBMS environment with both traditional and non-traditional source systems as well as RDBMS and NoSQL data stores for data access and analysis.
• Worked on a POC to compare processing time of Impala with Apache Hive for batch applications to implement the former in project.
• Worked on Cluster of size 130 nodes.
• Worked extensively with Sqoop for importing metadata from Oracle.
• Responsible for developing data pipeline with Amazon AWS to extract the data from weblogs and store in HDFS.
• Involved in creating Hive tables, and loading and analysing data using hive queries.
• Developed Hive queries to process the data and generate the data cubes for visualizing.
• Implemented schema extraction for Parquet and Avro file Formats in Hive.
• Good experience with Talend open studio for designing ETL Jobs for Processing of data.
• Implemented Partitioning, Dynamic Partitions, Buckets in HIVE.
• Setup data authorization roles for Hive and Impala usingApacheSentry.
• Monitor Hadoop cluster connectivity and security using Apache Sentry.
• Experience with continuous Integration of application using Jenkins.
• Used Reporting tools like Tableau to connect with Hive for generating daily reports of data.
• Collaborated with the infrastructure, network, database, application and BI teams to ensure data quality and availability.

Environment: Hadoop YARN, Spark Core, Spark Streaming, Spark SQL, Scala, Python, Kafka, Hive, Sqoop, Amazon AWS, Elastic Search, Impala, Cassandra, Tableau, Talend, Oozie, Jenkins, Cloudera, Oracle 12c, Linux.

Confidential, Irving, TX

Hadoop Developer

Responsibilities:

• Worked on analysing Hadoop cluster using different big data analytic tools including Pig, Hive and MapReduce.
• Managing fully distributed Hadoop cluster is an additional responsibility assigned to me.
• I was trained to overtake the responsibilities of a Hadoop Administrator, which includes managing the cluster, Upgrades and installation of tools that uses Hadoop ecosystem.
• Worked on Installation and configuring of Zoo Keeper to co-ordinate and monitor the cluster resources.
• Implemented test scripts to support test driven development and continuous integration.
• Consumed the data from Kafkausing Apache spark.
• Load and transform large sets of structured, semi structured and unstructured data.
• Involved in loading data from LINUX file system to HDFS.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Implemented Partitioning, Dynamic Partitions, Buckets in Hive.
• Worked in creating HBase tables to load large sets of semi structured data coming from various sources.
• Extending HIVE and PIG core functionality by using custom User Defined Function’s (UDF), User Defined Table-Generating Functions (UDTF) and User Defined Aggregating Functions (UDAF) for Hive and Pig using python.
• Experienced in running Hadoop streaming jobs to process terabytes of xml format data.
• Involved in scheduling Oozie workflow engine to run multiple Hive and pig jobs
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig.
• Installed Kerberos for the cluster.
• Implemented Cluster Security using Kerberos and HDFS ACLs.
• Responsible for loading data files from various external sources like ORACLE, MySQL into staging area in MySQL databases.
• Executed Hive queries on Parquet tables stored in Hive to perform data analysis to meet the business requirements.
• Actively involved in code review and bug fixing for improving the performance.
• Experience in handling data manipulation using python.
• Involved in development, building, testing, and deploy toHadoopcluster in distributed mode.
• Deployed an Apache Lucene/Solrsearch engine server to help speed up the search of the technical documentation asset.
• Executed Hive queries on Parquet tables stored in Hive to perform data analysis to meet the business requirements.
• Created HBase tables to store various data formats of incoming data from different portfolios.
• Created Pig Latin scripts to sort, group, join and filter the enterprise wise data.
• Developed the verification and control process for daily load.
• Experience in Daily production support to monitor and trouble shootsHadoop/Hive jobs.

Environment: Hadoop, HDFS, Pig, Apache Hive, Sqoop, Kafka, Apache Spark, Storm, Solr, Shell Scripting, HBase, Python, Kerberos, Agile, Zoo Keeper, Maven, Ambari, Horton Works, MySQL.

Confidential

Big Data / Hadoop Developer

Responsibilities:

• Worked on the proof-of-concept for Apache Hadoop 1.20.2 framework initiation
• Installed and configured Hadoop clusters and eco-system
• Developed automated scripts to install Hadoop clusters
• Involved in all phases of the Big Data Implementation including requirement analysis, design, development, building, testing, and deployment ofHadoopcluster in fully distributed modeMapping the DB2 V9.7, V10.x Data Types to Hive Data Types and validations.
• Performed load and retrieve unstructured data (CLOB, BLOB etc.)
• Developed Hive jobs to transfer 8 years of bulk data from DB2, MS SQL Server to HDFS layer.
• Implemented Data Integrity and Data Quality checks inHadoopusing Hive and Linux scripts.
• Manage and administration of Hadoop cluster environment usingClouderaManager andCloudera Navigator
• Job automation framework to support & operationalize data loads
• Automated the DDL creation process in hive by mapping the DB2 data types
• Monitored Hadoop cluster job performance and capacity planning.
• Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
• Had experience in Hadoop framework, HDFS, MapReduce processing implementation.
• Tuning Hadoop performance with high availability and involved in recovery of Hadoop clusters.
• Involved in the User/Group Management in Hadoop with AD/LDAP integration.
• Responsible for coding Java Batch, Restful Service, Map Reduce program, Hive query’s, testing, debugging, Peer code review, troubleshooting and maintain status report.
• Designed Business classes and used Design Patterns like Data Access Object, MVC etc.
• Used AVRO, Parquet file formats for serialization of data.
• Good experience with ETL data flow using informatica power center.
• Developed several test cases using MR Unit for testing Map Reduce Applications
• Responsible for troubleshooting and resolving the performance issues of Hadoop cluster.
• Used Bzip2 compression technique to compress the files before loading it to Hive
• Used Flume to collect, aggregate, and store the web log data from different sources like web servers, mobile devices and pushed to HDFS.
• Experience in using HBase as backend database for the application development.
• Support/Troubleshoot hive programs running on the clusterand Involved in fixing issues arising out of duration testing.
• Prepare daily and weekly project status report and share it with the client.

Environment: Hadoop, Map Reduce, Flume, Sqoop, Hive, Pig, Restful Service, Linux, Core Java, Informatica,HBase, Avro, JIRA, Git, Cloudera, MR Unit, MS-SQL Server, UNIX, DB2

Confidential

Sr. Java/J2EE Developer

Responsibilities:

• Involved in Requirement Analysis, Design, Development and Testing of the risk workflow system.
• Involved in the implementation of design using vital phases of the Software development life cycle (SDLC) that includes Development, Testing, Implementation and Maintenance Support.
• Applied OOAD principle for the analysis and design of the system.
• Implemented XML Schema as part of XQuery query language
• Applied J2EE design patterns like Singleton, Business Delegate, Service Locator, Data Transfer Object (DTO), Data Access Objects (DAO) and Adapter during the development of components.
• Used RAD for the Development, Testing and Debugging of the application.
• Used WebSphere Application Server to deploy the build.
• Developed front-end screens using Struts, JSP, HTML, AJAX, JQuery, Java script, JSON and CSS.
• Used J2EE for the development of business layer services.
• Developed Struts Action Forms, Action classes and performed action mapping using Struts.
• Performed data validation in Struts Form beans and Action Classes.
• Developed POJO based programming model using spring framework.
• Used IOC (Inversion of Control) Pattern and Dependency Injection of Spring framework for wiring and managing business objects.
• Used Web Services to connect to mainframe for the validation of the data.
• SOAP has been used as a protocol to send request and response in the form of XML messages.
• JDBC framework has been used to connect the application with the Database.
• Used Eclipse for the Development, Testing and Debugging of the application.
• Log4j framework has been used for logging debug, info & error data.
• Used Hibernate framework for Entity Relational Mapping.
• Used Oracle 10g database for data persistence and SQL Developer was used as a database client.
• Extensively worked on Windows and UNIX operating systems.
• Used SecureCRT to transfer file from local system to UNIX system.
• Performed Test Driven Development (TDD) using JUnit.
• Used Ant script for build automation.
• PVCS version control system has been used to check-in and checkout the developed artifacts. The version control system has been integrated with Eclipse IDE.
• Used Rational Clear quest for defect logging and issue tracking.

Environment: Windows XP, Unix, RAD7.0, Core Java, J2EE, Struts, Spring, Hibernate, Web Services, Design Patterns, WebSphere, Ant, (Servlet, JSP), HTML, AJAX, JavaScript, CSS, jQuery, JSON,SOAP, WSDL, XML, Eclipse, Agile, Jira, Oracle 10g, Win SCP, Log4J, JUnit.

Confidential

Intern Java/J2EE Developer

Responsibilities:

• Impact analysis of change requests, preparing change release plan.
• Implemented the Inversion of Control in Spring Framework.
• Develop and maintain critical modules in Java, JSP & Servlets.
• Wrote WSDL and SOAP to send and get data from the third party external interface.
• Migrated the front end to Spring Framework from a simple Java, JSP.
• Used iBatis SQL maps to insert the third-party data into the database.
• Release changes to QA, STAGE and provide support to Production release team.
• Allocate and track work among team members.
• Involved in Code and test case reviews.
• Developed test cases using JUNIT for unit as well as integration testing.

Environment: Java, Servlets, JSP, Spring Framework, WebSphere, RAD, COBOL, DB2, AQT, WSDL, SOAP, iBatis, Log4j, Harvest

Confidential

Java/J2EE Developer

Responsibilities:

• Interaction with business team for requirements analysis and issue resolution.
• Involved in Design & Developing Object model and reporting solution.
• Designed use case diagrams, class diagrams and sequence diagrams using UML in Rational rose.
• Own the rapid development of the GUI, from specification through design, coding, debugging, and deployment.
• Convert approved design into functional CSS, HTML, Java,JSF Web pages.
• Developed charts, dashboards, and ad hoc analysis systems that leverage data from disparate data sources using Fusion Charts.
• Metric development to expose the metric in the UI and wrote Complex DB Queries to export metric data to excel sheet.
• Used Hibernate framework SQL mapper to generate the queries and insert the data in the data base.
• Unit testing and working extensively with the testers and business analyst to finalize the portal.
• Setting up role based user access privileges.

Environment: Java 1.5, JSF 1.x, IceFaces, JSP, Servlets, EJB 3.0, Hibernate, Fusion Charts, XML, LDAP, JBOSS, MySQL, SVN.