SUMMARY

• 5 + years of total experience as Software Engineer wif IT Technologies and good working knowledge in Java and BIG Data Hadoop Ecosystems. Microsoft Certificate in Data Science professional wif hands on experience in all phases of diverse technology projects specializing in Data Science & Analytics, Data Warehousing, Azure Machine Learning and Tableau.
• Strong experience in Core Java programming using Collections, Generics, Exception handling, multithreading.
• Hands - on experience in developing applications using advanced Java technologies like JSP, Servlets, JSTL, JDBC, AngularJS, XML, Ajax, JavaScript, JQuery.
• Extensive experience in installing, configuring and administrating Hadoop cluster for major Hadoop distributions like CDH5 and HDP.
• Hands-on experience on major components in Hadoop Ecosystem including Hive, HBase, HBase & Hive Integration, Sqoop, Flume & knowledge of Mapper/Reduce/HDFS Framework.
• Set up standards and processes for Hadoop based application design and implementation.
• Worked on NoSQL databases including HBase, Cassandra and MongoDB.
• Experience onHorton worksandCloudera Hadoopenvironments.
• Good experience in analysis using Pig and Hive and understanding of SQOOP and Puppet.
• Expertise in database performance tuning data modeling.
• Strong knowledge on creating and monitoring Hadoop cluster on VM, Hortonworks Data Platform 2.1 7 2.2, CDH3, CDH4 Cloudera Manager on Linux, Ubuntu OS.
• Good hands-on experience in PySpark.
• Experienced in providing security to Hadoop cluster wifKerberosand integration wif LDAP/AD at Enterprise level.
• Knowledge on MS SQL Server 2012/2008/2005 and Oracle 11g/10g/9i.
• Involved in best practices forCassandra, migrating application toCassandradatabase from the legacy platform for Choice, upgradedCassandrafrom 2.0 to 2.2.5.
• Experienced in developing MapReduce programs using Apache Hadoop for working wif Big Data.
• Good understanding of XML methodologies (XML, XSL, XSD) including Web Services and SOAP.
• Used the Spark -CassandraConnector to load data to and fromCassandra.
• Experience in creatingdatabases, users, tables, triggers,macros, views, stored procedures, functions, Packages, joins and hash indexes in Teradata database.
• Experience in working wif Teradata utilities likeBTEQ, Fast Export, Fast Load, Multi Loadto export and load data to/from different source systems including flat files.
• Strong knowledge in Software Development Life Cycle (SDLC).
• Strong understanding in Agile and Waterfall SDLC methodologies.
• Involved in log file management where the logs greater TEMPthan 7 days old were removed from log folder and loaded into HDFS and stored for 3 months.
• Experienced in installing, configuring and administrating Hadoop Clusters.

TECHNICAL SKILLS:

Big Data: Hadoop, HDFS, Map Reduce, Hive, Sqoop, Pig, Ambari, HBase, MongoDB, Cassandra, Spark, Flume, Impala, Kafka, Oozie, Zookeeper, Cloudera Manager

Hadoop Distribution: Cloudera, Horton Work, AWS

Analysis Tools: DataRobot, Trifacta

Project Management: MS-Project

Programming & Scripting Languages: Java, SQL, PL-SQL, JavaScript, Scala, Unix Shell Scripting, C, Python, R programming

Reporting Tools: Qlik View, Qlik Sense, Tabule, SOAP UI

IDE/GUI: Eclipse, IntelliJ IDEA, Net Beans, Visual Studio.

Database: MS-SQL, Oracle Database, MS-Access, AWS, Teradata, Mongo DB, Cassandra

Other Tools: Wireshark, Cisco Packet Tracer

Operating Systems: Window10, Windows 8, Windows 7, Windows Server2008/2003, Mac OS, Ubuntu, Red Hat Linux, Linux, UNIX

PROFESSIONAL EXPERIENCE:

Confidential, Boise

Big Data / Hadoop Developer

Responsibilities:

• Use ApacheSqoop to automatically load relational data from MySQL intoHDFS, while preserving the structure.
• Take advantage of Hive's flexible SerDes (serializers / deserializers) to parse the logs into individual fields using a regular expression & correlating two data sets.
• Indexing data to be searchable to a variety of audiences using the MapReduce Indexing.
• Storing and loading the data from HDFS to Amazon S3 and backing up the Namespace data.
• Setup and monitored Cloudera CDH5 cluster wif Hadoop2/YARNrunning to read data from the Cluster
• Working wif data delivery teams to setup new Hadoop users. dis job includes setting up Linux users, setting upKerberosTEMPprincipals and testing HDFS, Hive.
• Involved in collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
• Reviewed basic SQL queries and edited inner, left, and right joins inQlik ViewDesktop by connecting live/dynamic and static datasets
• Used Ganglia to Monitor and Nagios to send alerts about the cluster around the clock.
• Involved in creating Hadoop streaming jobs using Python.
• Created concurrent access for hive tables wif shared and exclusive locking dat can be enabled in hive wif the help of Zookeeper implementation in the cluster.
• Worked on various performance optimizations like using distributed cache for small datasets, partition and bucketing in hive, doing map side joins etc..
• Worked on importing and exporting data from Oracle and DB2 into HDFS and HIVE using Sqoop for analysis, visualization and to generate reports.
• Extensively worked on several ETLAbInitioassignments to extract, transform and load data into tables as part of Data Warehouse development wif high complex Data models of Relational, Star, and Snowflake schema.
• ImplementedPySpark andSparkSQL for faster testing and processing of data.
• Developed multiple MapReduce jobs in java for data cleaning.
• Expert in implementing advanced procedures like text analytics and processing using the in-memory computing capabilities like ApacheSparkwritten in Scala
• Ran many performance tests using theCassandra-stress tool in order to measure and improve the read and write performance of the cluster.
• Worked on migrating MapReduce programs intoPySpark transformation.
• Built wrapper shell scripts to hold Oozie workflow.
• ConfiguredAbInitioenvironment to connect to different databases using DB config, Input Table, Output Table, Update table Components.
• PerformedQlik Viewtype conversion functions when connected to relational data sources.
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs,Pythonand Scala.
• Provided ad-hoc queries and data metrics to the Business Users using Hive, Pig.
• Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.
• Used Pig as ETL tool to do transformations, event joins and some pre-aggregations before storing the data onto HDFS.
• Worked on MapReduce Joins in querying multiple semi-structured data as per analytic needs.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Used Hive and created Hive tables and involved in data loading and writing Hive UDFs.
• Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
• Familiarity wif NoSQL databases including HBase, MongoDB.
• Wrote shell scripts for rolling day-to-day processes and it is automated.
• Involved in story-driven agile development methodology and actively participated in daily scrum meetings.

Environment: Hadoop, Hadoop 2, MapReduce, Hive, HDFS, Cassandra, PIG, Python, Sqoop, Oozie, Cloudera, Flume, HBase, ZooKeeper, CDH5, MongoDB, Cassandra, Oracle, ETL, NoSQL and Unix/Linux, Kafka, Amazon web services.

Confidential, NY

Big Data / Hadoop Developer

Responsibilities:

• Deployed Hadoop Cluster in the following nodes.
• Managing and scheduling jobs on Hadoop cluster
• Manage Hadoop clusters: Monitor, maintain, setup.
• Involved in analyzing system failures, identifying root causes and recommended course of actions.
• Worked on Hive for exposing data for further analysis and for generating transforming files from different analytical formats to text files.
• Handled importing of data from various data sources, performed transformations using Hive, Pig andSparkand loaded data into HDFS.
• Wrote queries to create, alter, insert and delete elements from lists, sets and maps in Datastax Cassandra.
• Involved in NoSQL (DatastaxCassandra) database design, integration and implementation.
• Developed multiple MapReduce jobs in java for data cleaning and accessing.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Created indices for conditioned search in DatastaxCassandra.
• Working on generating various dashboards inQlik ViewServer using different data sources such as Teradata, Oracle, Microsoft SQL Server and Microsoft Analysis Services.
• Worked on the core andSparkSQL modules ofSparkextensively.
• Implemented NameNode backup using NFS. dis was done for High availability.
• Worked on importing and exporting data from Oracle and DB2 into HDFS using Sqoop.
• Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.
• Worked on custom Pig Loaders and Storage classes to work wif a variety of data formats such as JSON, Compressed CSV, etc.
• Monitored workload, job performance and capacity planning using Cloudera Manager.
• Created Hive External tables and loaded the data in to tables and query data using HQL.
• Wrote shell scripts to automate document indexing to SolrCloud in production.
• Used Flume to collect, aggregate, and store the web log data from different sources like web servers, mobile and network devices and pushed to HDFS.
• Worked for Amazon Elastic Cloud project using agile methodologies.
• Analyzed the web log data using the HiveQL to extract number of unique visitors per day, page-views, visit duration, most purchased product on website.
• Converting the Oracle table components to Teradata Table Components in Abilities Graphs.
• Used Ambary to manage, provision and monitor Hadoop cluster.
• Implemented Fair schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.

Environment: Hadoop 1x, Hive, Pig, HBASE, Sqoop, Flume, Zookeeper, Qlik View, Pig, HDFS, Ambary, Cassandra, Oracle, CDH4, HPD 2.2

Confidential

Big Data / Hadoop Developer

Responsibilities:

• Loaded the customer profiles data, customer usage information, billing information etc. onto HDFS using Flume.
• Installed and configured Apache Hadoop, Hive and Pig environment on the prototype server.
• Importing and exporting data into HDFS, Pig, Hive and HBase using SQOOP.
• Loaded and transformed large sets of structured, semi structured data using Pig Scripts.
• Extensive knowledge on PIG UDF’s to pre-process the data for analysis.
• Involved in loading data from UNIX file system to HDFS.
• Used Hive to analyze the unified historic data in HDFS to identify issues and behavioral patterns.
• Developed simple and complex MapReduce program in Java for Fata Analysis.
• The Hive tables created as per requirement were internal or external tables defined wif appropriate static and dynamic partitions, intended for efficiency.
• Optimized Hive queries to extract the customer information from HDFS or HBase.
• Developed Pig Latin scripts to aggregate the log files of the business clients.
• Developed the Pig UDF’s to pre-process the data for analysis.
• Involved in writing shell scripts in scheduling and automation of tasks.
• Managed and reviewed Hadoop log files to identify issues when job fails.
• Exported analyzed data to relational database using SQOOP foe visualization to generate reports for the BI team.
• Collaborated wif the infrastructure, network, database, application and BI teams to ensure data quality and availability.
• Developed a workflow using Oozie to automate the tasks of loading the data into HDFS from analyzing the data.

Environment: Hadoop (Gen 1), HDFS, Flume, MapReduce (MR1), Pig, Hive, Sqoop, Oozie, Java, Linux Shell, Cloudera, Scripting and Big Data.

Confidential

J2EE/UI Developer

Responsibilities:

• Interacted wif Business Users to capture business needs, functional needs, managing expectations, and reviewing deliverables to ensure a successful implementation.
• Used Grunt.JS and node.JS for Building Application.
• Involved in creating the various user interfaces (UI) using JSP, DHTML, JavaScript, and CSS.
• Developed all the UI pages using HTML5, DHTML, XSL/XSLT, XHTML, CSS3, JavaScript, jQuery, Ajax, Perl.
• Responsible for writing J2EE compliant code using Java for an application development effort. dis involves working wif JSP, Servlets, JDBC, XML, EJB etc.
• Used jQuery to traverse through a DOM tree and manipulated the nodes in the tree. Implemented the Drag and Drop functionality using jQuery framework.
• Experience wif Test Driven Development and Node.js.
• Fixed bugs, providing production support & enhanced the applications wif additional features.
• Involved in developing HTML, and JavaScript for client side presentation and, data validation on the client side wifin the forms.
• Enforced quality and validity of data affected by Java programs via database in core java.
• Worked wif core java by resolving data, application and connectivity issues related to Java programs' functionality.
• Maintained, developed and fixed bugs for applications.
• Documented and unit tested website code and applications.
• Responsible for setting up Angular JS framework for UI development. Developed html views wif HTML5, CSS3, JSON and AngularJS.
• Involved in Enhancement of existing application utilizing AngularJS, created HTML navigation menu dat is role based wherein menu items changes dynamically based on the values derived from the database in the form of XML.
• Used Bootstrap for responsive web design.

Environment: JavaScript, JQuery, HTML5, XML, & CSS3, AngularJS, Require.js, bootstrap, MVC, AJAX, XML, Eclipse, Internet Explorer, Chrome, PL/SQL, Windows.