SUMMARY:

• Result Oriented Professional building on 13+ years of progressive experience in Software Development includes application architect, administration, design and development along with 4+ years in Big data/ Hadoop experience in Hadoop ecosystem such as Hive, Pig, Flume, Sqoop, Zookeeper, Hbase, SPARK, Kafka, MapReduce.
• Understanding to identify the viability of a business problem for a big data solution. Defining a logical architecture of the layers and components of a big data solution like data capacity planning and node forecasting. Selecting the right products to implement a big data solution.
• Familiar with data architecture including data ingestion pipeline design, Hadoop information architecture, data modeling and data mining, machine learning and advanced data processing. Experience optimizing ETL workflows.
• Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, SPARK, Cassandra and MongoDB and Flume.
• Experience in architecting, designing, installation, configuration and management of Apache Hadoop Clusters & Cloudera Hadoop Distribution
• Prior experience working as Software Developer in Java/J2EE and related technologies such as JSP, Servlets, Hibernate, JDBC.
• Designed and implemented Stream processing pipeline workflow for user which update user’s data nearly real - time
• Good hands-on experience on data visualization tools such as Tableau, QlikView and DataTorrent
• Experience in Data Analysis, Data Cleansing, Data Validation and Verification, Data Conversion, Data Migrations and Data Mining
• Experience working on LDAP user accounts and configuring ldap on client machines
• Experience in developing applications using Object oriented design and Development using Microsoft .Net technologies including ASP.NET, C#.Net, VB.NET, ADO.Net, WPF, WCF, Silverlight and XML for Web and Win Forms development.
• Experience in utilizing SQL integration Services (SSIS), SQL Reporting services (SSRS), SQL Management studio and SQL tools
• Highly resourceful in Project Management/Leading activities like estimation, planning, scope definition, resource administration, process compliance.

TECHNICAL SKILLS:

Technology/Platform/ToolsBig Data Platform: Hortonworks (HDP 2.2)/AWS (S3, EMR, EC2)/Cloudera (VDH3)

OLAP Concepts: Data warehousing, Data mining conceptsApache Hadoop Yarn 2.0HDFS, HBase, Pig, Hive, Sqoop, Kafka, Zookeeper, Oozie

Real Time Data Streaming: Apex, Malhar, Spark (Scala)

Source Control: GitHub, VSS, TFS

Databases and NoSQL: MS SQL Server 2012, Oracle 11g (PL/SQL) and MySQL 5.6, MongoDB

Data Visualization Tools: Tableau, Qlik1View and DataTorrent

Development Methodologies: Agile and Waterfall

Development Tool: Eclipse, Toad, Visual Studio

Programming Languages: Java, .Net

Scripting Languages: JavaScript, JSP, Python, XML, HTML and Bash

PROFESSIONAL EXPERIENCE:

Confidential

Sr. Hadoop Developer/Admin

Responsibilities:

• Designed and build data pipeline which stream data from client apps using web-sockets to server and from there Kafka Consumer which consumes that data and write to HDFS data store. From HDFS store different spark jobs are reading this data using Spark-SQL and processing this data in stream and batches jobs
• Shared responsibility for administration of Hadoop, Hive and Pig
• Managed, reviewed and interpreted Hadoop log files. Involved with the application teams to install Hadoop updates, patches and version upgrades as required
• Leadership: Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hive and Sqoop. Responsible for building scalable distributed data solutions using Hadoop
• Data Ingestion: Involved in importing and exporting data (SQL Server, Oracle, csv and text file) from local/external file system and RDBMS to HDFS. Load log data into HDFS using Flume
• ETL Data Cleansing, Integration & Transformation using Pig: Managing data from disparate sources.
• Exported analyzed data to the relational databases using Sqoop for visualization & Report generation
• Data Warehousing: Designed a data warehouse using Hive, created and managed Hive tables in Hadoop
• Workflow Management: Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig

Sr. Hadoop Developer/Admin

Responsibilities:

• Capacity Planning to setup Hadoop Cluster on AWS-EC2 for data in petabytes
• Setup local cluster with Hortonworks distribution for pilot run
• Developed MapReduce programs to parse the semi structured data, populate staging tables
• Used Sqoop to dump data from relational database into HDFS for processing and exporting data
• Used Pig (Pig Latin scripts) and Hive in the analysis of data
• Worked on Sequence and ORC files, bucketing, partitioning for Hive performance and storage improvement
• Used Oozie scheduler to automate the pipeline workflow and orchestrate the map reduce jobs that Extract the data on a timely manner
• Created Design documents, Architectural Documents and Technical documents for POC
• Managed and reviewed Hadoop log files to identify issues when job fails

Technical Manager/ Architect

Responsibilities:

• Developing parser and loader MR application to retrieve data from HDFS and store to HBase and Hive
• Built ETL workflow to process sales and marketing data on hive tables
• Importing the data from the MySql and Oracle into the HDFS using Sqoop
• Experienced on loading and transforming of large sets of structured and unstructured data
• Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS
• Created Hive Internal and External tables and loaded the data in to tables and query data using HQL
• Used Hue for UI based PIG script execution, Oozie scheduling and creating tables in Hive
• Deployment of applications using AWS EC2
• Written Map Reduce java programs to analyze the log data for large-scale data sets
• Participated in building CDH4 test cluster for implementing Kerberos authentication. Installing Cloudera manager and Hue
• Work within the enterprise architecture team and designed system using SOA architecture
• Worked with development team in analyzing data obtained from production systems and massaging them into readable format using MS SQL Server SSIS and Analytical Service (SSAS)
• Designing Applications Architecture, HLD, LLD and Test plan and other architectural documents
• Database designing and Written T-SQL, SQL queries, stored procedures, functions, triggers etc.
• Client Management and Delivery Management under cost, quality and schedule.
• Project Management activities like planning, resource allocations, tracking and people management
• Created use case, class, package, sequence diagrams using MS Visio.
• Performing User Acceptance test & Analyzing the Data and Defect fixing

Project Lead

Responsibilities:

• Application framework designing and development including database designing.
• Designed and developed data pipeline using SQL Server 2010, SSIS, SSAS and related technologies.
• Configuration Management, Code Review, Build Deployment Management and Releases.
• Provided data modeling and database design using ER-Studio and Enterprise Architect.
• Involved in various phases of Software Development Life Cycle (SDLC) of the application
• Developed user interfaces using JSP, HTML, XML and JavaScript.
• Used JavaScript for client side validation and created external style sheets using CSS.
• Web application development using J2EE: JSP, Servlets, JDBC, Hibernate, JUnit and Apache Log4J, Web Services, Message Queue (MQ).
• Involved in the preparing the Unit test plan documents
• Support to setup process implementation for ISO 9001, ISMS 27001 process documentations.
• Implemented security in modules as Identity, Authentication, and Authorization

Lead Analyst

Responsibilities:

• Application designing and development including database modeling.
• Export or Import data from other data sources like flat files using Import/Export of DTS
• Requirement Analysis, design, development and implementations.
• Broadly involved in Data Extraction, Transformation and Loading (ETL process) from Source to target systems using SSIS
• Developed an API to write XML documents from a database. Utilized XML and XSL Transformation for dynamic web-content and database connectivity.
• Used ASP.Net web application in LINQ to SQL for database connectivity.
• Developed the necessary Stored Procedures and created Complex Views using Joins for robust and fast retrieval of data in SQL Server using PL/SQL.
• Experienced in designing report layouts in SSRS and deployed cubes using SSAS
• Worked with Team Foundation Source (TFS) control which stores all code, as well as a record of all changes and current check-outs in SQL Server database.
• Coordinated with technical team for production deployment of software applications for maintenance

Sr. Software Engineer

Responsibilities:

• Application framework designing support and development including database designing.
• Documentation designs like HLD, LLD and Presentations.
• Designed the complete solution using N-tier Architecture model and design patterns mainly Abstract Factory and Singleton
• Assisted in creation of ETL processes for data transformation sources from SQL database and Legacy systems
• Created entire project in Subversion and created Ant build script for compiling and building the application for various environments
• Created Use case, Sequence diagrams, functional specifications and User Interface diagrams using Star UML
• Implemented Model-View-Control (MVC) software architecture in web applications to view the html.
• Used List, Trees, Toolbars, Menus and Context Menus for navigating between pages in WPF.
• Developed Windows based GUI using WPF, Expression Blend and done data binding using one way, two ways and one way to source data binding.
• Configured Windows Communication Foundation (WCF) service to authenticate clients with Windows credentials for intranet applications for login validations

Software Engineer

Responsibilities:

• Module Testing, Writing Test-case’s, Implementation and Client Training.
• Involved in Analyzing User Requirements, Design, Development and Implementation and in preparation of the Software Design Document
• Designed User Interface and Implemented the Application Logic under Microsoft .NET Framework 1.0 using ASP.NET and VB.NET to use .NET features that powered with Common Language Runtime. Served as Application Management, Build Management and Admin Management modules
• Produced User Controls for Common Header's in the ASP.NET web pages.
• Used Microsoft Visio 2003 to create Use Case, ER-Diagrams and Class Diagrams.
• Crystal reports on web platform from databases to generate reports based on viewer customization
• Created Web Services used for the application as well as for some other departments to Reusable application components
• Developed graphical charts in the web application using Office Web Components tool
• Involved in System Testing and Bug Fixing