SUMMARY:

• 11+ years of outstanding development experience with recent 4 years of expertise in Big Data and Hadoop technologies. Proficient in data analysis, data mining and data landing techniques. Extensive data engineering experience with sound knowledge on Hadoop ecosystems such as HDFS, MapReduce, YARN, Hive, Pig, Sqoop, Spark, Python, Parquet, Flume, Kafka, Oozie, Zookeeper, AWS and CDH(Cloudera Distribution for Hadoop).
• 4 years of design and development experience on Hadoop ecosystem (MapReduce(YARN), HDFS,HIVE,Sqoop,Oozie,Flume,Kafka,Zookeeper,HBase and Spark).
• Worked in analyzing data using Hive QL and custom MapReduce programs in Python and Java.
• Skilled in performance tuning of Hive QL.
• Proficient in data analysis using Python and Spark SQL in Spark framework.
• Proven ability to excel in fast paced development environment using latest framework/tools(SPARK with PYTHON - PYSPARK)
• Experienced in data landing techniques to importing and exporting data using SQOOP from RDBMS (SQL Server and MySQL) to HDFS and vise-versa.
• Worked on Job/workflow scheduling and monitoring tools like Oozie and Zookeeper.
• Handled designing both time driven and data driven automated workflows using Oozie.
• Worked on Hadoop clusters using major Hadoop Distributions — Cloudera (CDH4 and CDH5) deployed in AWS EC2.
• Developed and consumed REST APIs in Java applications using spring boot framework.
• .Net Programmer with 7 years of extensive programming experience in developing web based applications and client-server technologies using C#, ASP.Net, SQL Server, Javascript, Python and Oracle 9i/11g, PL/SQL.
• Sound knowledge on logical and physical Data Modeling using normalizing techniques.
• Developed Complex database objects like Stored Procedures, Functions, Packages and Triggers using SQL and PL/SQL.
• Experienced in creating, optimizing and troubleshooting ETL jobs using Microsoft SSIS packages.
• Proficient in development methodologies such as Agile and Waterfall.
• Worked on high availability and high traffic applications.
• Expertise to deep dive into technical holes and coming out with solutions.
• Excellent in handling Production Support issues and providing permanent fixes within SLA.
• Implemented data integration techniques on traditional and Hadoop environments.
• Worked with end users to formulate and document business requirements.
• Responsiblefor designing and implementing strategies, architectures, ingestion, storage,consumption and delivery processes for complex, large-volume, multi-variate,batch and real time data sets used for modeling and data mining.
• Strong problem solving & understanding of data modeling concepts
• Design and development of server jobs to extract data into SQL SERVER and flat files.
• Involved in providing technical design review, development plan review, code review, test plans, and results as per Microsoft and Apache recommendations.
• Root cause analysis, fix and documentation for the production defects identified as part of stabilization
• Developed Metrics, Filters, Prompts to achieve the functionality of the reports and created metrics.
• Worked closely with all aspects of the Business Intelligence group in order to deliver the required solutions in a timely manner.
• Worked with end users along with Business Analysts to build reports and dynamic dashboards.
• Worked with the DBA on issues of database performance and security.
• Involved extensively in the overall lifecycle of the Data Warehouse integration project.

TECHNICAL SKILLS:

Big Data Ecosystems: Hadoop, MapReduce, HDFS, HBase, Hive, Pig, Sqoop, Oozie, Zookeeper, Flume,Kafka,Spark.

Scripting Language: JavaScript, JQuery XML, HTML.

Programming Languages/Tools: C#, VB .Net, ASP .Net, Python, Java.

Operating System: Windows 7,8,10, Server 2012,2016, UNIX, Linux and Ubuntu.

IaaS: AWS EC2

Databases: RDBMS Oracle 9i/10g, MS SQL Server, MS Access, MySql and HBase.

Browser Ianguage: HTML5, CSS3

Methodologies: Agile,Waterfall

Repository Tools: SVN, Team Foundation Server, Git.

PROFESSIONAL EXPERIENCE:

Confidential

Sr. Hadoop Developer

Responsibilities:

• Gathered requirements and developed Proof of Concept for a project proposal as customer was analyzing with tool performing slowly.
• Worked on analyzing Hadoop cluster and different big data analytic tools including Hive, Spark, Python, Sqoop, flume, Oozie.
• Involved in importing and exporting data (SQL Server,XML, csv and text file) from local and external file system and RDBMS to HDFS.
• Worked extensively with HIVE DDLs and Hive Query language (HQL)
• Designed a data warehouse using Hive, created and managed Hive tables in Hadoop.
• Involved in creating Python map reduce jobs, UDFs and HIVE queries by following best coding standards.
• Implemented SCD (Slowly Changing Dimension) concepts in Hive.
• ETL Data Cleansing, Integration &Transformation using Hive and PySpark. Responsible of managing data from disparate sources.
• Tuned performance in Hive using Partitions, Bucketing, Indexes and Parallism concepts
• Avoided MapReduce by using PySpark for boosting performance to 3x times.
• Solved performance issues in Hive with understanding of joins, Groups, and aggregation and how does it translate to MapReduce jobs.
• Worked on RDD and dataframe techniques in PySpark for processing data at a faster rate.
• Loaded stream data into HDFS using Flume, Kafka and Spark Streaming.
• Lead developer in the project of Datalanding. Imported data from critical applications to HDFS for data analysis.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the R&D team
• Designed and Developed jobs that handles the Initial load and the Incremental load automatically using Oozie workflow.
• Developed and consumed REST APIs in Java applications using spring boot framework.
• Adequate knowledge and working experience with Agile methodology.
• Worked in setting up Hadoop on Pseudo distributed environment
• Experienced in setting up Spark, Python and Jupyter notebook on Ubuntu Operating system.
• Involved in unit testing activities and test data preparation for various business requirements.
• Worked on the CDH upgrades in AWS EC2 environment along with admin team and did the regression testing.
• Replaced the existing data analysis tool with Hadoop.
• Sound working knowledge of HBase and NoSQL DB concepts.
• Moved between agile and waterfall approaches depending on project specifics and client goals, creating detailed project road maps, plans, schedules and work breakdown structures.
• Created and maintained Technical documentation for launching Hadoop Clusters and for executing Hive queries.
• Worked on ETL reports using .Net reporting application and created statistics dashboards for business users.
• Used MS Team Foundation Service for project tracking, bug tracking and project management.
• Involved in Scrum calls, Grooming and Demo meeting.

Confidential

Hadoop Developer

Responsibilities:

• Datalanding from the different Data sources like (SQL server, CSV and text files) into HDFS using Sqoop and load into Hive tables.
• Developed Hive queries for analysis across different banners.
• Developed Hive UDFs in Python to bring all the customers Email Id into a structured format.
• Developed Oozie Workflows for daily incremental loads, which gets data from external applications and then imported into hive tables.
• Exported analyzed data from HDFS to RDBMS using Sqoop.
• Worked as a lead developer in projects involving SQL server and .Net technologies.
• Involved in development of new enhancements in MVC architechture using entity framework.
• Responsible to create new WCF web services using ASP.Net with C#.
• Used the AJAX tools like UpdatePanel etc. for partial post back.
• Involved in Database diagram design and Tables design in SQL Server.
• Support/modify the exiting web services as per user requirements.
• Troubleshooting and bug fixing by identifying the root cause of problem.
• Developed and implemented SSIS packages for ETL jobs to import and transform data.
• Created and deployed SSIS packages to production, schedule them to run on routine basis and monitor error log for errors.
• Managing the SQL server databases and tuning performance.
• Provide Support to other teams to fetch the data from the centralized database using ASP.net XML Web Services.
• Manage the Dev, QA, Staging and Production environment.
• Worked in a collaborative development environment, providing assistance & technical guidance to fellow team members.
• Prepare technical design documents, perform code reviews, debugging and testing
• Provide inputs in architecting, designing, and implementing a Service Oriented Architecture (SOA) for the application development group
• Involved in Web Service testing using soapUI tool.
• Created utility tools for estimation templates.

Confidential

Lead .Net & SQL Developer

Responsibilities:

• Developed .NET library project using C# and provides middle layer components with HTTP Module and will be distributed to all users via MSI package.
• Created different business classes and interfaces using .Net Framework 4.5 with Object Oriented Programming Methodology.
• Designing and developing the web forms using ASP.Net 4.5.
• Used CSS (Cascading Style Sheets) in ASP.Net pages.
• Created User Controls and Custom Controls for common purpose in the whole application.
• Used ASP.Net telerik Kendo Controls for like gridview, autocompete etc.
• Used XML for request and response from the WCF Services and manipulated the result as accordingly.
• Created the Custom User Grid View Control which includes the Sorting and Paging.
• Used Dataset, Data View and Data Adapter to manipulate and display data.
• Involved in relational database diagram design and Tables design in SQL Server.
• Created Stored Procedures, Views, Triggers and Complex T-SQL queries in SQL Server.
• Involved in Unit testing and Bug fixing.
• Involved in the Build and Release activities.
• Involve in creating the read only API’s application for other applications.
• Ingenuity around creation of reusable objects and engines
• Facilitated problem solving and collaboration
• Requirement analysis and understanding
• Coordinated with offshore for day to day activities and assured quality deliverables to customer
• Prepared high-level and detailed design documents
• Designed and developed ETL jobs that handled the Initial load and the Incremental load

Confidential

.Net & SQL Developer

Responsibilities:

• Involved in Analysis, Designing and coding and implementation of the application.
• Designed and developed the web forms using ASP.Net 2.0.
• Design and develop front ends for the application to support business rules using C#.
• Development of component classes and interfaces in C# encapsulating the business logic with strict adherence to Object Oriented Paradigms.
• Used CSS (Cascading Style Sheets) in ASP.Net pages.
• Created the own Controls library with Infragistics Controls.
• Created User Controls and Custom Controls for common purpose in the whole application.
• Responsible for implementing and deployment of Web Services.
• Designed and developed the complete front-end application using ASP.Net with C# as code behind for the internal processing.
• Developed and hosting Web Services on IIS by following Service Oriented Architecture (SOA).
• Implemented the functionality to render data using XML files and format them using XSLT.
• Involved in Database diagram design and Tables design in SQL Server.
• Created Stored Procedures, Views, Triggers and Complex T-SQL queries in SQL Server.
• Involved in the development and execution of unit test cases.
• Created and deployed the Windows Service for Emails.
• Implemented the Email utility using windows service.
• Used Team Foundation Server (TFS) for project repository and versioning control.
• Prepared high level and low level system design diagrams (class diagrams, sequence diagrams).
• Effectively communicated with application representatives from both IT and Business to resolve issues.