SUMMARY:

• 9+ years of overall experience in the IT industry with 2+ years of experience in SPARK and SCALA
• Have worked extensively in the field of Data Management including Business Intelligence, Data warehousing, Master Data Management and Data Quality using various Big Data technologies and Traditional ETL and Database technologies.
• Have worked with clients in various Domains like Cloud Computing, Media and Entertainment, Telecom and Banking.
• Excellent experience across Analysis, Design, Development, Testing, Performance tuning and Production support of complex information systems
• Have worked extensively on designing complex Data processing, Data warehousing systems and designed complex ETL design to handle large volumes of data using tools like SSIS and Informatica.
• Have worked on Extracting, Batch Processing and Storing Big Data using technologies such as SPARK/SCALA, S3, EMR, Hadoop and Hive
• Have good experience in performing complex data transformations on Confidential cluster using Functional programming language like SCALA 2.11 and good knowledge on Python. Have developed complex functional programming in SCALA using LAMBDA’s, RDD, Data Frames, Data sets, Recursive functions, Maps, Tuples, Vector etc.,
• Outstanding technical skills in SQL Server 2005/2012/2016, SSAS, SSIS, Confidential
• Hands on experience in ETL Redesign and Migration from Informatica Power Center to SSIS. Hands on in Informatica Power Center Workflow Manager, Designer and Monitor 9.5.1
• Hands on experience in data conversion and migration from an on - premise SQL Database to the Cloud based AWS RedShift data warehouse and S3
• Experience in Dimensional modelling using Kimball EDW Model tool. Have the experience of working on both Bill Inmon and Ralph Kimball models of data warehousing
• Experienced in working with SQL Query Analyzer, SQL Server Management Studio and SQL Server Data tools , scheduling jobs with SQL Server Agent, SQL Server Storage Structures, Index and SQL Profiler
• Hands on experience in designing dashboards using Power BI tools like Power View and Power Pivot. Expertise in creating Ad-hoc, Confidential Parameterized and Parameterized reports involving Confidential, Drill through, Chart and Sub reports using Confidential
• Experience in SharePoint 2013 BI Integration for the Power Pivot and Power View reports.
• Extensive experience in designing Multidimensional SSAS cubes using star and snow flake schema, identifying the dimensions, possible hierarchies, data granularity for the reports. Experience in handling Late arriving dimensions & facts, designing Partitions, Aggregations, Many to Many relationships, MDX queries, Calculated Members and KPI’s
• Significant Expertise in Requirement Gathering (Business and Technical document), Support Integration Testing and UAT, Source to Target mapping, OLTP, OLAP, Incremental/Delta Data Load, Full Load, Confidential and Scorecards, SQL Server Complex Store procedures, Indexing, Linked Servers, dynamic open queries, Query Performance Turning and Error handling in SQL Server, ETL and BI reporting
• Expertise in developing complex SSIS ETL packages using the various Control Flow components like Data flow task, Execute SQL Task, Foreach Container, Sequence containers and various data flow components like Conditional split, Merge Join, Union, Look up, Fuzzy Look up, Row Count, Conversion
• Experience in providing Logging, Error handling by using Event Handler, and Custom Logging for SSIS Packages
• Implemented Custom SSIS component in dataflow & deployed to various environments. Experience in Performance Tuning in SSIS packages by using Row Transformations, Block and Unblock Transformations
• Working knowledge of .Net Framework 3.5 (VB.NET and C#.NET), IIS Web Server
• Proven experience in customer-facing situations. Quick learner and excellent team player having ability to meet tight deadlines, willing to take initiatives and take on new responsibilities.
• Exceptional versatility and adaptability, extremely diligent having excellent communication skills and ability to work both as individual and as Confidential team member.

TECHNICAL SKILLS:

Databases: AWS RedShift, SQL Server 2012, Hive, HBase, MongoDB, Postgres, Oracle 9i, MYSQL, DB2, MS Access

Data Storage: AWS S3, HDFS

Big Data: SPARK 2.1, SCALA 2.11, Hadoop, HDFS, Hive

ETL Tools: SSIS, Informatica Power Center 9.0

Reporting Tools: Qlik view, SQL Server Reporting Services

Analytical Tools: SSAS

Languages: SCALA 2.11, Python, T-SQL, PL/SQL, C#, VB.Net, VBA

Architecture: Relational DBMS, OLTP, OLAP

Data Services: Master Data Services 2012(MDS), Confidential 2012(DQS)

Tools: IntelliJ, Eclipse, SBT, GitHub, SNOW, Aginity For RedShift, Cloud Berry, SSDT, SSMS, Toad, BIDS, TFS

Operating System: Windows 2003/2012, Linux

PROFESSIONAL EXPERIENCE:

Confidential, San Antonio, Texas

Big Data ETL Developer

ETL Lead Developer/ ETL Developer/ Senior Analyst

Responsibilities:

• Actively worked with the Business Analysts and Data Architects to understand new Data processing and data storage requirements in Confidential Team
• Worked in Agile model with Confidential 3 Week sprint on both Kanban and Scrum modes to develop and deploy various Big Data solutions in Hortonworks data platform and AWS Cloud
• Have developed complex Data warehouse structures in Hive for various modules like Global Sales, VM Billing, Avaya, LBAAS, DBBAS, Location Manager, IP Commander etc.,
• Implemented Hive solution using various features like Managed tables, Static and Dynamic Partitions, Buckets, Indexes, Joins. Implemented SCD Type 1 and SCD Type 2 in Hive tables.
• Implemented batch data processing using SPARK 2.1 using SCALA 2.12 for processing huge volumes of Sales data and Polling data in Batch mode
• Have handled large volumes of data to the tune of 100’s of GB’s of data from AWS S3 into the EMR clusters and performed complex data transformations like aggregating the Sales amounts by using various features of SPARK/SCALA like RDD’s, Data Frames, Broad Cast variables, Mappers, Reducers, various collections such as Maps, Tuples, Vectors etc., to process the huge volumes of data with in defined SLA
• The processed data was loaded to Hive data warehouse and other Relational databases both ODS and data warehouse for the end users to consume
• Implemented job scheduling using oozie to schedule and automatically trigger various jobs in the Hadoop environment.
• Worked on designing Dimensions, Facts and Reporting tables in Datamart for the Global Sales Dataset project that involves entities like Opportunities, Accounts, Customers etc.,
• Worked on maintaining the GitHub repository for storing and maintaining various source codes like SPARL/SCALA code, Hive scripts, SQL scripts etc.,
• Created Technical Design document, UAT and handover documents on the various projects like GSD, ETL Migrations and UKBI Consolidation. Closely worked with the Quality Engineering Team for Testing various solutions.

Environment: Hortonworks Data Platform, AWS EMR Clusters, AWS S3, Hive 2.1.1, SPARK 2.1/SCALA 2.12, HADOOP 2.7, MS SQL Server 2012, Service Now, GitHub, JIRA.

Confidential, New York City

Responsibilities:

• Migrating the existing BI system to the new AWS cloud environment
• Involved in data conversion activity which involved converting the data from existing Media planning application and exporting the same to the new Wide Confidential
• Understanding the new Wide Orbit Media planning system and building the data mapping and extraction queries for the new Confidential
• Actively worked with the BA’s to understand the existing systems and limitations, user requirements.
• Analysis of existing Competitor BI domain to be able to suggest improvement for the new design
• Identifying and designing Dimensions and Facts.
• Created Constraints, Indexes, Stored Procedures, and User Defined Functions and maintained Primary/Foreign Key to enforce data and referential integrity
• Created User Defined Functions to encapsulate frequently and commonly used business logic making the system more modular, secured and extensible. Extensively used joins and sub queries to simplify complex queries involving multiple tables
• Dimensional modelling for the new Competitor BI Data Mart using the Kimball EDW data model for the Dimensions and Facts by following industry best practice
• Designing the Meta data driven ETL solution using config tables in SQL Server database.
• This involved defining source system, source query, target table, S3 buckets, SSIS packages, Scheduling and Audit.
• Worked on technical design document for the new Competitor BI Data Mart
• Development of SSIS packages for extracting the data from heterogeneous sources like flat file from FTP server to the new AWS S3 buckets using AWS SDK’s, C# Script task with the S3 Copy Commands
• Worked on Migrating the Informatica ETL to SSIS
• Development of SSIS package for importing the data from the S3 buckets into the new AWS Confidential with meta data driven solution using the AWS SDK’s and AWS RedShift Import command
• Extensively worked on Control Flow tasks like For Each Loop Container, Sequence Container, DFT, Execute SQL Task, Script Task and DFTs like Data conversion, Derived Column in SSIS Designer.
• Implemented the concepts of Historical load, Incremental Data Load, Slowly Changing Dimensions Type 1 for updating the dimensions.
• Analysis of the existing SSAS Multidimensional cube, identifying design dependencies, flaws, limitations and suggesting new design changes to address limitations and improve performance.
• Design of new SSAS Multidimensional cubes using SSDT, creating data source views, dimensions, facts, aggregations, partitions and dimension usage. Creating calculated measures using MDX, creating KPI’s, perspectives, roles and security.
• Design and Implementing of Linked Cubes which links Competitor BI cube with Confidential cube to bring to keep in frequently changing data from cube processing.
• Implemented Power view and Power Pivot for the visualization of reports.

Environment: MS SQL Server 2014/2016, Power BI, AWS RedShift, AWS S3, SSAS, SSIS, SSDT 2013, Visual Studio 2008, Excel, SharePoint, T-SQL, Windows 7, SVN, Informatica Power center 9.

Confidential

Responsibilities:

• Lead the Finance domain with team of 5 developers in ensuring smooth delivery of Monthly and Quarterly deliverables related to the Customer Information Data warehouse(CIW) and Operational Data Source(ODS)
• Involved in the discussions of new requirements with the users and BA’s.
• Requirement Analysis, solution feasibility and accordingly estimating the work items using the estimator tool
• Working with various third party vendors like Net Cracker, Scorecard Team on data extraction and integration
• Worked independently in delivering complex work items and have also involved in monitoring and helping the team of developers in ensuring the timely completion of the Monthly and Quarterly deliverables
• Coordinating with the Testing team in execution of various test cases related to the deliverables.
• Have involved with the End Users and the BA’s in ensuring the timely completion of the UAT testing
• Confidential business requirements into working logical and physical data models.
• Have worked on developing Confidential custom SSIS transformation for the Address Parsing and Cleansing by utilizing the API’s provided by the SAP and with the C# assembly that utilized the API’s to output the parsed Addresses.
• Developed SSIS Packages for integrating data using OLE DB connection from heterogeneous sources like (Excel, CSV, Oracle, flat file, Text Format Data) by using multiple transformations.
• Developed SSIS Packages using Derived Columns, Condition Split, Aggregate, Execute SQL Task, Data Flow Task, Execute Package Task etc. to transform and load the data into the CIW data warehouse.
• Developed Complex ETL Packages using SSIS to extract data from staging tables to partitioned tables with incremental load.
• Handled Performance Tuning and Optimization on SSIS, Confidential and database objects, with strong analytical and troubleshooting skills for quick issue resolution in large-scale production environments.
• Developed ordinary to complex queries involving correlated sub queries, CTE’s, Unions, multi table joins and XML techniques for different business requirement.
• Involved in monitoring existing scheduled jobs, time-sensitive bug fixes and applying enhancements requested by users.
• Created alerts and notifications on Job errors, system errors, insufficient resources, and fatal database errors.
• Maintaining and supporting the existing scheduled jobs, time-sensitive bug fixes and applying enhancements requested by users.
• Managed Offshore and Onshore co-ordination.

Environment: MS SQL Server 2012, SSIS, Confidential, .Net Framework 3.5, C#, Business Intelligence Development Studio, T-SQL, Windows Server 2005/2008, SVN.

Confidential

Responsibilities:

• Worked as Confidential Senior Microsoft BI Developer
• Involved in the Design and Implementation of Confidential new 3 rd normal form Teradata industry standard Enterprise level Data Warehouse in SQL 2012 using Bill Inmon model
• Worked on the design of Confidential new Data mart for various subject areas like Customer, Sales, Party, Account built on top of Data warehouse
• Was involved in the implementation of Confidential new SSAS Tabular solution with DAX but was re-solution using SSAS Multidimensional 2012
• Worked on the development of Confidential reports using various features like Confidential, Confidential, drill through, parameterized and data driven subscriptions
• Implemented geographical templates using Power View and Power Pivot models for the end users
• Worked on the AGILE delivery model with Confidential 2 Week sprint
• Participated in daily stand up calls to present the development status and participated in the daily poker meetings to discuss the user stories and estimates the same
• Implemented full load, Incremental load, Slowly Changing Dimensions SCD type 2 for preserving the old and new data.
• Extensively involved in development, unit testing of ETL using SSIS 2012 using Fuzzy Lookup, Derived Columns, Data Conversion, Merge Join, Condition Split, Execute SQL Task, Data Flow Task, For each loop and Execute Package Task using industry standard best practices by comparing different approaches
• Worked on designing the Master Data Model for the Customer domain.
• Developed SSIS packages for updating and adding new Customer data into MDS and Excel was used as the tool for the updates made by the end users.
• Worked on Confidential (DQS 2012) for cleaning the customer data by using SSIS DQS Transformations.
• Extensively used Joins and sub-queries for complex queries involving multiple tables using T-Sql. Optimized the database by creating various clustered, non-clustered indexes and indexed views
• Developed Stored Procedure and triggers to facilitate consistent data entry into the enterprise data warehouse.
• Worked on designing Partitions, Aggregations and created calculated Measures using MDX, KPI’s, Roles, Perspectives, Security.

Environment: MS SQL Server 2012, SSAS, SSIS, Confidential, Power BI - Power View, Power Pivot, .Net Framework 3.5, C#, Business Intelligence Development Studio, T-SQL, Windows Server 2012, Team foundation Server TFS.

Confidential

Responsibilities:

• Worked as Confidential Microsoft BI and VBA Developer
• Involved in the Monitoring, Support and Troubleshooting of the various SQL Agent and MS Access jobs
• Developed reports for the various teams like Credit Reporting team, Terms and Conditions team, Document Capture Management Systems, Standard Letter Systems SLS team, Retention Teams
• Worked on migrating the MS Access database solutions into the SQL Server 2005 with SSIS and Confidential
• Created packages in SSIS with error handling and worked with different methods of logging in SSIS
• Using Integration Services (SSIS), built high performance Data Integration Solutions, including extraction, transformation and loading (ETL) packages for data warehousing
• Worked on the development of Confidential reports using various features like Confidential, Confidential, drill through, parameterized and data driven subscriptions.
• Deploying and scheduling Reports using Confidential to generate all daily, weekly, monthly and quarterly Reports
• Participated in gathering Business rules and requirements, Prepared Technical Specifications and Documentation
• Writing stored procedures, triggers, cursors, and views to improve the quality of application, and manage mission-critical information using T-SQL
• Fine-tuned and checked the performance of existing objects in SQL Server and implemented transactional logic for them
• Scheduled and monitored all maintenance activities of SQL Server 2000/2005 including database consistency checks and index defragmentation

Environment: Windows Server 2003, SQL Server 2005, DTS, SSIS, Confidential, MS Access 2003, VBA, VSS, StarTeam.