SUMMARY

• 8 years of experience as Data Engineer and Data Analyst including designing, developing and implementation of data models for enterprise - level applications and systems.
• Experience in designing the Conceptual, Logical and Physical data modeling using Erwin and E/R Studio Data modeling tools.
• Extensive experience in development of T-SQL, Oracle PL/SQL Scripts, Stored Procedures and Triggers for business logic implementation.
• Expertise in SQL Server Analysis Services (SSAS) and SQL Server Reporting Services (SSRS) tools.
• Expertise in synthesizing Machine learning, Predictive Analytics and Big data technologies into integrated solutions.
• Involve in writing SQL queries, PL/SQL programming and created new packages and procedures and modified and tuned existing procedure and queries.
• Hands on experience in ELK (Elasticsearch, Logstash, and Kibana) stack.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems (RDBMS) and from RDBMS to HDFS.
• Experience in designing, building and implementing complete Hadoop ecosystem comprising of MapReduce, HDFS, Hive, and MongoDB.
• Experience in various Teradata utilities like Fastload, Multiload, BTEQ, and Teradata SQL Assistant.
• Extensively worked wif Teradata utilities BTEQ, Fast Export and Multi Load to export and load data to/from different source systems including flat files.
• Design, build and manage the ELK (Elasticsearch, Logstash, and Kibana) cluster for ... SQL/NoSQL Databases: Oracle, Teradata, MongoDB, Dynamo DB.
• Good in System analysis, ER Dimensional Modeling, Database design and implementing RDBMS specific features.
• Experience in Worked on NoSQL databases - HBase, Cassandra & MongoDB, database performance tuning & data modeling.
• Experience in designing the Conceptual, Logical and Physical data modeling using Erwin and E/R Studio Data modeling tools.
• Good in System analysis, ER Dimensional Modeling, Database design and implementing RDBMS specific features.
• Experience in Worked on NoSQL databases - HBase, Cassandra & MongoDB, database performance tuning & data modeling.
• Strong experience in writing SQL and PL/SQL, Transact SQL programs for Stored Procedures, Triggers and Functions.
• Expertise in designing complex Mappings and has expertise in performance tuning and slowly-changing Dimension Tables and Fact tables.
• Experienced in Data Scrubbing/Cleansing, Data Quality, Data Mapping, Data Profiling, Data Validation in ETL

TECHNICAL SKILLS

Tools: & Software: TOAD, MS Office, BTEQ, Teradata SQL Assistant

Other Tools: Teradata SQL Assistant, Toad 9.7/8.0, DB Visualizer 6.0, Microsoft Office, Microsoft Visio, Microsoft Excel, Microsoft Project

Project Execution Methodologies: Ralph Kimball and Bill-Inmon data warehousing methodology, Rational Unified Process (RUP), Rapid Application Development (RAD), Joint Application Development (JAD)

Data Modeling Tools: Erwin 9.7, ER/Studio v17, Sybase Power Designer

Big Data Technologies: Hadoop 3.0, Hive 2.3, HDFS, HBase 1.2, Apache Flume 1.8, Sqoop 1.4, Spark 2.4, Pig 0.17, Impala 3.0, and MapReduce MRv2/MRv1.

Programming Languages: SQL, Python, PL/SQL, UNIX shell Scripting, AWK, SED

Databases: Oracle 12c, Teradata R15, MS SQL Server 2017,PostgresSql

Operating System: Windows 10/8, Linux, UNIX, Sun Solaris

ETL/Data warehouse Tools: Informatica 9.6, SAP Business Objects XIR3.1/XIR2, Web Intelligence, Talend, Tableau, Pentaho

PROFESSIONAL EXPERIENCE

Confidential -Boston, MA

Data Engineer

Responsibilities:

• As a Data Engineer, provided technical expertise and aptitude to Hadoop technologies as they relate to the development of analytics.
• Responsible for building scalable distributed data solutions using Big Data technologies like Apache Hadoop, MapReduce, Shell Scripting, Hive.
• Worked in Azure environment for development and deployment of Custom Hadoop Applications.
• Used Agile (SCRUM) methodologies for Software Development.
• Wrote complex Hive queries to extract data from heterogeneous sources (Data Lake) and persist the data into HDFS.
• Involved in all phases of data mining, data collection, data cleaning, developing models, validation and visualization.
• Worked on machine learning on large size data using Spark and MapReduce.
• Designed and implemented scalable Cloud Data and Analytical architecture solutions for various public and private cloud platforms using Azure.
• Designed and develop Big Data analytic solutions on a Hadoop-based platform and engage clients in technical discussions.
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing wif Pig.
• Responsible in loading and transforming huge sets of structured, semi structured and unstructured data.
• Developed Spark scripts by using python and bash Shell commands as per the requirement.
• Worked wif NoSQL databases like HBase in creating tables to load large sets of semi structured data coming from source systems.
• Implemented Security in Web Applications using Azure and deployed Web Applications to Azure.
• Responsible for translating business and data requirements into logical data models in support Enterprise data models, ODS, OLAP, OLTP and Operational data structures.
• Created SSIS packages to migrate data from heterogeneous sources such as MS Excel, Flat files and Confidential files.
• Developed numerous MapReduce jobs in Scala for Data Cleansing and Analyzing Data in Impala.
• Applied various machine learning algorithms and statistical modeling like decision tree, logistic regression, Gradient Boosting Machine to build predictive model using scikit-learn package in Python
• Created Data Pipeline using Processor Groups and multiple processors using Apache Nifi for Flat File, RDBMS as part of a POC using Amazon EC2.
• Worked wif Microsoft Azure Cloud services, Storage Accounts, Azure date storage and Azure Data Factory.
• Worked closely wif the SSIS, SSRS Developers to explain the complex data transformation using Logic.
• Designed Data Marts by following Star Schema and Snowflake Schema Methodology, using industry leading Data modeling tools.
• Experience in administration and configuration of ELK Stack (Elasticsearch, Logstash, Kibana) on AWS and performed Log Analysis.
• Configured Azure SQL database wif Azure storage Explorer and wif SQL server.
• Created data integration and technical solutions for Azure Data Lake Analytics, Azure Data Lake Storage, Azure Data Factory, Azure SQL databases and Azure SQL Data Warehouse for providing analytics.
• Assigned name to each of the columns using case class option in Scala.

Environment: Hadoop 3.0, MapReduce, Hive 2.3, Agile, Azure, Oozie 5.1, Pig 0.17, HDFS, Spark 2.4, Python, HBase 1.2, OLAP, OLTP, Scala, SSIS, SSRS

Confidential - New Jersey

Data Engineer

Responsibilities:

• Worked as a Data Engineer to review business requirement and compose source to target data mapping documents.
• Involved in all phases of SDLC using Agile and participated in daily scrum meetings wif cross teams.
• Developed Big Data solutions focused on pattern matching and predictive modeling
• Implemented Installation and configuration of multi-node cluster on Cloud using Amazon Web Services (AWS) on EC2.
• Implemented the Big Data solution using Hadoop, hive and Informatica to pull/load the data into the HDFS system.
• Responsible for loading, extracting and validation of client data.
• Used Python programs for data manipulation, automation process of generating reports of multiple data sources or dashboards
• Deployed and configured Elastic Search, Log stash and Kibana (ELK) for log analytics, full text search, application monitoring in integration wif AWS Lambda and Cloud Watch.
• Worked on NoSQL databases including Cassandra. Implemented multi-data center and multi-rack Cassandra cluster.
• Coordinated wif Data Architects on AWS provisioning EC2 Infrastructure and deploying applications in Elastic load balancing.
• Created sheet selector to accommodate multiple chart types (Pie, Bar, Line) in a single dashboard by using parameters.
• Performed Reverse Engineering of the current application using Erwin, and developed Logical and Physical data models for Central Model consolidation.
• Performed Data Analysis and Data Profiling and worked on data transformations and data quality rules.
• Developed data pipelines to consume data from Enterprise Data Lake (MapR Hadoop distribution - Hive tables/HDFS) for analytics solution.
• Created Hive External tables to stage data and tan move the data from Staging to main tables.
• Developed incremental and complete load Python processes to ingest data into Elastic Search from Hive.
• Created data models for AWS Redshift and Hive from dimensional data models.
• Pulled the data from data lake (HDFS) and massaging the data wif various RDD transformations.
• Created Oozie workflow and Coordinator jobs to kick off the jobs on time for data availability.
• Developed Rest services to write data into Elastic Search index using Python Flask specifications.
• Worked in data management performing data analysis, gap analysis, and data mapping.

Environment: Agile, AWS, Hadoop 3.0, Hive 2.3, HDFS, Python, Cassandra 3.11, NoSQL

Confidential

Data Analyst/ Data Modeler

Responsibilities:

• Worked as a Data Analyst/ Data Modeler me was responsible for all data related aspects of a project.
• Assisted Reporting developers in building Reports using Crystal Reports.
• Used Erwin for reverse engineering to connect to existing database and ODS to create graphical representation in the form of Entity Relationships and elicit more information.
• Created SQL scripts to find Data quality issues and to identify keys, Data anomalies, and Data validation issues.
• Actively involved in writing T-SQL Programming for implementing Stored Procedures and Functions and cursors, views for different tasks.
• Used MS Visio for business flow diagrams and defined the workflow.
• Performed Data analysis for the existing Data warehouse and changed the internal schema for performance.
• Performed Data analysis and Data profiling using complex SQL on various sources systems.
• Wrote SQL Stored Procedures and Views, and coordinate and perform in-depth testing of new and existing systems.
• Performed Data Manipulation using MS Excel Pivot Sheets and produced various charts for creating the mock reports.
• Used SQL Server and MS Excel on daily basis to manipulate the data for business intelligence reporting needs.
• Involved wif data profiling for multiple sources and answered complex business questions by providing data to business users.
• Worked wif DBA's and the security coordinators to get access to the team members.
• Developed conceptual model using Erwin based on business requirements.
• Developed and normalized logical and physical database model using OLTP systems for finance applications.
• Extensively used normalization techniques (up to 3NF).
• Produced functional decomposition diagrams and defined logical data model.
• Involved in development and implementation of SSIS, SSRS and SSAS applications.
• Generated ad hoc SQL queries using joins, database connections and transformation rules to fetch data from legacy oracle and SQL server database systems.

Environment: SQL, T-SQL, MS Visio, MS Excel, OLTP, OLAP, 3NF, SSIS, SSRS, SSAS

Confidential

Data Analyst

Responsibilities:

• Worked wif Data Analyst for requirements gathering, business analysis and project coordination.
• Responsible for the analysis of business requirements and design implementation of the business solution.
• Translated business concepts into XML vocabularies by designing XML Schemas wif UML
• Gathered business requirements through interviews, surveys wif users and Business analysts.
• Involved in designing and developing SQL server objects such as Tables, Views, Indexes (Clustered and Non-Clustered), Stored Procedures and Functions in Transact-SQL.
• Performed Data analysis of existing data base to understand the data flow and business rules applied to Different data bases using SQL.
• Performed data analysis and data profiling using complex SQL on various sources systems and answered complex business questions by providing data to business users.
• Used MS Visio and Rational Rose to represent system under development in a graphical form by defining use case diagrams, activity and workflow diagrams.
• Wrote a complex SQL, PL/SQL, Procedures, Functions, and Packages to validate data and testing process.
• Worked in generating and documenting Metadata while designing OLTP and OLAP systems environment.
• Established a business analysis methodology around the RUP (Rational Unified Process).
• Developed stored procedures in SQL Server to standardize DML transactions such as insert, update and delete from the database.
• Created SSIS package to load data from Flat files, Excel and Access to SQL server using connection manager.
• Developed all the required stored procedures, user defined functions and triggers using T-SQL and SQL.
• Produced report using SQL Server Reporting Services (SSRS) and creating various types of reports.

Environment: XML, T-SQL, SQL, PL/SQL, OLTP, OLAP, SSIS, SSRS