We provide IT Staff Augmentation Services!

Big Data Bi Engineer Resume

3.00/5 (Submit Your Rating)

Charlotte, NC

SUMMARY

  • Over 15 + years of experience in IT working with Banking, Financial and Pharmacy Clients.
  • 5+ years of experience working on Big Data Analytics Technologies and successfully implementing Big data business intelligence and Data Warehouse initiatives.
  • Expert experience in developing Tableau dashboards with Hadoop Hive, Impala, Redshift.
  • Has Expert experience in understanding Business Requirements and implementing the requirements through well design and development process.
  • 3 + years’ experience in end to end design of a big data lake with reports/dashboards in Tableau for Confidential Virtual Assistant ERICA (It is similar to Alexa, Siri, Google Assistant).
  • Has 5 + years in Tableau development and self service tableau server maintenance .
  • Has expertise in migrating legacy mainframe Application to newer product migrations. I have successfully helped team to migrate to FICO DM9 application from mainframe.
  • Has expert knowledge in successfully migrating from one bank source application to another.I have successfully worked in Confidential & T merge acquisions of Susquehanna and Penn Bank and handled migrating RMS application from source to target bank.
  • Has Experience in Latest Tableau versions 2019.x and 2020.x.
  • Proficiency in importing and exporting data using Sqoop, ETL Tools from Relational database systems to HDFS and vice versa.
  • Expert in designing hive database system.
  • Hands - on experience in Unix/Python scripting.
  • Has experience in bringing data from datalake using Spark SQL .
  • Proficiency in BigData tools Hadoop, Spark,Sqoop and hive related technologies.
  • Strong knowledge of Hadoop Architecture and Daemons such asHDFS,JOB Tracker,Task Tracker,Name Node,Data NodeandMap Reduceconcepts.
  • Good experience in writing Spark applications using Python.
  • Worked on Apache Spark by analyzing text analytics needed for predictive modeling.
  • Has the ability to understand and design an end to end big data analytics requirement by using data from a traditional data warehouse and from a Hadoop big data system.
  • Big Data Analytics experience using Microsoft Business Intelligence suite, Hadoop Cloudera and Tableau.
  • Strong Data Warehousing experience in designing data marts needed for the business requirements.
  • Ability to write complex SQLs needed for ETL jobs and analyzing data, and is proficient and worked with databases like SQL Server, DB2, Excel sheets, Flat Files and XML files.
  • Worked on AWS S3 for pulling data from S3 and loading to AWS Redshift.
  • Have excellent understanding of Scala language and its implementation with Spark.

TECHNICAL SKILLS

Big Data/Hadoop Stack: Hadoop, Hive, Sqoop, Flume,Apache Spark

Database: SQL Server, Oracle, Teradata

Programming Languages: Python,Scala, Unix/Linux Shell Scripting

ETL Tools: SSIS

Cloud: AWS

Operating Systems: Windows, UNIX/Linux, UBUNTU, Mac OS 10x

Hadoop Distributions: Hortonworks HDP, Cloudera

Visualization: Tableau 10.2, Tableau 2019.2.4

PROFESSIONAL EXPERIENCE

Confidential, Charlotte, NC

Big Data BI Engineer

Responsibilities:

  • Developing the new reporting tables in hive /impala based on the business requirements for Chatbot Conversation reporting.
  • Lead team to design tableau dashboards as per Business User requirements.
  • Redesigned the existing Insights data tables to include new insights. This involved designing of new tables and creating a daily load process similar to a data ware house in hive.
  • Involved in creating CONTACT US Deflection Reporting to evaluate the usage of ERICA in directly handling Contact US Center Calls.
  • Handled end to end development of ERICA Weekly Update dashboard involving different metrics.
  • Created Executive dashboards with bar charts, line graphs, dual axis charts.
  • Has expert knowledge in implementing parameters,filters, hierarchies,sets.
  • Worked on migrating all the reports from tableau 10.2 server to new tableau 2019.2.4 server without impacting business.
  • Designing the new Big Data Analytics system with existing OLTP database and generating Visualizations with Tableau needed by business team.
  • Creating Ad Hoc reports from big data tables needed for the business users.
  • Analyzing the OLTP Teradata warehouse databases to get the needed data for business requirements.
  • Improving the performance of existing tableau reports that are slow.
  • Designing and developing the databases and reports for various functionalities of ERICA Virtual Assistant features like Insights reporting, Help Suggestions reporting, Intent Reporting.
  • Handled end to end UAT testing of Hadoop files from ICH platform to Hive tables in UAT environment for all the projects.
  • Handled end to end table loading of intent mapping loads.
  • Designed and implemented a typical daily flow of data in hive tables for Insight reporting.
  • Analysed various existing hive tables and re designed by partitioning those tables for better efficiency.
  • Created daily Hive load jobs for daily reporting ETL.
  • Daily Handled entire support of hive ETL reporting jobs working with various prod support team for any issues.
  • Wrote Hive/Spark queries for LOB to provide a consolidated view of data for analysis and adhoc querying.

Environment: Cloudera, Tableau, Flat files, UNIX scripting, Python, Teradata, Hive, Impala, pyspark.

Confidential, Charlotte, NC

Python Data Analyst

Responsibilities:

  • Fetching the data available in the data source location for processing through Python.
  • Developing the data processing and parsing rules required for each data source.
  • Developing a process in python to make the parsed file ready for the database loading.
  • Creating the database tables needed for each data source.
  • Loading the clean data file into oracle database using python.
  • Validating the load process and making sure that source data is loaded without any missing data.
  • Interacting with the NLP team on the data availability that is needed for the data models and ensuring that data is ready for further use.

Environment: Anaconda Python, Flat files, Oracle database.

Confidential, Charlotte, NC

ETL Big Data BI Engineer

Responsibilities:

  • Developing the new reporting tables in hive /impala based on the business requirements for Chatbot Conversation reporting.
  • Developing the new reports/dashboards in Tableau needed by business users.
  • Designing the new Big Data Analytics system with existing OLTP database and generating Visualizations with Tableau needed by business team.
  • Creating Ad Hoc reports from big data tables needed for the business users.
  • Analyzing the OLTP Teradata warehouse databases to get the needed data for business requirements.
  • Improving the performance of existing tableau reports that are slow.
  • Performing text analytics to understand the occurrence of Intents in the conversation history data with machine learning.
  • Designing and developing the databases and reports for various functionalities of ERICA Virtual Assistant features like Insights reporting, Help Suggestions reporting, Intent Reporting.

Environment: Cloudera, Tableau, Flat files, UNIX scripting, Python, Teradata, Hive, Impala,pyspark.

Confidential, Wilson, NC

Data Analytics Consultant

Responsibilities:

  • Designing the new ETL architecture as part of DM9 application Migration.
  • Designing the new reporting database to generate Tableau dashboards needed by business users.
  • Designing the new Big Data Analytics system with existing OLTP database and generating Visualizations with Tableau needed by business team.
  • Analysis on the Retail loans, Commercial loans and Mortgage loans applications and their various feeds to the DataMart.
  • Multiple mapping documents are implemented for various feeds of Retail Loans, Commercial Loans and Mortgage Loans etc.
  • Detailed design of the various Ad Hoc fields that are needed for the DataMart that is not available in the original source systems.
  • Actively involved in the transformation process of the source application feeds to destination DataMart.
  • Involved in the various test region setups for the ETL environment and code migration activities from one testing region to another.
  • Work with Business owners and Business Analysts to understand the Business Requirements
  • Work with Technical Lead and Solution Engineer to come up with technical design documents
  • Work with Data Architects and SMEs to come up with the data mapping documents and the data transformation
  • Developed ETL mappings, transformations using SSIS.
  • Helped end users to design test cases and worked through multiple UAT cycles.

Environment: SSIS, Tableau, Flat files, T-SQL, UNIX scripting, Python, Hortonworks HDP

Confidential, Charlotte, NC

Technical Data Analyst

Responsibilities:

  • Being part of the initial requirement gathering and project kick off discussions.
  • Creating reports with MicroStrategy.
  • Created Multidimensional cubes and Tabular models with SSAS.
  • Guide and Mentor for Team.
  • Interacting with the clients to understand the business and mapping rules of all the interfaces.
  • Involved in creating Low level and High level design documents.
  • Creating the big data architecture for the existing OLTP database.
  • Worked on SQOOP to import data from RDBMS to Hadoop system.
  • Worked on hive to extract data from Hadoop and traditional systems.
  • Designing ETL framework for establishment of batches.
  • Responsible for performance tuning of the ETL process and upgrading the ETL best practices document.
  • Developed batch jobs to maintain day to day updates and a process to check if source and target are in sync or not.
  • Investigate data to identify errors and validate the contents of fields in a data file.
  • Created shell scripts for scheduling Informatica Workflows.
  • Working on SQLs to create needed data from datasets.
  • Design Autosys scheduling scripts to schedule Informatica Workflows and run then through Autosys scheduler.
  • Provided post implementation support.

Environment: Informatica,SQL Server, Cloudera Hadoop.

Confidential

Technical Data Analyst

Responsibilities:

  • Preparing of Analysis and design documents.
  • Developing the Informatica Workflows and apply data transformations rules using the Informatica Powercenter 8.6 according to requirements and load the data to the File.
  • Extensively used Informatica Client Tools like Informatica Designer, Informatica Workflow Manager, Informatica Workflow Monitor to develop Informatica Mappings and Workflows.
  • Used Informatica Workflow Monitor to see the logs and monitor the data flow and fix the jobs.
  • Involved in logging error messages and fixed the errors.
  • Parsing high-level design specifications to simple ETL coding and mapping standards.
  • Developed mappings using different types of transformation like Sorter, Filter, Aggregator, Router, Lookup, Update Strategy, Joiner and Source Qualifier.
  • Developed, executed and documented Unit testing, System Integration testing of workflows.
  • Extensively used Normalizer (for COBOL source files), Router, Lookup, Aggregator, Expression and Update Strategy Transformations.
  • Translation of Business processes into Informatica mappings for building Data marts.
  • Created and Documented ETL Test Plans, Test Cases, Expected Results, Assumptions and Validations.
  • Extensively worked with the Debugger for handling the data errors in the mapping designer
  • Involved in the Migration process from Development, Test and Production Environments.
  • Used SQL tools like TOAD to run SQL queries and validate the data.
  • Wrote stored procedures and triggers in Oracle 10g for managing consistency and referential integrity across data mart.
  • Debugging any failed sessions using breakpoints, testing of stored procedures and functions, testing of Informatica sessions, batches and the target Data.
  • Identified bottlenecks in sources, targets, mappings and sessions and tuned them to improve performance.

Environment: Informatica 8.6.0, DB2, Oracle 10g,Mainframe Cobol files, Z/OS.JCL

Confidential

Data Analyst

Responsibilities:

  • Created Dimension Tables and Fact Tables based on the warehouse design.
  • Involved in creating Logical and Physical design of databases using Erwin.
  • Participated in build of the Data Warehouse, which includes the Design of Data mart using Star Schema.
  • Created ETL detail design document and ETL standards document.
  • Created Repository using Informatica Power Center - Repository Manager.
  • Extracted data from flat files and oracle database, applied business logic to load them in the central oracle database.
  • Involved in Extraction, Transformation and Loading (ETL) Process.
  • Worked on SQL queries and tuned them to improve performance.
  • Extensively used Shell scripts to automate the Pre-Session and Post-Sessions processes.
  • Performed data manipulation using basic functions and Informatica Transformations.
  • Designed & developed reports, Drilldown reports using Microstrategy Reporting module, designed reports with Slice and Dice & Drill down operations.

Environment: IBM DB2, XML files, Flat Files,SQL Server.

Confidential

Business Analyst

Responsibilities:

  • Working to automate repetitive tasks and processes in order to maximize performance.
  • Collating business requirements, then translating these into functional specifications and detailed test plans.
  • Providing information to operations management teams to enable them to look at different angles.
  • Maintaining system protocols by writing and updating procedures.
  • Participating in meetings, workshops, and technical meetings.
  • Establishing and maintaining communication channels with senior managers, key team members and stakeholders.
  • Write up and maintain complete and accurate business process documentation.
  • Conducting research into project-related issues and products.
  • Document business requirements and also define its scope and objectives.

Confidential

Business Analyst

Responsibilities:

  • Understanding of the business requirements and enhancing the existing data warehouse architecture design for a better performance.
  • Worked with Informatica Designer to create, Load, Update mappings using different transformations to move data to different data marts in Data Warehouse.
  • Identified and tracked the slowly changing dimensions, heterogeneous Sources and determined the hierarchies in dimensions.
  • Created users and user groups with appropriate privileges and permissions, folders and folder permissions in Repository manager.

We'd love your feedback!