We provide IT Staff Augmentation Services!

Sr. Etl Engineer/da Resume

5.00/5 (Submit Your Rating)

Fremont, CA

SUMMARY

  • 7+ years of IT Experience as Data Analyst/ ETL Developer especially in Data Warehousing and Business Intelligence applications
  • Experience of data warehouse with a good knowledge and expertise in all phases of SDLC methodology including analysis, design, development, testing and maintenance.
  • Experienced in generating and documenting Metadata while designing OLTP and OLAP systems environment
  • Experience in ER Modeling, Dimensional Modeling (Star Schema and Snowflake Schema) Data Warehousing, OLAP tools.
  • Write code to extract data from the Cosmos data center using Scope on theCosmossystem, C, batch files, T - SQL for the MSN NIF pipeline, aggregate data for forwarding to internal teams. Configure and host Malevich site for efficient code review process.
  • Designed dynamic SSIS to transfer data across different platforms, validate data during transferring, and archived data files for different RDBMS.
  • Extensive experience in designing professional User Interactive (UI/UX) web applications with the help of HTML5, DHTML, XML, XHTML, JavaScript, JSON, React JS, JQuery, Ajax, Bootstrap and CSS3
  • Very good exposure to cloud and big data technologies like Azure, AWS, Hadoop, Hive, HDFS, NoSQL Database (MongoDB and Cassandra) and Sqoop.
  • Involved in designing the user experience interface UI/UX strategy, UI requirements, converting findings into UI designs.
  • Comprehensive experience in the implementation of Continuous Integration, Continuous Deployment, Continuous Delivery and DevOps Operations for Agile projects, working with technologies and platforms including UNIX/Linux, Java, Subversion.
  • Create reports in Power BI preview portal utilizing the SSAS Tabular via Analysis connector.
  • Experience in Power BI, creating interactive visualizations, dashboards, reports based on business requirements.
  • Experience with TFS (Team Foundation Server).
  • Data Modeler with strong Conceptual, Logical & Physical DataModelingskills, DataProfilingskills, Maintain DataQuality, experience withJADsessions for requirements gathering, creating data mapping, writing functional specifications, queries.
  • Good knowledge of AWS (Amazon Web Services), S3 Bucket and Redshift (AWS Relational Database).
  • Experience in query optimization, performance and tuning (PL/SQL) using SQL Trace, Explain Plan, Indexing, Hints, Bulk Binds, Bulk Collect, Creation of global temporary tables and table partitioning.
  • Mainframe systems COBOL, JCL, CICS, VSAM, DB2, IMS, IDMS and conversion of Mainframe data toETLStaging tables.
  • The development was prototyped using WhereScape Red and continued throughout the project.
  • Expertise in Gathering and Analyzing Information Requirements, Data Analysis, Data ArchiteASPJcture, BusinessE-Rmodeling, Dimensional Modeling, ETL Design
  • Experience in Informatica Power Center 9x/8.x/7x with Oracle 11g/9i/10g, SSIS andSQLServer Data Warehouse in Microsoft DW/BI environments.
  • Have extensively worked in developing ETL program for supporting Data Extraction, transformations and loading using Informatica Power Center 9.5.1.
  • Expert level capabilities in the design and implementation of policies, standards, and processes supporting Data Governance, Data Quality, Metadata Management, Data Protection, Information Lifecycle Management, Change Management, Issue Management and Continual Service Improvement
  • Design and Implementation of KPIs, Metrics, and Measurements serving the continuous improvement of data governance capabilities.
  • Heavily interacted with Data ware housing team in Metadata Modeling and gathering requirements.
  • Assisted new team members on MDM technologies and project domain.
  • Strong functional knowledge of SAP MDM Architecture, Data Manager, Materials & Services Data Model, tables, MDM Console, TIBCO MDM.
  • Strong experience inData Migration, Data Cleansing, Transformation, Integration, Data Import, andData Exportusing ETL tools such as Microsoft SQL Server Integrated Services and Informatica
  • Experience inETL testing using tools such as Informatica and SSIS
  • Designed Logical/ Physical DataModelfor E-Delivery System UsingER-Studio.
  • Created SQL Loader Scripts, table sizing, indexing, table partition,SQLtuning.
  • Created/ TunedPL/SQLProcedures, SQL queries for DataValidationforETLProcess.
  • Created Validation reports/System integration reports by Oracle Developer Suite 10g.
  • Demonstrated strong development skills in creating custom web parts, timer jobs, event receivers, and workflows
  • Experience in developing conceptual, logical and physical data models as per enterprise standards
  • Proficient in Data Analysis on Oracle, MS SQL Server & MS Access with sound knowledge in extraction of data from various database sources like Oracle, MS SQL Server, DB2 and Flat files into the Data Stage.
  • Experienced in designing theConceptual,Logicaland Physicaldatamodeling using Erwin and ER Studio,Datamodeling tools.
  • Proficiently transforming data from sources (flat files, mainframe, Oracle) to Data warehouse using SQL*LOADER,
  • Experience in leading and managing teams. Handled multiple roles including DataArchitect, Onsite coordinator
  • Experience in Teradata Enterprise Data Warehouse (EDW) and Data Mart.
  • Participated in development/implementation of Cloudera Hadoop environment.
  • Proficient in Normalization/Denormalization techniques in relational/dimensional databases.
  • Experience in writing SQL queries and optimizing the queries in Sybase, Oracle and SQL Server

TECHNICAL SKILLS

Programming Languages: R and Python

Documentation Tools: Microsoft Office (Word & Excel)

Methodologies: Data Modeling -Conceptual/Logical/Physical/Dimensional, Star/Snow flake Schema, ETL, OLAP/OLTP, and Software Development Lifecycle (SDLC) - Waterfall and Agile

Databases: SQL Server, Teradata, Oracle, and MySQL

Data Modeling Visualization Tools: MS Visio, ERwin, and Lucidchart

ETL Tools: Informatica and Microsoft SQL Server Integrated Services (SSIS)

BI Tools: Tableau, Micro Strategy, Power BI, and Microsoft SQL Server Reporting Services (SSRS)

Project Management Tools: Jira and HP ALM

Collaboration and Content Management Tools: Confluence and SharePoint

PROFESSIONAL EXPERIENCE

Confidential, Fremont, CA

Sr. ETL engineer/DA

Responsibilities:

  • Transformed business requirements into database structures which can efficiently store, manipulate, and retrieve information
  • Worked with Business Users to define and document explicit business requirements for implementing subject areas in the Enterprise Data Warehouse and corresponding reports
  • Identifying and translate data and reporting needs into clearly defined requirements
  • Developed a Web service on the Postgres database using python Flask framework which was served as a backend for the real-time dashboard..
  • Performed data analysis by writing complex PL/SQL queries on various source systems and data warehouse to find trends and anomalies in the data in support of the requirements for various strategic and/or tactical projects * Serve as Subject Matter Expert on a particular system and its data, and guide the development activities with assumptions and data dependencies
  • Integrated custom visuals based on the business requirements using Power Bi desktop
  • Deployed the SSRS reports in Microsoft office share point portal server MOSS 2007.
  • Worked on HIPAA Transactions and Code Sets Standards according to the test scenarios such as 837 health care claim transactions.
  • Designed the schema, configured and deployed AWSRedshiftfor optimal storage and fast retrieval ofdata.
  • Involved in creating dashboards and reports in Tableau 8.1. Created report schedules on Tableau server.
  • Extensive experience in enterprise Data flow analysis, data modeling and analytics (SSRS, SSAS, SSIS, ETL, MDM, Entity Data Models and data warehousing) incorporating transaction-oriented business/service level data processes and metadata collection techniques to enhance informatics strategies.
  • Developed complex SQL queries using stored procedures, common table expressions (CTEs), temporary table to support Power- BI and SSRS reports.
  • Developed and architected ETL solutions using IBM DataStage, WhereScape Red, SQL Server SSIS, Teradata Utilities, Ab Initio, Informatica, Oracle Data Integrator, Cognos Data Manager, SAS Data Integrator and Pentaho DI.
  • Preparing Dashboards using calculations, parameters in Tableau.
  • Written Python script for AWS AMI backups in east and west region. This script will create AMI with current date and time. It will delete old version and will keep latest AMI versions in both regions
  • Data Extraction, aggregations and consolidation of Adobe data within AWS Glue using PySpark and developed and executed a migration strategy to move Data Warehouse from an Oracle platform to AWS Redshift.
  • Prepared complex reports using SSRS and deployed into MS CRM.
  • Develop ETL/ELT using Informatica, Wherescape RED, SSIS, and Apache NiFi within the company’s global warehousing environment
  • Researched the existing client processes and guided the team in aligning with the HIPAA rules and regulations for the systems for all the EDI transaction sets.
  • Created Calculated Columns and Measures in Power BI and Excel depending on the requirement using DAX queries.
  • Given Power BI training to Business users which help them to create their own Reports/Dashboards in the Front End.
  • Worked with Tableau and Alteryx Schedulers and workflows to provide end-to-end automation.
  • Written Python script for AWS AMI backups in east and west region. This script will create AMI with current date and time. It will delete old version and will keep latest AMI versions in both regions
  • Designed the schema, configured and deployed AWSRedshiftfor optimal storage and fast retrieval ofdata and performed PoC for Bigdatasolution using ClouderaHadoopfordataloading anddataquerying
  • BuildingHadoopbased ETL workflow to transform and aggregatedata.
  • Updated Python scripts to match training data with our database stored in AWS Cloud Search, so that we would be able to assign each document a response label for further classification
  • Extensively used Tab admin and Tab cmd commands in creating backups and restoring backups of Tableau repository.
  • Created, Implemented, Modified the visualizations in Power BI reports and Dashboards on client requests.
  • Written Python script for AWS AMI backups in east and west region. This script will create AMI with current date and time. It will delete old version and will keep latest AMI versions in both regions
  • Executed ad-hoc data analysis for customer insights using SQL using Amazon AWS Hadoop Cluster.Worked onNormalizationandDe-Normalizationtechniques for both OLTP and OLAP systems.
  • Build strong relationships within the business units and support functions to understand the business needs
  • Develop ETL/ELT processes using Informatica, Wherescape RED, SSIS, and Apache NiFi for Corporate warehousing
  • Responsible for different Data mapping activities from Source systems toTeradata
  • Designed and Developed ETL jobs to extract data from Salesforce replica and load it in data mart in Redshift.
  • Created Managed and External tables in Hive, loaded data from HDFS and performed complex HiveQL queries on the tables based on business needs for reporting
  • Scheduled several times based Oozie workflow by developing Python scripts.
  • Data Profiling, Mapping and Integration from multiple sources to AWS S3.
  • Designed SSRS and excel reports, expense analysis, and balance sheet analysis.
  • Design and Develop ETL Processes in AWS Glue to migrate Campaign data from external sources like S3, ORC/Parquet/Text Files into AWS Redshift.
  • Developed Shell, Perl and Python scripts to automate and provide Control flow to Pig scripts.
  • Responsible for full data loads from production to AWS Redshift staging environment and complete Data loading from PostgreSQL to AWS Redshift Data Lake.
  • Created Power BI Dashboards and integrated them to SharePoint.
  • Worked on DTS/SSIS for transferring data from Heterogeneous Database (Access database and xml format data) to SQL Server.
  • Utilized SSRS reporting services to generate several types of reports from the database.
  • Used DTS/SSIS and T-SQL stored procedures to transfer data from OLTP databases to staging area and finally transfer into data marts and performed action in XML.
  • Developed the complete HTML, CSS and JavaScript of the pages with emphasis on performance, accessibility (AA) and SEO standards.
  • Involved in monitoring of production ETL applications, Production Support through tickets, executing Scripts / managing scheduled activities based on business requirements.
  • Developed data warehouse model in snowflake for over 100 datasets using whereScape.
  • Created and maintainedTest ScriptsandTest Casesbased onHigh Level Functional Requirements Document (FRD)utilizingVisual Studio Team System (VSTS) 2008, Team Foundation Server (TFS), formanual, automated (Functional & Regression), and performance/stress testing
  • Involved in deployment of SSRS reports on the SharePoint Portal.
  • Performance tuning of SQL queries and stored procedures using SQL Profiler and Index Tuning Wizard.
  • Wrote stored procedures to get the fields required for the reports.
  • Created datasets using stored procedures and reports using multi value parameters.
  • Developed several detail and summary reports including line and pie charts, trend analysis reports and sub reports according to business requirements using SQL Server Reporting Services (SSRS).
  • SSIS, IBM Info sphere Data Stage, MS office, MS Access, MS Excel, SharePoint, SQL Server, VBA, SQL Server Management Studio, T-SQL, Microsoft Business Intelligence Studio, Informatica, MS PowerPoint and Power BI, QUEST Foglight, Red Gate.

Confidential, Columbus, OH

Data Analyst

Responsibilities:

  • Lead interviews and meetings with business associates to gather business requirements for Client Billing, Claims Processing applications.
  • Worked with Oozie workflow engine to manage interdependentHadoopjobs and to automate several types ofHadoopjobs.
  • Developing the reports using POWER BI, Power Pivot and SSRS and SSIS, SSAS
  • Conducted research on claims, provider relations and enrollment issues to identify root cause.
  • Involved in Facets System implementation, Electronic Claims and Benefits configuration set-up testing, Inbound/Outbound Interfaces and Extensions, Load and extraction programs involving HIPAA 835 and proprietary format files and Reports development.
  • Build and maintain relationships with business partners to gain an understanding of current and future project scope and intent. Attend and/or facilitate discovery sessions, JAD sessions, requirements reviews, and other sessions with business and technical stakeholders to define and finalize scope and objectives of technology initiatives.
  • Assisted in the design and support of ETL jobs and SSIS packages.
  • Distributed Tableau reports using techniques like Packaged Workbooks, PDF to different user community.
  • Responsible for full data loads from production to AWS Redshift staging environment and complete Data loading from PostgreSQL to AWS Redshift Data Lake. Responsible for developing, support and maintenance for the ETL (Extract, Transform and Load) processes using Informatica Power Center 10.x
  • Migrate the data from on premise database to AWS S3 using Database Migration Service over a Direct Connect link.
  • Worked with Business Analyst in UAT testing and involved in testing in Production region.
  • Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Python and Scala.
  • Building, publishing customized interactive reports and dashboards, report scheduling using SSRS 2016.
  • Experience working with SAS Language for validating data and generating reports.
  • Negotiated, closed and managed projects worth USD 150K in revenue.
  • Meta Data testing, Data validation and Data verification in AWS cloud computing environment.
  • Proficient in performance analysis, monitoring and SQL query tuning using EXPLAIN PLAN, Collect Statistics, Hints and SQL Trace both in Oracle as well as MS SQL.
  • Worked and learned a great deal from AWS Cloud services like EC2, S3, EBS, RDS and VPC.
  • Migrated an existing on-premises application to AWS. Used AWS services like EC2 and S3 for small data sets processing and storage, Experienced in Maintaining the Hadoop cluster on AWS EMR.
  • Imported data from AWS S3 into Spark RDD, Performed transformations and actions on RDD's.
  • Implemented Elastic Search on Hive data warehouse platform.
  • Involved in version migration of workflows from Informatica 8.6 to Informatica 9.1
  • Involved in requirement gathering, design and development for extracting data from the source systems.
  • Worked with analysts and data source systems experts to map requirements to ETL code and conducted analysis with the SME’s.
  • Used Collibra and Excel to document data governance flows, metadata, and data quality metrics and data lineage.
  • Identified and worked with Parameters for parameterized reports in SSRS 2012.
  • Used SSRS to create reports, customized Reports, on-demand reports, ad-hoc reports and involved in analyzing multi-dimensional reports in SSRS.
  • Involved in installation of Tableau desktop 7.0, Tableau server Application software
  • SQL, SAS, and OLAP, Business Objects or equivalent for quantitative/qualitative analysis.
  • SSIS, Informatica power center 9.5.1,TOAD, SME, MS office, Crystal Reports MS Access,

Confidential, FL

Data Analyst

Responsibilities:

  • Implemented complex conceptual database design and database architect into SQL Server 2008/2012 using various constraints and triggers.
  • Played a key role in System Development Life Cycle Process consisting of: Design and Gap Analysis, Business Requirements, Systems Requirements, Test Criteria, and Implementation to have the outputs of the project dealt with the automation of correspondence directed to Insurance policy owners.
  • Worked as an ETL Developer in order to gather requirements and reaching each deadline well ahead of the timeline. Methods employed followed IIBA standards.
  • Performed analysis on enterprise data/report integration & provided functional specification to development team to build Enterprise Reporting Systems.
  • Collected business requirements to set rules for proper data transfer from Data Source to Data Target in Data Mapping.
  • DefectDetection, Reporting and TrackingusingTFS & Participated in Bug-Review meetingsand also usedQuality Center 10.0for other Apps.
  • Defined Functional Test Cases, documented, Executed test script in Facets system.
  • Performing data mapping for the application, confirm and vacillating the requirement at time of testing.
  • Created SQL Server Reports and developed query for generating drill down and drill through reports using SSRS.
  • Build the Dimensions, cubes with star schema using SQL Server Analysis Services (SSAS).
  • Understanding the OLAP processing for changing and maintaining the Warehousing Optimizing Dimensions, Hierarchies and adding the Aggregations to the Cube.
  • Using Erwin and Visio 5.0 design different kind of Data Models for various kind of databases as per the requirement.
  • By using Log Shipping Mechanism extensive knowledge of backing, copying and restoring the transaction database to primary server, secondary server and monitor server.
  • Created SSIS package to load data from Flat File (2GB -- 4GB) to Flat File and Flat File to SQL Server 2005 using Lookup, Fuzzy Lookup, Derived Columns, Condition Split, Term Extraction, Aggregate, Pivot Transformation, and Slowly Changing Dimension.
  • Created reports from data warehouse using SSRS i.e. Drill Down, Drill Through, Sub Reports, Charts, Table.
  • Expert in planning and implementing MS SQL Server Security and database permissions.
  • Created Various DTS Packages to Extract/Transform/Load source data into SQL Server in a highly efficient way.
  • Created DTS Packages to load data from Source like Excel, ORACLE, DB2, SQL 2012, and MS Access.
  • Wrote stored procedures using Dynamic SQL, Cursor to handle special case which can't be handled using static T-SQL or set based operations.
  • Used ETL to implement Slowly Changing Dimension to maintain historical data in Data Warehouse.
  • Implemented replication in SQL Server for managing and scaling databases in an enterprise environment.
  • Loading data in Dimension Tables in SQL Server using SSIS Packages.
  • Used Bulk Copy Program and Bulk Insert to load data from different data sources to SQL Server.
  • Created reports from Data Warehouse using SQL Server Reporting Services (SSRS).
  • Troubleshooting DB performance issue and implementing necessary DB changes and implemented processes to ensure Optimal Query Performance.
  • Used Spark Streaming APIs to perform transformations and actions on the fly for building common learner data model which gets the data from Kafka in Near real time and persist it to Cassandra.
  • Developed Kafka consumer's API in Scala for consuming data from Kafka topics.
  • Consumed XML messages using Kafka and processed the xml file using Spark Streaming to capture UI updates.
  • Set up Collibra Communities, Domains, Types, Attributes, Status, Articulation, and Workflow and customize attribution and solution including custom dashboard with data quality, metrics, status, workflow initiation and issue management for each Domain specific requirements.
  • Manage integration of Collibra Connect with various 3rd-party sources using Mule soft/Any point Studio including Databases, Applications (SAP, PeopleSoft), Ftp, Excel, Text files, etc.
  • Manage workflow process in Collibra via Activity.
  • Developed Preprocessing job using Spark Data frames to flatten Json documents to flat file.
  • Load D-Stream data into Spark RDD and do in memory data Computation to generate Output response.

Environment: SQL Server 2012, SQL Server Integration Services (SSIS), SQL Server Reporting services (SSRS), SQL Server Analysis Services (SSAS), DTS and PowerBI

We'd love your feedback!