Sr. Data Engineer Resume
0/5 (Submit Your Rating)
Herndon, VirginiA
SUMMARY
- Over 7+ years of Experience in implementing analytical database applications with RDBMS databases Developed ETL based applications using Talend DI.
- Experienced in Cloud technologies like AWS, Azure.
- Experienced with, UNIX and Linux.
- Designed workflows using Talend Data Integration jobs to extract/load from sources like SAP, Salesforce, REST API endpoint, SOAP API, MQ - Kafka, Active MQ, Microsoft MQ, RDBMS, FTP, Hive, Impala, etc.
- Experience in database development, for creating complex database Queries, Indexes, Views, Stored Procedures, Cursors and Triggers in Oracle, SQL Server.
- Managing and setting up TAC server with SVN/GIT repository.
- Working with AWS components S3, EC2, Redshift.
- Hands-on experience with Amazon EC2, Amazon S3, Amazon RDS, VPC, IAM, Amazon Elastic Load Balancing, Auto Scaling, Cloud Front, CloudWatch, Simple Notification Service and other services of teh AWS eco system.
- Hands on experience in Slowly Changing Dimensions & Changing Data Captures.
- Configured DI jobs with connectors and loading data into different target Databases like MySQL, MS SQL, Excel, Flat files, Snowflake, etc.
- Handling transformation on various data formats like JSON, XML, flat files, semi -structured etc.
- Handled teh encrypted SOAP API calls using routine and Talend ESB services.
- Experience in developing REST API endpoint using Talend ESB services and deploying it as OSGI bundle in Apache Karaf.
- Design and Develop Source-to-Target mappings using java routines within teh Talend ETL tool.
- Proficient in troubleshooting, writing bug reports and working with bug tracking tools like JIRA,Bugzilla,etc.
TECHNICAL SKILLS
- Requirements Elicitation
- Database Administration
- Data & Solution Architecture
- Data Integration
- Use case Development
- Talend Workflows
- Technical Management
- Data Stewardship
- Talend Best practice
- Data Ingestion
- Pipelines
- SQL
PROFESSIONAL EXPERIENCE
Confidential - Herndon,Virginia
Sr. Data Engineer
Responsibilities:
- Extraction, Transformation and data loading were performed using Talend into teh MS SQL server.
- Migrated teh code into QA (Testing) and supported QA team and UAT (User)
- Created detailed Unit Test Document with all possible Test cases/Scripts.
- Working with high volume of data and tracking teh performance analysis on Talend job runs and session.
- Conducted code reviews developed by my teammates before moving teh code into QA.
- Used Talend reusable components like routines, context variable and globalMap variables.
- Provided support to develop teh entire warehouse architecture and plan teh ETL process.
- Modified existing mappings for enhancements of new business requirements.
- Worked on Migration projects to migrate data from data warehouses on Oracle/DB2 and migrated those to Netezza.
- Prepared migration document to move teh mappings from development to testing and tan to production repositories.
- Configured teh hive tables to load teh profitability system in Talend ETL Repository and create teh Hadoop connection for HDFS cluster in Talend ETL repository
- Works as a fully contributing team member, under broad guidance with independent planning & execution responsibilities.
- Designed teh ETL processes using Talend to parse teh JSON data from Oracle, Flat Files to target MS SQL database
- Based on teh requirements created Functional design documents and Technical design specification documents.
- Responsible for performing production support.
- Ensure teh timely code changes as and when teh business requirement changed and to make sure teh business continuity.
- Bug fixes on existing Talend workflows.
- Involved in testing of teh Talend jobs and data validations.
- Created tables as per teh required target database metadata.
- Designed workflows using Talend Data Integration jobs to extract/load from sources like SAP, Salesforce, REST API endpoint, SOAP API, MQ - Kafka, Active MQ, Microsoft MQ, RDBMS, FTP, Hive, Impala, etc.
- Created data pipe line using Talend ESB services to read SOAP API call and loaded into Data warehouse tables.
- Worked on agile methodology and used JIRA tool for sprints and bug tracking.
Confidential, Irving, TX
Data Engineer/Talend Consultant
Responsibilities:
- g teh ETL architecture and Source to Target mapping to load data into Data warehouse.
- Performed data manipulations using various Talend components like tMap, tJavaRow, tjava, tOracleRow, tOracleInput, tOracleOutput, tMSSQLInput and many more.
- Designed and customized data models for Data warehouse supporting data from multiple sources on real time
- Designed ETL process using Talend Tool to load from Sources to Targets through data Transformations.
- Developed advanced Oracle stored procedures and handled SQL performance tuning.
- Involved in creating teh mapping documents with teh transformation logic for implementing few enhancements to teh existing system.
- Developed teh Talend mappings using various transformations, Sessions and Workflows. Sql server was teh target database, Source database is a combination of Flat files, Oracle tables, Excel files and Sql server database.
- Used Talend to Extract, Transform and Load data into Data Warehouse from various sources like Oracle and flat files.
- Created connection to databases like SQL Server, oracle, Netezza and application connections.
- Created mapping documents to outline data flow from sources to targets.
- Involved in Dimensional modeling (Star Schema) of teh Data warehouse and used Erwin to design teh business process, dimensions and measured facts.
- Responsible for developing, support and maintenance for teh ETL (Extract, Transform and Load) processes using Talend.
- Created SSIS Packages to back up teh database and compress and zip teh compressed database backup.
- Handled teh Errors at Control and Data flow level in SSIS Packages.
- Maintained stored definitions, transformation rules and targets definitions using Informatica repository
Confidential
MS SQL Server developer
Responsibilities:
- Used various transformations like Filter, Expression, Sequence Generator, Update Strategy, Joiner, Stored Procedure, and Union to develop robust mappings.
- Developed mapping parameters and variables to support SQL override.
- Developed mappings to load into staging tables and tan to Dimensions and Facts.
- Worked on different tasks in Workflows like sessions, events raise, event wait, decision, e-mail, command, worklets, Assignment, Timer and scheduling of teh workflow.
- Unit testing, code reviewing, moving in UAT and PROD.
- Designed teh Talend ETL flow to load teh data into target tables and create teh Talend jobs to load teh data into Oracle tables.
