Sr. Data Engineer / Talend Cloud Platform Owner Resume
0/5 (Submit Your Rating)
SUMMARY
- Senior Data Engineer with 10 years of IT experience inDesign, Development, Enhancementsof which includes Data warehousing using ETL tool Talend Cloud, Snowflake and AWS Services.
- Experienced in ETL Tools - Talend Big Data Cloud DI, Talend MDM, SSIS
- Experienced in designing and developing generic ETL frameworks, AWS Data Pipelines which are used organization wide.
- Experienced in optimizing ETL workflows, using techniques like parallelization, multithreading, memory optimization, error handling.
- Experienced in establishing overall ETL practices and design standards and adhere to new and existing standards when designing solutions.
- Hands-on experience in SQL and Data warehousing
- Built Talend Cloud generic framework solutions using Java and SQL as needed
- Built CI/ CD standards
- Built FWs to extract or load data from or to different source/targets like files (XML, JSON, delimited, parquet), databases (Amazon RDS, PostgreSQL, MSSQL, Snowflake), Salesforce
- Worked as admin for the scheduling tools like TMC, TAC and Rundeck
- Performed ETL Processes deployment as per established change control processes and procedures.
- Performed code-reviews to check the quality and performance of the code.
- Patch and Upgrade ETL platform
- Experienced inleading mid-size team and offshore-onsite team coordinator.
- Self-Starter and team player with excellent communication, organizational and interpersonal skills with the ability to grasp things quickly.
PROFESSIONAL EXPERIENCE
Confidential
Sr. Data Engineer / Talend Cloud Platform Owner
Responsibilities:
- Developed generic Talend Cloud ETL frameworks
- To load data from files to Snowflake, AWS S3 and to unload data from different RDBMS to files which has different and complex transformations and load types
- To generate file from different RDBMS and load to AWS S3
- ServiceNow integration with Talend Cloud - whenever there is a failure in the ETL jobs, an automated incident would be raised with all the details and would be assigned to the specific support team with the incident priority
- To load data from an API (using rest components) to Snowflake
- To retry sending an email if incase email was not sent to the users due to connection issues
- Developed data pipelines using Talend Cloud ETL and AWS services like Lambda, S3.
- Developed generic FW extract data from SalesForce
- Introduced TMC public API - Swagger API to the team which saved the license cost (~200K USD per year)
- These frameworks saved a lot of time and manpower
- Used TMC to schedule and execute tasks
- Upgradation of Talend Cloud remote server, Talend Cloud studio
- Used Git in association with Talend Cloud as code repository.
- Lead offshore team to develop and deliver different frameworks.
- Tested the code using the Test Data Management practices.
- Worked as an Admin - perform and manage tasks to add/delete users, role permissions, managing and tracking of licenses.
- Deployed jobs to higher environments as needed using TMC.
Confidential
Advisory Services Analyst
Responsibilities:
- Implemented DW jobs - Dimensions and Fact loads using Talend Cloud.
- Automated the process of loading connection details from a file saved on shared drive.
- Handling daily loads using Rundeck.
Confidential
Sr. ETL/ Talend Cloud Developer
Responsibilities:
- Participated in all phases of development life-cycle with extensive involvement in the definition and design meetings, functional and technical walkthroughs.
- Responsible to build the jobs by looking to theETL Specificationdocuments.
- Experienced in implementing the project end to end Solutions.
- Experienced using the Talend Cloud with Big data usingHive, Hdfscomponents.
- Developedjobletsthat are reused in different processes in the flow.
- Implemented pagination using tLoop.
- Dealt with loading high volumes of data to Greenplum Database using the custom component tGeGreenplumGpLoad.
- Implemented the execution of multiple SQL queries in Hive data loads using tFlowtoIterate, tHiveRow.
- Implemented the features context variables.
- Worked on ETL code optimization to reduce the data load time.
Confidential
Sr. ETL Developer
Responsibilities:
- Understanding the business requirements and also the future roadmaps - multiple discussions with the stakeholders-solution architects, business analysts.
- Documented all the functional requirements and converted them to technical document.
- Assessed various source systems that will feed data to the Centralized DB.
- Created jobs which extract data from different source systems like DBs, Files, etc.
- Implemented parallelization and other optimization techniques to tune the performance of the ETL jobs
- Experience in resolving issues and bug fixes.
- Implemented SCDs.
- Involved in code reviews.
- Involved in issue resolution, UAT support and future enhancements of the requirements.
- Involved in system Integration testing support and responsible to drive until SIT signoff
- Lead a team of three members for new enhancements and delivering the tested code.
- Interacted with Business owners in status meeting and understand their requirements for enhancing and fixing the current issues in the existing system.
- Come up with recommendations on improvements to the current applications.
Confidential
Senior Software Engineer
Responsibilities:
- To build MDM solution using Talend Cloud MDM tool.
- Created solution to load the golden record to the target system using different MDM components like tMDMBulkLoad, tMDMClose, tMDMCommit, TMDMConnection, tMDMDelete, tMDMInput, tMDMOutput, tMDMReceive, tMDMRollback, TStewardshipTaskDelete, tStewardshipTaskInput, tStewardshipTaskOutput.
- Performed Data Validation using the validation rules.
- To build solution using Talend Cloud Preparation Tool to showcase the data quality performance of the tool.
Confidential
Associate Software Engineer
Responsibilities:
- Validate the data received in XMLfiles using tXSDvalidator against the XSD provided, parse the XML data using tXMLMap and load data to the targets.
- Worked with XML components like tXSDValidator, tFileInputXML, tXMLMap, tExtractXMLField, tAdnvancedFileOutputXML.
- Migrating/Backfilling the historical reference/market data using ETL.
- Data corrections and enhancement requests through ETL raised by QA.
- Developed MISMO jobs which migrates the data from Tigris to Encompass.
- Build and Deploy ETL jobs using TAC.
- Scheduling of jobs in TAC.
- Used SVN for version control tool for Talend Cloud jobs.
- Monitoring the daily/weekly DW ETL workflow.
Confidential
Trainee Software Engineer
Responsibilities:
- Role as an Android developer.
- Participated in project planning and requirements gathering.
- Designed and developed UI using material design.
- Implemented GPS Tracker.
- Tested code using Appium and Sauce Labs.
Confidential
Trainee Software Engineer
Responsibilities:
- Role as a Mobile developer.
- Participated in project planning and requirement gathering.
- Implemented JSON web parsing using web Services.
- Designed and developed the UI.
