Big Data Developer Resume
3.00/5 (Submit Your Rating)
TexaS
SUMMARY
- 10 years IT experience in Design, Development, Maintenance, Enhancements and Production Support of which includes Data warehousing and Legacy applications using ETL tools like Talend, Informatica and Big Data Technologies (Apache Hadoop stack and Apache Spark).
- Having Good experience in writing Spark, Scala, PIG Script and Shell script on Big DATA platforms.
- IT Experience in various industries like CRM - Address Standardizations (Establishing Global Customer Processes Address Cleansing and Standardized customer data), Telecom (CRM, Billing, provisioning, Order management, Inventory System), US - Healthcare (RxClaim pharmacy PBM (Integration OR Implementation OR SDLC, Claims OR Benefits OR Eligibility, Medicare &Medicaid like Rebates, Iris, Pharmacy, Claims, Membership Data.
- Experience in adhering software methodologies like waterfall, Agile.
- Experience in data analysis, mapping & modelling and experienced in Designing ETL models ( MDM models using Talend) for the multi-platform applications.
- Experienced in using Talend Data Fabric tools ( Talend DI, TBD, Talend MDM, Talend DQ, Talend Data Preparation, ESB, TAC ), BIG DATA (Map Reduce, Spark, Scala, PIG, Hive, Shell Script)
- Experienced in using All Talend DI, MDM, DQ, DP, ESB components.
- Experienced in match/merge in MDM to run match rules to check the effectiveness of MDM process on data.
- Experienced in Data Ingestion projects to inject data into Data lake using multiple sources systems using Talend Bigdata
- Experienced in Talend Service Oriented Web Services using SOAP, REST and XML/HTTP technologies using Talend ESB components .
- Experienced in scheduling Talend jobs using Talend Administration Console (TAC) .
- Experienced in ETL Talend Data Fabric components and used features of Context Variables, MySQL, Oracle, Hive Database components .
- Good Understanding of relational database management systems, experience in integrating data from various data source like Oracle, MSSQL Server, MySQL and Flat files too.
- Experience with Talend to develop processes for extracting, cleansing, transforming, integrating, and loading data into data mart database.
- Capable of processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture.
- Good experience on Talend architecture and Talend installation .
- Familiar with data architecture including data ingestion pipeline design, Hadoop information architecture, data modelling and data mining, machine learning and advanced data processing and Experience optimizing ETL workflows.
- Good experience in Big Data and Hadoop Ecosystem components like, Pig, Hive, Sqoop, Flume, MapReduce.
- Experience in Debugging, Error Handling and Performance Tuning of sources, targets, Jobs etc.
- Profound experience with Informatica Power Center tool 9.6.1/9.1/8.6/8.1 Power Exchange.
- Good Knowledge and experience in using distributed Environments for providing BI solutions to build analytics systems.
- Experience in writing UNIX scripts for Informatica jobs that are used for data movement and transformation purpose.
- Good Knowledge and experience in performance tuning in the live systems for ETL jobs that are built on Informatica as well as Talend.
- Experience in writing database objects like Stored Procedures, Triggers for Oracle, MYSQL databases and good knowledge in PL/ SQL, hands on experience in writing medium level SQL queries.
- Experience in converting the Store Procedures logic into ETL requirements.
- Hands on experience in scheduling tools like Talend Administrator, Zena, Autosys, Informatica Scheduler, Control M for ETL jobs.
- Hands on experience in working with Cloudera.
- Hands on experience in working with Jira, Rally, HP QC - Quality Center Systems, Perforce - Versioning applications, BMC Remedy, Ticketing &Change Request Management applications
- Experience in leading mid-size team and offshore-onsite team coordinator.
- Experience in Big Data technologies like Hadoop (HDFS systems) and it supported databases like hive, Mango DB, pig and NoSQL .
TECHNICAL SKILLS
- Talend Data Fabric 6.2, 6.1,6.4 Informatica Power Center 9.6.1/9.1/8.6/8.1 , Informatica IDQ, Cloudera, Hadoop, MapReduce, HDFS, Hive, Pig, Zookeeper, Sqoop, Scala
- HDFS, YARN, Spark.
- TAC, Informatica Scheduler
- Netezza, Oracle, MY SQL, DB2, Hive, MySQL
- Toad, SQL Developer, WinSQL, Squirrel Client & SQL Assistant
- Core Java, J2EE
- Window Family, Linux and Solaris
PROFESSIONAL EXPERIENCE
Confidential, Texas
Big Data Developer
Responsibilities:
- Experienced in writing Spark and Scala Scripts to extract the data from multiple source systems.
- Using python and SQL scripts, data required for observing objective, quantifiable relationships between the features selected as well as identifying new business opportunities from it.
- Experienced in Architect, Design, Develop, and Improve databases and ETL processes in scope of Application development.
- Utilized the following Big Data technologies for Big Data Warehouse: Hadoop (HDFS, MapReduce, Talend, Hive, Pig, spark and Scala).
- Developed logical and physical data flow models for ETL applications. Cleansed large data sets using ETL and Big Data Tools.
- Experienced in Scheduling the Zena job to schedule and to configure the production jobs.
- Experienced in Encounter Repository project to develop the file level and Data Level controls.
- Hands on experience in Hadoop Ecosystem components such as Hadoop, Spark, HDFS, YARN, Hive, Sqoop, MapReduce, Pig.
- Transporting, and processing real-time stream data using Spark. Working with NO-SQL databases such as Hive.
- Having Good experience to Support multiple project and different vendor issues.
- Responsible to Support AMS team-based incident tickets to track and fix the issues like Bugs/Enhancement.
- Experienced in Transition of application to the maintenance/Support team once the warranty period is completed.
- Responsible for deliverables as committed in the sprint planning.
- Having experience in performance tuning techniques.
- Having experience working with oracle BI tools and GitHub management.
- Good experience in troubleshooting the issues when the vendor raised issues.
- Experienced in writing pig and Shell Scripts to extract the data for multiple vendors on multiple requirements.
- Writing Map Reduce jobs in Java.
- Writing HQL queries in Hive Data warehouse.
- Responsible for modification of ETL data load scripts, scheduling automated jobs and resolving production issues (if any) on time.
- Maintained technical documentation for launching and executing jobs on Hadoop clusters.
- Involved in story-driven agile development methodology and actively participated in daily scrum meetings.
- Responsible for programming code independently for intermediate to complex modules following development standards.
- Responsible for modifying the code, debugging, and testing the code before deploying on the production cluster.
- Implementation on client premises and help in hosting distributed application solution on Cloud facility.
- Involve in Knowledge transfer framework and documentation for the application.
- Implementation signoff by client.
- Creation of archived software artifacts.
Confidential, IL
Senior ETL Talend Developer
Responsibilities:
- Responsible to build the jobs by looking to the ETL Specification documents.
- Having good experience in ETL concepts and analytics.
- Responsible to migrate the People soft data into Oracle system using Talend DI programs.
- Having good experience in developing the master child jobs using tRunjob components.
- Good experience in working all Data Integration components like tmap, tfileInput Delimiter, tfileOutputDelimiter, TflowToiterate, tunique and tjava components.
- Responsible for developing the DI jobs to implement the address validations, cleans and standardization on Talend ETL with different components and used features such as context variables, parameter files database components.
- Working with mainframe technology to process the files from egateway
- Having experience with Talend with big data and distribution is Cloudera and Hortonworks
- Responsible for understanding & deriving the new requirements from Business Analysts/Stakeholders.
- Responsible for Data Ingestion to inject data into Data Lake using multiple sources systems using Talend Bigdata using Hive and HBase components.
- Responsible for deliverables as committed in the sprint planning.
- Having experience in performance tuning techniques.
- Having experience working with Oracle BI tools and GitHub management.
- Good experience in troubleshooting the issues and Talend components.
Confidential
Big Data Developer
Responsibilities:
- Responsible to build the jobs by looking to the ETL Specification documents.
- Having good experience in ETL concepts and analytics.
- Responsible to migrate the People soft data into Oracle system using Talend DI programs.
- Having good experience in developing the master child jobs using tRunjob components.
- Good experience in working all Data Integration components like tmap, tfileInput Delimiter, tfileOutputDelimiter, tflowToiterate, tunique and tjava components.
- Responsible for developing the DI jobs to implement the address validations, cleans and standardization on Talend ETL with different components and used features such as context variables, parameterfiles database components.
- Working with mainframe technology to process the files from egateway
- Having experience with Talend with big data and distribution is Cloudera and Hortonworks
- Responsible for understanding & deriving the new requirements from Business Analysts/Stakeholders.
- Responsible for Data Ingestion to inject data into Data Lake using multiple sources systems using Talend Bigdata using Hive and HBase components.
- Responsible for delivarabels as committed in the sprint planning.
- Having experience in performance tuning techniques.
- Having experience working with Oracle BI tools and Github management.
- Good experience in troubleshooting the issues and Talend components.
Confidential
Senior ETL Consultant
Responsibilities:
- Developed number of complex Informatica Mappings, Mapplets and reusable Transformations to implement the business logic and to load the data incrementally ( delta load/full load )
- Extracting the data from the flat files and heterogeneous Relational databases into staging area and populated into data warehouse using SCD logic to maintain the history.
- Created and Used Workflow Variables, Session Parameters and Mapping Parameters, partitioning, Incremental aggregation, scheduler, Indirect method loading, Constraint Based loading, Incremental loading, Target Load Plan, SCD-Type1 and Type2, Tasks.
- Worked on optimizing and tuning the Teradata views and SQL’s to improve the performance of batch and response time of data for users.
- Created Informatica Mappings to load data using transformations like Filter, Expression, Router, Sorter, Rank, Transaction Control, Source Qualifier, Stored Procedure, SQL Transformation, Normalizer, Sequence Generator, Aggregator, Union, Joiner, Update Strategy, Dynamic Lookup, and Connected and unconnected lookups, Source Qualifier
- Implementing the error handling strategies whenever we required in the mappings and applying the default values
- Experienced in the performance tuning part in SQL/PLSQL quires and Informatica level
- Identified and resolved the bottlenecks in source, target, transformations, mappings and sessions to improve performance.
- By looking into logs based on busy percentage, identify the issue whether bottleneck is at reader/writer/transformation thread and fixing the same based on that log to improve the performance.
- Implemented Slowly Changing Dimensions ( SCDs, Type1and Type 2 )
- Maintained Code standards in warehouse metadata, naming standards and warehouse standards for future application development
- Created & maintained tables, views, synonyms and indexes from Logical database design document.
- Extensively Worked on Extraction, Transformation and Load (ETL) process using PL/SQL to populate the tables in OLTP and OLAP Data Warehouse Environment
- Tested the mappings using the Test Load option and Unit Test cases
- Performed the Unit and Integration Testing, which validated that the data is mapped correctly which provides a qualitative check of overall data flow.
- Created and Documented ETL Test Plans, Test Cases, Expected Results, Assumptions and Validations
- Created Teradata External loader connections such as MLoad, UPsert and Update, Fastload while loading data into the target tables in Teradata Database
- Provided technical Support in deployment of workflows, worklets and sessions for weekly releases using repository manger and maintained the same Runbook.
- Data corrections - problem finding using custom scripts written in SQL/PLSQL and problem solving using SQL/PLSQL with procedures and functions
- Regular maintenance of billing activities and daily issues resolving using SQL/PLSQL
- Efficient design and construction of the system to extract data from operational systems into the defined physical data model per the business and technical requirement.
- Review the requirement documents with the Client Data Analyst team and the Business teams as Business System analyst.
- Strong experience in production support & implementation part and RCA part and fixing the issues as per SLA and Provided the L1 and L2 level support for team as well client for normal and emergency issues on production environment.
- Developed and executed Talend jobs one module using DI for Rebate systems.
- Extensively used Talend Administrator Console (TAC) for running the jobs on different servers by passing various context parameters.
- Have the knowledge on scheduling jobs on TAC.
- Used SVN for version control tool for Talend jobs.
- Developed jobs to perform Address Standardization where customer addresses are standardized and loaded into HBase table which involves Talend jobs.
- Developed joblets that are reused in different processes in the flow.
Confidential
Senior Technical Support Engineer
Responsibilities:
- Experienced in writing database objects like Stored Procedures, Triggers for Oracle, MY SQL databases and used to write in PL/SQL quires.
- Created and maintained tables, views, synonyms and indexes from Logical database design document.
- Worked on optimizing and tuning the Oracle views and SQL’s to improve the performance of batch and response time of data for users.
- Strong experience in production support and implementation part and RCA part and fixing the issues as per SLA and Provided the L1 and L2 level support for team as well client for normal and emergency issues on production environment.
- Created Informatica Mappings to load data using transformations like Filter, Expression, Router, Sorter, Rank, Transaction Control, Source Qualifier, Stored Procedure, SQL Transformation, Normalizer, Sequence Generator, Aggregator, Union, Joiner, Update Strategy, Dynamic Lookup, and Connected and unconnected lookups, Source Qualifier
- Strong Experience developing REST APIs & Web Services and used to write custom java code to handle the monthly Billing and Provisioning modules.
- Extensive knowledge with application development using core Java, Spring and Spring Boot framework.
- Proficient in Core Java & JavaScript development.
- Responsible for remodelling the Existing business logic to new Netezza models for EDW
- Understanding the exiting SQL Server Store procedures logic and convert them into ETL Requirements.
- XML Generation Process - Identifying the required NZ source tables from the re-modelled NZ tables
- Creating the hybrid mappings for XML generation process for different frequencies
- Identifying the Rulesets to be applied on each Client Info along with Members &Providers’ info.
- Validate the data received & generate the XML files for each client and transferred to require to third parties/downstream systems.
- Modifying the generated XML files using XML formatter/Validator/Beautifier as per business owner/third-party requirements
- Preparing the UNIX scripts for SFTP of XML files to different vendors on external Servers.Unit testing and System Testing of mappings scheduling the ETL jobs using Control M scheduler
- Monitoring the daily/weekly DW ETL workflows