SUMMARY

• I have held responsible positions in several companies specializing in IT services as ETL/Data Engineer/BI developer for past thirteen years.
• My practical experience includes software development, testing, support with solid knowledge and extensive experience in Perl scripting, C, C++, Java, Oracle 10/11g, Unix (HP, AIX, Solaris, RHEL), Shell scripts (Korn), IBM Infosphere DataStage 8.1/8.5/11.5 , Talend 6.3 (Big Data Integration),Hadoop (Cloudera/Hortonworks Big Data echo Systems which includes HDFS, SQOOP, Pig, Hive, Spark, Beeline, Hue) for developing Enterprise Data warehouse, Enterprise Data Lake, OLAP and OLTP systems in the field of Wealth Management, Capital Market, Small Business Banking, Corporate Actions and Supply Chain Management.

TECHNICAL SKILLS

Programming: C, C++, JAVA, Perl, Shell Scripting, Python, Apache Beam

BI Tools: Oracle BI, Tableau 10.5

ETL Tools: IBM Data Stage (8+,11), Talend 6.3(Big Data Integration).

CI/CD Tools: Jenkins, Urban Code

Big Data Tools: HDFS, SQOOP, Hive, Beeline, Pig, PySpark

Databases: Oracle, Sybase, SQL Server, DB2

Data Modelling Tool: Erwin, Power Designer

Agile Tools: JIRA, Rally, Confluence

Version Control: MKS, CA, PVCS, SVN, Bitbucket

Development Tools: Eclipse, VI editor, SQL Navigator, Beyond Compare

GCP Tools: Big Query, Pub/Sub messaging, stack driver, DataFlow, DataProc

PROFESSIONAL EXPERIENCE

Sr. data engineer

Confidential

Environment: Hortonworks Big Data Echo System, Spark, Hive, Python, Oracle 19c, Sql Server 2016, Hue, Beeline

Responsibilities:

• Handled large amounts of data with different file formats such as EBCIDIC, UTF - 8, Fixed Length, Delimited, XML and CSV from various source systems such as mainframes, Oracle and SQL Servers.
• Moving data from HDFS to RDBMS and vice-versa using SQOOP.
• Implemented Partitioning, Dynamic Partitions and Buckets in HIVE for efficient data access.
• Defined and implemented the logging retention period for Hadoop components.
• Saved over half of running time of ETL process time and improved data warehouse application performance and availability by creating UNIX/SQL scripts that automated a monthly HIVE partitioning exercise.
• Translated business requirements into data conversion requirements for business rules, data mapping, data cleansing routines and load specifications.
• Prepared Data Mapping documents, TDD, process documents to help on-call team reduce the turnaround time when there is an issue.
• Proficient in using Agile Scrum methodologies, performed roles of Scrum Master following sprint/standup sessions.
• Worked extensively through Agile development methodology by dividing the application into iterations
• Building CI/CD pipelines using Jenkins.

Sr. Data engineer

Confidential

Environment: Core Java 8, Google cloud Big Query, Data Flow, Apache Beam, Pub Sub, Storage, Jenkins, Confluence, Bit Bucket, IntelliJ IDE, Maven, HortonWorks, Python, Sqoop, Hive. Erwin

Responsibilities:

• Creating logical and physical data model for building EDWH.
• Extract data from Hive tables to import into Data warehouse system in GCP.
• Creation of physical data model for Star-Schema, Fact and Dimension tables.
• Created entity relationship diagrams and multidimensional data models, reports and diagrams.
• Developed ELT jobs using Apache beam to load data into Big Query tables.
• Building CI/CD pipelines using Jenkins.
• Documented problem resolution procedures, reducing future problem-solving time and contributing to staff knowledge base.

ETL BIG DATA DEVELOPER

Confidential

Environment: RHEL-7, Tableau-10.5, SSIS, Talend-6.3, SQL Server-2014, Hadoop(Cloudera 5.7), Shell Scripting, Pig, Sqoop, Hive, Python, Spark, Beeline, Hue, Core Java 8, Autosys, Jenkins, Nexus, Confluence, Bit Bucket

Responsibilities:

• Moving data from HDFS to RDBMS and vice-versa using SQOOP.
• Worked on various input/output serialization frameworks such as ORC, Avro, Text ORC and Parquet with different compression techniques such as LZO and Snappy.
• Implemented Partitioning, Dynamic Partitions and Buckets in HIVE for efficient data access.
• Extensive experience in designing and development end-to-end data acquisition processes to be used in population of data warehouse/data marts and data flow diagrams and workflow/process models.
• Experience in Hive, SQL, Spark, Sqoop, HDFS.
• Developed ETL jobs using Talend Tool.
• Korn Shell scripting to automate various jobs. Creating Autosys jobs for various production/Dev/UAT activities.
• Perform data Lineage between different systems like DAS, LOAN, Non-Term Deposits etc.
• Reconcile the reports between the finance and source system and identify the root cause of the variances.
• Prepare the data model for migrating the application from Sql Server 2014 to HIVE.
• Extensive experience in Star Schema Design and Development.
• Provide technical solution to the development team.
• Work with BA/BSA to finalize the requirement specification
• Prepare MTP, TOM and test case reviews meeting to provide inputs.
• Investigating and resolves technical level 1 &2 operational issues of the applications.
• Perform Technical and Business PIV after the deployment.
• Conduct daily standup meeting

ETL/BI developer

Confidential

Environment: Sun Solaris 10, Perl Scripting, Korn Shell Scripting, Oracle 11g, JCL, Core Java, IBM-AIX 7, Sybase, C, C++, Python, Jenkins, Urban Code, Solaris

Responsibilities:

• Involved in all aspects of product development lifecycle (business requirement, functional specification, development, test plan and documentation).
• Primary on-site technical/Project Lead during the analysis, development timeline resource planning and design phase.
• Working as team coordinator to provide all information to the offshore team.
• Provide technical solution to the development team.
• Work with solution architect to finalize the design and get sign off from team.
• Work with BA/BSA to finalize the requirement specification based on the BRD and final mockup\markup.
• Participate in QA MTP, TOM and test case reviews meeting to provide inputs.
• Working with development team project plan to ensure task completed on time.
• Creation of implementation plan for each release
• Provide constant technical support for Portfolio management applications to ensure optimal durability and reliability
• Investigating and resolves technical level 1 &2 operational issues of the applications.
• Identify and fixed interface and batch engines performance issues.
• Reduced processing time by analyzing job schedules and implementing appropriate modifications.
• Create and maintain batch incidents log.
• Scheduling of batch jobs using ZEKE.
• Creation and maintenance of daily and monthly batch flow.
• Lead the deployments to each environment within the development lifecycle.
• Creation of activity record using HPSM9 for production deployment.

ETL/BI developer

Confidential

Environment: RHEL, DataStage 8.x/11.5, Sql Server 2008, shell scripting, .Net, Crystal Reports

Responsibilities:

• Worked on the migration of DataStage from 8.1 to 8.5 from windows to Linux.
• Worked on the migration of DataStage from 8.5 to 11.5 from on Linux.
• Designing and development of parallel jobs.
• Create a design to convert server jobs into parallel jobs.
• Developed and maintained ETL (Extract, Transformation and Loading) mappings to extract the data from multiple source systems and loaded into Sql Server Database
• Involved in creating new table structures and modifying existing tables and fit into the existing Data Model.
• Designed and developed complex mappings by using Lookup, Filters, Aggregator, Router, Stored Procedure, etc., transformations to implement complex logics while coding a mapping
• Work with solution architect to finalize the design and get sign off from team
• Scheduling jobs on ZEKE.

Technical specialist

Confidential

Environment: Sun Solaris 10, Oracle 10g, Perl, BI Publisher, Core Java 8

Responsibilities:

• Development of reports using oracle pl/sql and BI Publisher.
• Creation of physical data models for the reporting system.
• L3 support and enhancement to the application.
• Responsible and manage the quality of the code being developed using software change management tools.
• Creation of release notes and implementation plan for each release
• Development of ETL jobs to load data for reporting.

Technical lead

Confidential

Environment: Linux, C++, ProC, Oracle 10g, Perl Scripting

Responsibilities:

• Onsite- offshore coordination.
• Environment setup for the application.
• Design and Development of the application.
• Development of Class diagrams from the Business Requirement Document (BRD), Functional Specification.
• Development of Test Cases and Test Plans.
• Hands on in the respective technologies.
• Review of LLD, Code and Unit Test Cases.
• Version control using SVN.
• Key contribution to Defect Prevention and Quality Management Processes.

Developer

Confidential

Environment: C++, Core Java, DB2, JCL, J2EE-Strust,Hibernate,HTML,XML,JavaScript

Responsibilities:

• Develops software solutions by studying information needs; conferring with users; studying systems flow, data usage, and work processes; investigating problem areas; following the software development lifecycle.
• Provides information by collecting, analyzing, and summarizing development and service issues.
• Accomplishes engineering and organization mission by completing related results as needed.
• Documents and demonstrates solutions by developing documentation, flowcharts, layouts, diagrams, charts, code comments and clear code.
• Took Part in POC (Proof of Concept).
• Preparation of LLD, UTS, and UTR.
• Modelling of function and services.