Data Engineer Resume
ClevelanD
SUMMARY
- Motivated hands - on IT professional offering 16 years of experience in information technology encompassing business analysis, technical design, development, testing and delivery of results-oriented solutions that exceed customer expectations.
- Pragmatic Developer with hands on experience in Python, pySpark, Hadoop, MapReduce, Hive, Sqoop, Oozie, Spark SQL, C#.Net, VB.Net, Web API and Web Services.
- Hands on experience in database technologies like SQL Server, Oracle and Teradata.
- Extensively used query tools like TOAD, Teradata Studio, SQL Developer, PLSQL developer and Teradata utilities like BTEQ, Fast Export, Fast Load, Multi Load to export & load data to/from different source systems including flat files.
- Strong in using Azure Data Platform services (Azure Data Lake, Azure Data Factory, Azure Synapse, Data Lake Analytics, Stream Analytics, Azure SQL Database, Azure SQL DW and Azure Data bricks).
- Deep understanding of Azure data storage technologies such as Azure Blob Storage, Azure Data Lake Gen2, Synapse and Azure SQL.
- Good understanding of Data warehousing concepts and Dimensional modelling (Star schema and Snowflake schema)
- Strong analytical and conceptual skills in database design and implementation of RDBMS concepts.
- Hands on experience in versioning tools (Artifactory, GIT, Subversion SVN, VSS and TFS), deployment tools (uDeploy) and scheduling tools (CA7 and Windows Scheduler).
- Experience in Captiva Input Accel 6.5, 7.1 and 7.5 technologies by OpenText where all the customizations to the product are handled through .Net.
- Good knowledge on agile methodology and the scrum process.
- Good understanding of Data Structures, Algorithms, Design Patterns, Object Oriented concepts and MVC architecture.
- Experience in Finance, Insurance and Healthcare domains.
- Quick learner with analytical thinking, decision making and problem-solving skills.
TECHNICAL SKILLS
Big Data and programming languages: Python, pySpark, Hadoop, Sqoop, Hive, Impala, Oozie, C#.NET, VB.Net, ADO.NET, ASP.NET, JavaScript, Web API, Web Services and LINQ
Cloud Data Services: Azure Data Lake, Azure Data Factory, Azure Synapse, Code Pipelines, Azure Blob Storage, Azure Data Lake Gen2
Databases: SQL Server 2008, Oracle 11g, Teradata 16.2, 17.x and Teradata Utilities BTEQ, MLOAD, FASTLOAD
Image Scanning System / Document Capture Software: Captiva Input Accel, Dispatcher 6.5, 7.1 and 7.5 (OpenText products)
Operating Systems: Windows XP, Windows 7, Windows 10 and Windows Server 2012, 2017, 2019
Build Tools: GIT, Subversion SVN, TFS, VSS, Artifactory, Endevor and IBM UrbanCode Deploy
IDE/Interfaces: TOAD, Informatica Power Center, Teradata Studio, MobaXterm, Enterprise Library 4.1, FxCop, SPDisposeChecker, HTML Agility Pack, Infragistics & Telerik controls and Visual Studio.NET 2005, 2008, 2012, 2015, 2017 and 2019
Methodologies and defects tracking: Agile, Waterfall, ALM, ServiceNow, JIRA
PROFESSIONAL EXPERIENCE
Confidential, Cleveland
Data Engineer
Responsibilities:
- Actively involved in initial requirements to build TAE with senior level Executives and Business Teams.
- Actively worked with Enterprise architects and software administration teams to define logical design, enterprise standards and data flow.
- Defined processes and tools best suited to project. Moved between agile and waterfall approaches depending on project specifics and client goals, creating detailed project road maps, plans, schedules and work breakdown structures.
- Led development team to analyze function requirements into technical requirements.
- Define and implement code and design standards.
- Analyze, design and develop data ingestion into HDFS from RDBMS systems (Oracle, SQL Server, Teradata and Postgres) using python, Sqoop, Hive, Flume Kafka and Oozie.
- Work with different file formats (CSV, JSON, Mainframe dataset and delimited) and build data ingestion pipeline to load data from these files into Hive tables using CA7 scheduler.
- Developed spark jobs and hive jobs to apply rules, logic and transform data.
- Used spark structured streaming to perform transformations in data lake which gets data from Kafka and send to HDFS.
- Mitigated risk factors through careful analysis of financial and statistical data.
- Involved in loading and transforming large sets of Structured, Semi-Structured and Unstructured data and analyzed them by running Hive queries using sqoopeval.
- Did various performance optimizations like using distributed cache for small datasets, Partition, Bucketing in hive and Map Side joins.
- Developed Hive tables and Teradata tables, views and procedures
- Developed Oozie workflows to schedule import/export jobs.
- Used Informatica 10.2 to move data from TD stage to EDW.
- Supported integrated/ independent releases, software/hardware upgrades, server upgrades in big data/Hadoop and also participated in Disaster Recovery.
- Coordinated with Release manager for project deployments as per project release schedules.
- Collaborated effectively with product and program managers to ensure project deliverables remained on track and aligned with business goals and expectations.
- Maintained project schedules, work breakdown structures (WBS), issue/decision logs, meeting summaries and status reports to ensure effective, ongoing communications across all team members.
- Motivate, mentor the develop team members.
- Participated in Workshops in understanding Business process and the functional requirements.
- Closely worked with executives to give report demo before deploying.
- Define roles, privileges required to access different database objects.
- Own and drive team’s confluence SharePoint site with documentation and conduct knowledge sharing sessions to RTB support team.
Confidential, Carmel
Application Developer Lead
Responsibilities:
- Lead offshore team and coordinated with onshore team.
- Collaborated with various client business units, vendors and different teams to build a robust windows application with 89 profiles connecting various upstream and downstream systems.
- Involved in analysis of requirements, design, development and delivery of DataImEx project built on Visual Studio 2012 and Oracle 11g technologies.
- Actively involved in project delivery and status calls with client.
- Implemented coding and design standards and conducted effective code reviews.
- Mentor and provide both technical and professional guidance to team members.
- Work with business by providing demos to end users and business stakeholders.
- Established a WIKI site to maintain the documentation of our team’s applications that is used by Developers, Analysts, Support and Business Users.
- Responsible for knowledge sharing sessions to support team.
Confidential, Carmel
Application Developer Lead
Responsibilities:
- Lead offshore team and coordinated with onshore team.
- Involved in analysis of requirements, design, development and delivery of Email Retrieval Service project built on Visual Studio 2012 and Oracle 11g technologies.
- Understand and analyze the existing Java service and apply my technical skills to propose a new architectural change in the current windows service while validating the compatibility with an existing Email Outlook Plug-in client app.
- Implemented coding and design standards and conducted effective code reviews.
- Mentor and provide both technical and professional guidance to team members.
- Responsible for knowledge sharing sessions to support team.
Confidential
Application Developer Lead
Responsibilities:
- Lead offshore team in Visual Studio 2008 and Oracle 10g technologies.
- Involved in re- design, development and delivery of BPA Email Outlook Plug-in Client and BPA Email Web Service.
- The Email Outlook Plug-in client is installed for 600+ business users of different departments in Confidential Financial Group, which saved business user’s time by avoiding unnecessary routing through scan plus when indexes are already known.
- Responsible for handling support activities, incidents based on severity, minor enhancements in RUN team.
- Responsible for knowledge sharing sessions to support team and mentoring juniors.
Confidential
Application Developer Lead
Responsibilities:
- AR 145620 - Input Accel 6.5 to Captiva Capture 7.1 Upgrade - Data Director (DD) Enhancements
- This project will require new data validation integrations with the common remitter system, a proprietary application created by another group within our company.
- Another requirement will be to re-write how the application validates a policy number based on the specific block of work being validated.
- For example, if we are validating a Medicare supplement claim, then we will follow a different set of rules from those used for validating a supplemental health claim, which will be different from the rules used to validate a policy holder services document.
- There is also a requirement to create customized routing logic based on what they call PopCodes, which are values we read through an integration with a proprietary mainframe-based system that acts as an operational data store.
- AR 122352 - Confidential .BPA.GenerateXLS is a custom C# module which is called from within our BPA Standard Process. It is used to allow us to configure any value captured during the capture workflow process to be exported at the end of the process flow to an excel spreadsheet. This is being used heavily by our Cash Control group to process their live checks that require a journal voucher to be generated. The excel file is used as a key input to that process.
- Authored the following Custom Dispatcher Projects. All of these projects were created using the EMC2 Captiva Dispatcher Manager IDE and require in-depth knowledge of the vendor’s application, Form recognition strategies, Capture technologies (OCR, ICR & OMR) and Confidential ’s specific implementation.
- 4086 FORM MORT project is used to classify and extract information for the mortgage division of our company’s investment subsidiary. Was responsible for designing business forms with fill able PDF’s so that the forms can be printed and scanned as TIFFs to the capture system.
- 4086 FORM PP project is used to classify and extract information for the private placement division of our company’s investment subsidiary. Was responsible for designing business forms with fill able PDF’s so that the forms can be printed and scanned as TIFFs to the capture system.
- CPL NB project is used to classify and extract information for the new business area of our Colonial Penn insurance subsidiary.
- CPL PHS BACKSCAN project is used to classify and extract information for various areas within our Colonial Penn insurance subsidiary.
- CPL PHS FORM MULTI project is used to classify and extract information for our Colonial Penn insurance subsidiary.
- CPL PHS FORM PRIVACY project is used to classify and extract information related to our annual privacy mailings for our Colonial Penn insurance subsidiary.
- CPL PHS FORM SWEEP project is used to classify and extract information related to sweepstakes for our Colonial Penn insurance subsidiary.
- AR 140785 - Business Process Management Updates to the Input Accel Enterprise Capture Solution for the FileNet Image Services to FileNet P8 migration.
- This project requires changes to 24 customized capture workflow processes by adding a new configurable step to handle the switching between repository systems correctly and send items to the FileNet P8 system.
- A custom FileNet P8 export module is required for this project that will not only integrate with the EMC2 Captiva Input Accel product, but must also work with our proprietary database structure, our Data Director Multi-Value indexing protocol and the FileNet P8 system.
- AR 143710 - Colonial Penn Life Insurance Customer Portal integration with Input Accel
- Currently, the business reviews and manually works each email from an email inbox and back scans work into FileNet. This will be eliminated to reduce manual processing and increase efficiency through email importer configuration.
- This project will setup an internal Exchange email account for the Customer Portal to collect and send the information to.
- An email importer configuration will be created to poll the email account and turn the emails into images that can then be indexed and routed into the enterprise content repository and on to the enterprise workflow system for processing by the home office.
- A custom department will need to be established in order to prioritize and route the items to the appropriate operator for indexing.
- Lead teams across locations (Hyderabad, Chennai and Coimbatore) in Captiva Input Accel 6.5, 7.1, Dispatcher, Visual Studio 2008, 2012 and Oracle 11g technologies. Managed day to day activities of the team and sync up with onshore team. Actively involved in status calls with client.
- Collaborated with various business units, vendors and different teams to determine their goals to build technical visions and roadmaps. Followed by formulating the plans necessary to execute on those goals and planning schedules with onshore to ensure development resources can deliver.
- Responsible for design, development and delivery of custom modules & custom Input Accel processes.
- Responsible for review and approve the work done by third party vendor.
- Understand and analyze the requirements, existing interfaces and database.
- Applying my technical skills to propose a new architectural change in the current application by analyzing the business requirements and technical restriction.
- Involved in design, development and delivery of client script engine modules.
- Involved in design, development and delivery of Input Accel processes for various departments in business.
- Worked on several dispatcher projects and developed fill-able PDF forms with barcode for business.
- Overseen solutions development initiatives and actively took care of vendor management (EMC2 and Perficient) by collaborating with various client business units & internal teams involved.
- Established a WIKI sit to maintain the documentation of our team’s applications that is used by Developers, Analysts, Support and Business Users.
- Responsible for knowledge sharing sessions to support team and mentoring juniors.
- Implemented coding and design standards and conducted effective code reviews.
- Participated in Project delivery and status meetings.
Confidential, Groton, CT
Application Developer Lead
Responsibilities:
- Lead a group of 3 people in Visual Studio 2008, Oracle 10g and Documentum 6.5 technologies.
- Understand and analyze the requirements, existing interfaces and database.
- Applying my technical skills to propose a new architectural change in the current application by analyzing the business requirements and technical restriction.
- Involved in design, development and delivery of Export/Import tools.
- The Export tool extracts the documents with Meta data from Documentum docbase to local machine using Documentum Framework Classes.
- The Import tool uploads the documents with Meta data extracted by Export tool to SharePoint site.
- Lead the team in design, development and delivery of Oracle Migration tool which will fetch the data from Oracle tables and populates the SharePoint lists.
- Lead the team in design, development and delivery of Team Site Migration tool which will pick static ASP pages from the dump provided by client and extract the content from the ASP files to place it in the Page library of SharePoint as ASPX pages.
- Responsible for knowledge sharing sessions to support team as part of application transition.
- Implemented coding and design standards.
- Was appreciated several times for effective code reviews. Also ran tools like FxCop, SPDisposeChecker to ensure coding standards.
- Participated in Project delivery and status meetings.
Confidential, Groton, CT
Senior Developer
Responsibilities:
- Lead a group of 3 people in Visual Studio 2005 and Oracle technologies.
- Understand and analyze the requirements, existing interfaces and database.
- Applying my technical skills to propose a new architectural change in the current application by analyzing the business requirements and technical restriction. Client gave us a new project for this Application Redesign, of which a critical module was headed by me.
- Involved in design, development and delivery of an Automatic Payment Job in Oracle (TOAD interface) which saved client at least 500 business hours per week and user intervention was removed in the process of making payments.
- Involved in design, development and delivery of an Automatic Email Notification Job in Oracle (TOAD interface) which sends email notifications to users automatically based on valid conditions.
- Involved in migration of non-US payments to Pegasus application through an automated job.
- Responsible for knowledge sharing sessions to Accenture team as part of Pegasus application transition.
- Implemented coding and design standards.
- Reviewed documents.
- Participated in Project delivery and status meetings.
