Big Data Engineer Resume
Alpharetta, GA
PROFILE:
- 12+ years of IT experience in software development management, Big Data, Datawarehouse Architect, development and design using BI tools, Advanced knowledge in ETL and Data warehousing with OLAP technology.
TECHNICAL SKILLS:
Data Warehousing ETL: PIG, Hive, Sqoop, Flume, Spark, Informatica PowerCenter 9.X/8.6/7.x, Informatica MDM 9.5, Informatica Data Quality (IDQ), Power Exchange, Informatica Data Transformation Studio, SQL Server Integration Services (SSIS), DataMart, FACT & Dimensions tables, Physical & logical data modeling.
OLAP/BI Tools: BusinessObjects XI 3.1, BO Designer/BO Reporter, Congnos 10.x, InfoView,, Crystal Enterprise10/8.5, OBIEE, Micro Strategy.
Databases: Oracle EXADATA/10g/9i/8.0, MS SQL Server 7.0/2000, Sybase 12.5, Teradata, Netezza, DB2
Programming: SQL, T - SQL, PL/SQL, SQL*Plus, SQL*Loader, HTML, Unix, Shell Scripting, SQL Tuning/Optimization.
Tools: TestDirector, ERWin, TOAD, IBM Data Studio, WinSql, Excel, Word, MS-Project, Autosys, Citrix, CVS, SVN, Actimize.
Environment: UNIX, Windows 2000/NT, BIG DATA Hadoop.
PROFESSIONAL EXPERIENCE:
Confidential, Alpharetta, GA
Big Data Engineer
Responsibilities:
- Developed POC’s for our financial clients to prove the benefits of Spark Streaming and Kafka integrations and help them migrate their existing financial models on to Distributed architecture set up.
- Used Spark Streaming to ingest messages from Kafka and persist them as In-Memory Tables and expose them via embedded Thrift server for upstream applications.
- Designed and developed custom applications with Spark Scala and sqoop to ingest data into Hadoop and Hive from various data sources like RDBMS,CSV files, Google analytics API etc.
- Automated the ETL processes to ingest the incremental data on a daily basis into Hadoop and Hive using Spark,Hive and Oozie.
- Coach, mentor, and develop team members, including hiring, leading and retaining future and existing employees, and provide career development planning.
- Experience in planning, developing, and implementing state of the art information solutions facilitating corporate growth.
- Experience in successfully handling projects following water fall and Agile methodologies.
- Review and analyze existing application and software structures for efficiency and effectiveness, to ensure alignment with overall technical vision and architectural strategy.
- Acts as an escalation point for conflicts, risks, issues and action items as brought forth by direct reports or project team. Responds to issues or problems that arise in day-to-day project operations and/or during development efforts. Meets with appropriate staff to define and investigate problems or issues and propose solutions with time and cost estimates. Verifies results of all changes and/or enhancements.
- Manage resource capacity and financial planning for multiple projects
- Manages projects by outlining project requirements in terms of resources, equipment, time and costs. Responsible for development and coordination of business requirements and overall project plans. Communicates project progress, findings, output and deployment plans to leadership and all constituents. Lead cross-functional teams with diverse technical backgrounds.
Environment: Ambari 2.2, Hadoop 2.5.0, Map-Reduce, Pig 0.15.0, Hive 1.2.1, Sqoop 1.4.6, Impala, Oozie, Spark 1.5, Linux, Windows, Scala 2.10, Kafka, Zookeeper, Informatica PowerCenter 9.5.1, Informatica PowerExchange 9.5.1, MySql, Oracle EXADATA, DB2, Teradata, Autosys, UNIX, ERWin, SVN.
Confidential, Alpharetta, GA
Data Architect
Responsibilities:
- Designed and Led the Development effort for CCAT which is “automated” consumer transaction/event monitoring and reporting platform leveraging Actimize® rules modules executed against consumer data for consumer compliance event surveillance monitoring. The CCAT system integrates with various disparate Consumer and Small Business Card, Card Claims, Banking Center Deposits and Payment systems to extract data necessary for the Global Compliance - Consumer Compliance Testing (CCT) Team to follow an established framework for compliance testing which is in line with applicable regulatory and industry standards.
- SILAS (Short Interest Large Aggregation System) platform for Confidential Merrill Lynch Disclosure reporting, incorporating Short Selling, Large Shareholder Report and Reg K disclosure reporting capability.
- Equity Aggregation reporting system designed in order to generate accurate substantial position disclosure reporting, generate long position reporting thus calculating the percentage of ownership by the firm in AMRS, APR and EMEA regions.
- Demonstrated skills in interviewing business and technical leaders to gather and clarify business requirements. Authored High level design documents, Low level design documents, work break down structure. Took lead role in numerous projects and consistently exceeded customer expectations.
Environment: Informatica PowerCenter 9.5.1, Informatica PowerExchange 9.5.1 Actimize, Congnos 10.x, Oracle EXADATA, DB2, Teradata, Autosys, UNIX, ERWin, SVN.
Confidential, Jersey city
ETL Designer
Responsibilities:
- CSDR (Compliance Solution Data Repository) is the single source of data repository for all ML and BAC compliance requirements. Data received from various domain feeds gets loaded into CSDR. Multiple downstream systems consume the data of CSDR for various business requirements like Trade Surveillance, Reporting etc.
- My responsibilities are to coordinate with Business Analyst and Data Modeler and design the ETL for the new feeds.
- Prepare the ETL design document and coordinate with developers to build the ETL process.
- Perform data analysis and scheduling meetings to walkthrough ETL design documents with Onsite and offshore teams.
- Be the lead responsible person for complete Project lifecycle, from Initiation of the Project till code migration to Production and support.
- Built Microsoft SQL Server Integration Services (SSIS) packages for data integration solutions, including extraction, transformation, and load (ETL) packages for data warehousing.
Environment: Informatica PowerCenter 8.6 (Repository Manager, Repository Server Administration Console, Designer, Workflow Manager, Workflow Monitor), SQL Server Integration Services (SSIS), Oracle 10g, PL/SQL, Toad, OBIEE, Congnos 10.x Share point, Autosys, Windows, UNIX, UNIX Shell Scripting, CVS, SVN.
Confidential, CA
ETL/ Informatica Team Lead
Responsibilities:
- MIS Unification is a long-term strategic program established to enhance the efficiency, quality, and accuracy of financial management reporting. The program has defined multiple projects enabling the ability to achieve the strategic end-state. Asset flows capture changes in client account asset balances stemming from activities initiated by the client or client advisor as part of managing the client’s portfolio. It is a key performance metric for JPMorgan Private Bank The goal of this project is to track Flows so that management can assess banker and product area performance on a weekly, monthly, or annual basis. Transactions are utilized as the basis for Flow reporting and serve as the best measure to create an approximation of Flows within Private Banking.
- Built a CRM application using which users can view background information about the client or prospect before interacting with them in order to provide a better quality of service. Users can use activity records to log such details on a regular basis for assessing the past and present banking relationships.
- Built a Profitability mart using which users can generate profit and loss reports which help higher management in making critical decisions.
- Built Revenue and Assets application, the purpose of this application is to collect all the positions of the reporting month and calculate all the revenue components like Net Interest Income, Non Interest Revenue etc.
- Worked on building CMS mart which is the collateral and exposure management system for the Private Bank (PB). It provides a holistic view of assets and liabilities for all PB clients. System functions include lending value assignment, special deal creation and maintenance, collateral optimization, availability and buying power calculation, margin call generation and management, credit portfolio reporting, transaction reporting, and regulatory reporting.
- Participated in the data mart design sessions and was one of the key members in the design group.
- Manage multiple project teams onsite, onshore and offshore.
- Lead Development teams in Data Warehouse projects.
- Develop Informatica mappings to build the respective marts.
- Developed UDT transformations which have the logic to convert multi sheet xls to xml.
- Worked with COBOL files as source files for the mappings.
- Develop DTS packages in SQL server.
- Developed ETL code using SAS Data Integration studio for Profitability project.
- Understand the existing system, process, scripts and procedures used. Perform the required modifications or write new once and apply them to the current project.
- Schedule the jobs using CA Scheduler.
- Migrate code to SIT, UAT and PROD.
- Provide Production support for the applications which are in production and also work on the on going projects.
Environment: Informatica PowerCenter 8.6/7.x (Repository Manager, Repository Server Administration Console, Designer, Workflow Manager, Workflow Monitor), Power Exchange, Informatica Data Transformation Studio, Business Objects, Congnos 9.x SQL Server 2000, Sybase server, Netezza, PL/SQL, DBArtisan, Rapid SQL, SAS Data Integration Studio, CA Scheduler, DTS, Windows, UNIX, UNIX Shell Scripting, CVS, SVN.