Senior/lead Data Warehouse Engineer / Etl Developer Resume
4.00/5 (Submit Your Rating)
SUMMARY:
- I have about 16 years of experience in Information Technology Industry with Confidential as Senior/Lead Data Warehouse Engineer / ETL Developer. My tasks include ETL Design and Development, Informatica installation, upgrades, configuration, tuning, monitoring, migrations, backup and recovery, Mentoring Junior ETL developers etc. Worked on complete software life cycle starting from requirement analysis, design, development, coding, testing and implementation to production support. Actively involved in the full project life cycle from user requirements to implementation. Willing to learn and adapt to new challenges.
- Demonstrated strong expertise in datawarehouse/ dimensional modelling concepts, practices, and procedures related to ETL design and development. I have strong hands on experience in PL/SQL Programming, ETL Design and Development and other related tools. Work with ETL developers, business analyst, DBAs, system administrators, data modelling teams for ETL build and design. Translate business requirements into technical requirements. Designed, developed and maintained numerous ETL Jobs.
- Recently Involved in the adminstration/development tasks of OBIEE, TABLEAU, Datameer in Cloudera Hadoop environment. Developed processes which pulls data from Hive and transforms it in workbooks of datameer and load those back into Hive in Parquet format or Hive table format and make those files/tables avalable in impala.
- I have a Masters Degree in Statistics, Bachelors Degree in Mathematics and Bachelors Degree in Computers. Currently studying for Masters in Data Analytics(MSDA) from CUNY University.
PROFESSIONAL EXPERIENCE:
Confidential
Senior/Lead Data Warehouse Engineer / ETL developer
- Expert in ETL Development from requirement gathering to ETL Documentation. Designed, Developed and Tested ETL Mappings, Mapplets, Workflows, Worklets using Informatica Powercenter 9.x. Determined optimal approach for obtaining data from diverse source systems. Performance tuning of end - to-end ETL integration processes. Determine optimal approach for obtaining data from diverse source systems.Assumed a key contributor to initiatives that require technical expertise. Possess expertise in Project Migration and release co-ordination activities.
- Taken tech lead responsibilities for the development and implementation of the project. This include design, development, testing, co-ordination of QA, production deploys, co-ordination of integration testing, productionalizing of ETL code. Created all necessary scripts for the database changes. Also this proecss required to develop certain components in PL/SQL procedures. The process is running without any issues since 2005 and underwent several changes/additions as the business requirements changed. Currently the survey has followup emails and thank you emails.
- Taken tech lead responsibilities for the the development and implementation of the project. Priceline enabled the customers to buy the products in any currency of their choice. It was required to modify hundreds of processes including mappings, PL/SQL procedures, Table structures etc. This was done in co-ordination with DBAs, business teams, other ETL developers
- Data Warehouse Development: Created fact tables and dimension tables and developed processes for populating those using Informatica mappings. This required strong uderstanding of dimensional modelling and end to end understanding of the business. Created scripts for PL/SQL proceedures, DB changes etc. Also developed independent consolidated tables and agregated tables to feed BI tools for reporting purposes.
- Retired several legacy processes and data warehouses. This effort is always very sensitive as there are several reports and processes are interlinked with the processes we try to retire. Those type of tasks are done carefully after decoupling it from other live processes. Also retired several old processes like individualization etc as those are no more valid in the current business structure.
- Functioned as the leader in ETL administration and set standards. Performance tuning of end-to-end ETL integration processes.
- Created a URL which gives the list of all sessions Failed, Aborted, Stopped, Running etc. on the day in one page. That helps the on-call person to figure out the status of the daily load, if he missed to get an email for various reasons.
- Periodically extract the long running sessions from the meta data and sessionlogs and work on the sessions which takes more time. Find the bottle neck of those sessions and improve the performance and work to reduce their time.
- Periodically extract the unused mappings/sessions etc from the meta data and create a process to cleanup those.
- Implement naming conventions of workflow,worklet,workflowfile, parameterfile, mappings, sessions, Command tasks, Email Tasks and cron jobs etc. That helps for better communication and helps to search those in the meta data.
- Tool analysis, design and architecture and installation and configuration.
- Determined the need of upgrades, and understand the licensing requirements, analyze the need of grid and failover machines. Determine the capacity of the servers and specify the required configurations to the concerned departments.
- I have independently completed not less than 20 installations/upgrade of PowerCenter 8.x and PowerCenter 9.x on Unix/Solaris and Redhat Linux boxes. Mentored several junior Informatica professionals in administrative tasks. Documented all the procedures and made it available on a wiki page.
- Developed several scripts in AWK and korn shell to serve as the pre/post session command.
- In an effort to standardize the exchange rates across the globe, company decided to receive that information from bloomberg. That information was available in a flat file format for a daily load. It was required to cleanup using a complex logic before loading to the tables. That was done by AWK program.
- The dependent processes are driven by a set of available touch file created by other dependent sessions. The management of time and filelist etc are done through korn shell script as the pre-session command with defined exit codes and its own logs. The logs helps to determine the issues and enable the on-call person to the debugging
- The backup of the repository is done daily at 11:00PM by a script which retains one backup per week and the current week’s backup will be overwritten until the end of the week
- Developed processes which pulls data from Hive and transforms it in workbooks of datameers and load those back into Hive in Parquet format or Hive table format and make those files/tables avalable in impala.
- Installed/upgraded and configured datameer for cloudera cluster.
- Contributd to the requirements elicitation process, created application architecture documents.
- Functioned as a technical lead for several large projects. Mentored several ETL developers
- Suggested alternative methods or processing techniques that could be utilized across CIS and financial systems
- Designed and Implemented architectural improvements for several data warehouse process.
- Performance tuning of Mappings / repositories.
- Suggested best practices
- Worked 24 hours Tier 1 and Tier 2 on-call production support.
Tools: Informatica 4.x/5.x/6.x/7.x/8.x/9.x, PowerCenter debugging utilities, PL/SQLTOAD, SQL Navigator and other corporate tools., JAVA, JIRA Datameer, Tableau, OBIEE, Visualcron, Python. R
O/S: Solaris, RedHat Enterprise Linux, UNIX, AIX, 360Sharepoint, Windows 2008/2012windows 10
Databases: Oracle 10g, Oracle 11g, 12c, MySQL, Cloudera, Hadoop, Greenplum
Confidential, MICHIGAN
Computer Consultant
- STATISTICAL Analysis
- Development of New Mappings for both Dimensions and Fact Tables.
- Extrating Transforming and Loading.
- Developing models for reporting purpose.
- Written functions/procedures in PL/SQL
- Developing EIS screens.
- Schedule the Brio jobs in the Server
- Used Brio/Informatica to meet the reporting requirements
- Upload the reports in the OnDemand Server
- Monitoring Informatica / Brio Scheduled Jobs.
Hardware: SUN E-5500
Software: Sun Solaris 7.0, Windows NT 4.0, MS Exchange, Brio 6.2.1.27, Informatica 4.7 etc Oracle 8.0, SAS 8.0,Oracle SQL Plus, SQL Navigator etc.