PROFESSIONAL SUMMARY:

• 7+ Years of IT experience in data analysis, design, development, automation and testing of Data Warehouses & DataMart’s, using ETL processes, Python, SQL,PL/SQL and Shell scripting and other middleware tools and technologies.
• Hands - on experience across all stages of Software Development Life Cycle (SDLC). Proficiency in data analysis, business/system requirements, data mapping, unit testing, systems integration and user acceptance testing.
• Extensive experience in Extraction, Transformation and Loading(ETL) of data from multiple sources into Data Warehouse and Data Mart.Well versed with Star-Schema & Snowflake schemas used in relational, dimensional modeling.
• Strong knowledge of Hadoop, HBase, Pig, Hive, Hadoop, HDFS, Big Data.
• Involved in converting HIVE/SQL queries into Spark transformations using Spark RDDs, Scala .
• Strong expertise in using ETL Tool Informatica PowerCenter 9.6 and IDQ (Designer, Workflow Manager, Repository Manager, Data Quality (IDQ), Power Exchange and ETL concepts
• Strong knowledge in data modelling, effort estimation, ETL Design, development, system testing, implementation and production support . Experience in resolving on-going maintenance issues and bug fixes.
• Hands on experience working with various databases including DB2, Oracle, Teradata, and SQL Server.
• Experience in writing UNIX shell, Python and Perl scripts to implement business logic and to load/extract the data from tables.
• Expertise in database programming SQL, PL/SQL, Teradata Fast Load, Fast Export,Multi Load, DB2, SQL Server.
• Experience in designing of QVDs and generate reports, error dashboards using Qlikview reporting tool.
• Integrated Kafka and Spark Streaming for our solution and prepared Data Frame on the streaming data.
• Expertise in Tableau BI reporting tools & Tableau Dashboards Developments & Server Administration .
• Encoded and decoded Json objects using PySpark to create and modify the dataframes in Apache Spark.
• Experienced with work flow schedulers , data architecture including data ingestion pipeline design and data modelling.
• Coordinating with Business Users, functional Design team and testing team during the different phases of project development and resolving the issues.
• Developed Apache Pig Scripts to perform Extract-transform-load data pipelines and integrated the data and Processed terabytes of online advertising using Hive Query Language.

TECHNICAL SKILLS:

Big Data Technologies: MapReduce, HDFS, Hive, Pig, HBase, Zookeeper, Oozie, Impala, Sqoop, Spark, Kafka.

Database: SQL, Data Visualization, Teradata, Vertica.

Tools: Informatica Power Center, Beremiz, Unified Modelling Language (UML), Rational Unified Process (RUP), MS Office.

Languages: C++, Linux Shell Scripting, HiveQL, SQL, Python, PySpark, Java Scripting and XML.

Business Tools: Qlikview, Tableau 8.X/9.X, Talend.

Web/Application Server: Apache Tomcat, WebLogic, WebSphere Tools Eclipse, NetBeans.

No SQL Database: HBase, Cassandra, MongoDB, Accumulo.

PROFESSIONAL EXPERIENCE:

Confidential, Jersey City, NJ

Sr. ETL Developer

Responsibilities:

• Worked with variables and parameter files and designed ETL framework to create parameter files to make it dynamic.
• Proficient in extracting metadata from sources, such as Oracle, using ETL tool and transforming them using business logic and loading the data to the target warehouse.
• Responsible for data extraction and data ingestion from different data sources into Hadoop Data Lake by creating ETL pipelines using Pig, and Hive.
• Designed the Data Warehousing ETL procedures for extracting the data from all source systems to the target system.
• Developed Custom ETL Solution, Batch processing and Real Time data ingestion pipeline to move data in and out of Hadoop using Python and shell Script.
• Worked extensively with Data migration, Data cleansing, Data profiling, and ETL Processes features for data warehouses.
• Developed Hadoop integrations for data ingestion, data processing capabilities using different Big Data tools like Hive, Sqoop and Flume.
• Worked as ETL developer responsible for the requirements / ETL Analysis, ETL Testing and designing of the flow and the logic for the Data warehouse project.
• Used Test driven approach for developing the application and Implemented the unit tests using Python Unit test framework.

Environment: PL/SQL, Hadoop, MapReduce, Yarn, Python, Hive, Pig, HBase, Zookeeper, OozieSqoop, Flume, Oracle, Core Java,Cloudera, HDFS, Hue, Eclipse, Linux / Unix, ETL, Tableau 9.0Businessobjects 4.1, Teradata 15.0,HP Vertica 7.1, Spark SQL, Kafka.

Confidential, San Francisco, CA

ETL Developer

Responsibilities:

• Designed ETL process, load strategy and the requirements specification after having the requirements from the end users.
• Created and Implemented highly scalable and reliable highly scalable and reliable distributed data design using NoSQL HBase.
• Fine-tuning and enhance performance MapReduce jobs. Well Versed in NoSQL concepts and good at writing queries.
• Created database objects like views, materialized views, procedures, and packages using Oracle PL/SQL with SQL developer.
• Involved in migration projects to migrate data from data warehouses on Teradata and migrated those to Oracle. Created test cases and pseudo test data to verify accuracy and completeness of ETL process.
• Import and export data into HDFS and Hive using Sqoop.
• Used Hive to analyze the partitioned, bucketed data and compute various metrics for reporting and implemented business logic using Hive UDFs to perform ad-hoc queries on structured data.
• Performed data cleansing by removing NULL values and duplicates using Pig script.
• Transformed data into a specified format by writing Pig Scripts.
• Created complex SQL Queries, PL/SQL procedures and convert them to custom ETL tasks.
• Analyzed the data using Statistical features in Tableau and delivered reports in desired formats by using reporting tools Tableau and Business Object.
• Worked on ETL conversion from Teradata to Vertica. Used existing Framework to replicate the
• Data from Teradata to Vertica for the Dashboard.

Environment: Hadoop, Hive, Impala, Solr, HBase, Oozie, Flume, java, SQL Scripting, Teradata, Tableau, Informatica PowerCenter 9.6.

Confidential, San Francisco, CA

ETL Developer

Responsibilities:

• Designed the ETL processes using Informatica tool to load data from File system and Oracle into the target Oracle Database.
• Developed ETL processes to load data from multiple data sources to HDFS using Sqoop, analysing data using MapReduce, Hive and Pig Latin.
• Designed data mapping documents and detail design documents for ETL Flow.
• Extensively used Informatica PowerCenter Designer, Workflow Manager and Workflow Monitor to develop, manage and monitor workflows and sessions.
• Created new batch jobs for Informatica. Setting up Batches and sessions to schedule the loads at required frequency using Autosys.
• Involved in writing Shell scripts to automate the Pre-Session and Post-Sessions processes.
• Gained a good experience in Production Support & Supporting Application during warranty periods of various ETL Systems.
• Develop consumer-based features and applications using Python, Django, HTML, Behaviour Driven Development (BDD) and pair based programming.
• Used Informatica command task to transfer the files to bridge server to send the file to third party vendor.

Environment: Informatica PowerCenter, DB2, SQL Server, Flat Files, Oracle, Teradata, UNIX Shell Script, Autosys, Tableau.

Confidential, San Jose, CA

PL/SQL Developer

Responsibilities:

• Worked in development, testing and production environment using packages, stored procedures, functions, triggers.
• Built and modified PL/SQL Stored Procedures, Functions, Packages and Triggers to implement business rules into the application.
• Created database objects like tables, views, procedures, packages using Oracle tools like Toad.
• Extensively used error and exception handling techniques.
• Worked on daily sales load job for upstream and downstream data flow.
• Worked on SQL*Loader to Extract, Load and Transform data from varied data sources like Flat files and Oracle.
• Well versed in developing the complex SQL queries, unions and multiple table joins and experience with views.

Environment: Oracle 9i/10g,SQL, PL/SQL TOAD,SQL*Loader, UNIX, MS Windows XP/2000, Oracle Development Suite.