SUMMARY:

• An IT professional with 9+ years of experience in Software Systems Development, Business Systems and over 2 years of experience in Bigdata ecosystem related technologies.
• Strong Experience on major components in Bigdata ecosystem Hadoop, Hive, Scoop, .
• Experience in data management and implementation of Bigdata applications using Spark & Scala.
• Good understanding of Spark Data frames and RDD.
• Experience in analyzing data using Spark - SQL, Spark-streaming.
• Worked on building the data lake to extract data from traditional database to HDFS environment, performing the data transformations using Hive.
• Experienced in Design, Development, Testing and Maintenance of various Data Warehousing and Business Intelligence (BI) applications in complex business environments.
• Well versed in Conceptual, Logical/Physical, Relational, and Multi-dimensional modeling, Data analysis and Data Transformation (ETL).
• Extensively worked on the ETL mappings, analysis and documentation of OLAP reports requirements. Solid understanding of OLAP concepts and challenges, especially with large data sets.
• Implemented complex business rules by developing robust mappings/mapplets using various transformations like Unconnected and Connected lookups, Normalizer, Source Qualifier, Router, Filter, Expression, Aggregator, Joiner, Update Strategy etc.
• Proficient in developing Entity-Relationship diagrams, Star/Snow Flake Schema Designs, and expert in modeling Transactional Databases and Data Warehouse.
• Efficient in all phases of the development lifecycle, coherent with Data Cleansing, Data Conversion, Data Profiling, Data Mapping, Performance Tuning, and System Testing.
• Proficient in Normalization/De-normalization techniques in relational/dimensional database environments and have done normalizations up to 3NF.
• Efficient in Dimensional Data Modeling for Data Mart design, identifying Facts and Dimensions, creation of cubes.
• Excellent technical, logical, code debugging and problem-solving capabilities and ability to watch the future environment, the competitor and customer's probable activities carefully.
• Experience in UNIX shell scripting, CRON, and FTP in various UNIX environments.
• Experience with SQL Plus and TOAD as an interface to databases, to analyze, view and alter data.
• Experience in Change management and Incident management Processes.
• Good team player with strong analytical and communication skills.

WORK EXPERIENCE:

Confidential, CA

Technical Lead

Responsibilities:

• Worked on project design and implemented use cases to develop hive scripts.
• Implemented Sqoop in shell scripts to ingest data from MySql to GCP bucket as a raw layer.
• Implemented incremental model for multiple databases to ingest data from GCP MySql source to GCP bucket.
• Implemented ETL for raw and gold layers by using hive and pig scripts.
• Worked closely with GCP DevOps, Security developers to fix environmental issues and integrate projects into GCP environment.
• Worked closely with Product Owners (PO) for requirements, Jira stories and source to target mappings (STM). Analyzed STM’s based on source databases.
• Implemented streaming using Scala and flume to get Kohl’s Ecommerce site data daily three times.

Environment: Google Cloud Platform(GCP), HDFS, Hive, Sqoop, Git, Gerrit, Jenkins, Agile and Scrum model, Linux Shell, MySql, Teradata, Scala, Spark, Flume.

Confidential, AR

ETL Developer

Responsibilities:

• Used Sqoop to import data from RDMS (Informix,Oracle & Teradata) to HDFS and later analyzed data using various Hadoop components. Also automated the steps to import the data from various databases.
• Extensively worked on creating Hive external and internal tables and then applied HiveQL to aggregate the data.
• Migrated ETL jobs to Hive scripts for transformations, joins, aggregations.
• Implemented Partitioning, Dynamic Partitions and Buckets in HIVE for increasing performance benefit and helping in organizing data in a logical fashion
• Used Oozie as a job scheduler to run multiple Hive jobs.
• Collaborated with the infrastructure, network, database, application and BI teams to ensure data quality and availability.
• Developed SPARK CODE using SCALA and Spark-SQL/Streaming for faster testing and processing of data.
• Worked with Spark Context, Spark -SQL, DataFrame and Pair RDDs.
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Scala.
• Developed multiple spark programs using scala for data cleaning and preprocessing.
• Configured and deployed Spark applications on a cluster.
• Used Spark shell for interactive data analysis and process using Spark Sql to query structured data.
• Handled importing of data from various data sources, performed data control checks using Spark and loaded data into HDFS.
• Worked with datasets in Scala- creating loading and saving datasets using different dataset operations.
• Worked with JIRA and Git.

Environment: HDFS, Sqoop, Hive, Oozie, Informix,Oracle, Spark &scala.

SQL Developer

Responsibilities:

• Prepared UNIX scripts to validate the source files and moving them to respective source folders.
• Scheduling the UNIX scripts to run the required Jobs as per the schedule time.
• Develop the PL/SQL programs to validations at Database levels.
• Tuning the PL/SQL programs for improving the performance.
• Design and development of jobs using DataStage Designer to load data from different source files to target database.
• Extensively worked with performance tuning of Datastage Jobs and sessions.
• Analyzing the client requirements and Implement the changes to existing Datastage jobs based on the requirements.
• Run Data Stage jobs to process the data extracted (Biweekly) from the legacy systems and make it suitable to load into SAP systems.
• Monitor and validate load process for each source extract and fix the issues in case of any arises.
• Archiving files and maintenance of the development environment using Shell scripts.
• Resolve technical issues such as Data Stage server logins, Access to Database.As part of Data Validation (Using Informatica) I have fulfilled the following responsibilities:
• Developed shell scripts to manage (like prioritizing or deleting) the requests raised by the end users through the web portal.
• Automated the requests processing at different levels using Shell scripts.
• Developed on PL/SQL procedures to make File Validation Checks.
• Involved in Informatica Development, administration and fixing production issues.
• Designed and developed Informatica 8x mappings to extract, transform and loading of the data into Oracle 10g target tables.
• Worked on Informatica power center client tools like Designer, Workflow Manager, Workflow Monitor, and Repository Manager.
• Used Informatica Repository Manager to create Repositories and Users and to give permissions to usersGenerated ABAP programs with delta mode, function module and joiners via Informatica Power exchange.
• Implemented slowly changing dimensions methodology and developed mappings to keep track of historical data.
• Involved in performance tuning the Informatica mappings.
• Generated ABAP Programs using Informatica Designer to load the SAP Reference tables to Oracle Lookup tables.
• Expertise in using TOAD and SQL for accessing the Oracle database.
• Used TOAD to run SQL queries and validate the data in Data warehouse and Data mart.

Environment: Informatica Power Center v9.1, Datastage v8.5, Flat files, Oracle10g & Oracle 11g, PL/SQL, UNIX Shell Programming.

Senior Software Engineer

Responsibilities:

• Designed and Developed Informatica Mappings to Extract, Transform and Load data. The source and target are based on Oracle.
• Used various transformations like Source Qualifier, Aggregator, expression, Joiner, Connected and Unconnected lookups, Filters, Sequence Generator, Router, Update strategy, Union and Stored Procedures to develop the mappings.
• Developed several Mappings and Mapplets using corresponding Source, Targets and Transformations.
• Modified Existing mappings according to the changes in the business requirements.
• Worked with the Business Analysts to understand the requirements.
• Created Sessions and Workflows for the mappings to execute them.
• Performed Unit Testing and wrote various test cases and precise documentation to outline the dataflow for the mappings.
• Created various DDL Scripts for creating the tables with indexes and partitions.
• Created PL/SQL packages, Stored Procedures and Triggers for data transformation on the data warehouse.
• Effectively worked on the performance tuning of the mappings for better performance. Followed standard rules for performance tuning.
• Migrated the Mappings to different environments, development, testing, UAT and Production.
• Used parameter files to provide the details of the Source and Target databases and other parameters
• Preparing daily, weekly and monthly reports.

Environment: Informatica v8.X, CSV files, Oracle9i, PL/SQL Programming.

Senior System Engineer

Responsibilities:

• Automated the repeated tasks with the help of Shell scripts.
• Design and development of jobs using DataStage Designer to load data from different heterogeneous source files to target databases.
• Implemented many transformation activities in DataStage before loading the data into various dimensions and fact tables.
• Used Transformer stage, Aggregator stage, Merge stage and Sequential file stage and Sort stages in designing jobs.
• Used Row generator and Peek stage while testing the job designs.
• Worked with Local and Shared Containers.
• Used parallel processing capabilities, Session-Partitioning and Target Table partitioning utilities.
• Created Reusable Transformations using Shared Containers.
• Developed PL/SQL stored procedures.
• Identified bottlenecks and performance tuned PL/SQL programs.

Environment: Datastage v7.X, Flat files, Oracle9i, PL/SQL, UNIX Shell scripting.