SUMMARY

• 6+ Years of experience in the I.T industry, SQL database development, query performance tuning, ETL and data analysis in python and SAS.
• 2 years of experience with Big Data Ecosystems including Hadoop Spark, Scala, MapReduce, Hive, Impala, Sqoop, Flume, Oozie, Zookeeper and NoSQL databases
• Strong experience in importing and exporting data from different databases like Oracle,, MySQL into HDFS using Sqoop.
• Excellent understanding knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming paradigm.
• Good knowledge on relational databases like Microsoft SQL, MySQL.
• Strong experience in Application Development using Scala Spark, RDBMS and Linux shell scripting.
• Good working knowledge of Apache spark for fast large scale in memory MapReduce.
• Expertise in developing Pig Latin scripts and using Hive Query Language. Used Pig as ETL tool to do transformations, event joins, filter and some pre - aggregation.
• Capable of processing large sets of structured, semi-structured, unstructured data.
• Extensive knowledge of Linux OS, Utilities and Shell scripts.Strong analytical skills with problem solving and root cause analysis experience.
• Good working knowledge of Tableau visualization tool.
• Strong exposure to the Software SDLC and development Agile methodologies
• Excellent team player, with pleasant disposition and ability to lead a team and a proven track record.

TECHNICAL SKILLS

Big Data Ecosystems: Hadoop, HDFS, MapReduce, HBase, Hive, Scala, Sqoop, Flume, Oozie, Spark, Zookeeper

Databases: MySQL, HBase, MS SQL

Operating Systems: Linux, Windows

Tools: Eclipse IDE, Putty

Programming Languages: Scala, Pig Latin, SQL, NoSQL, PL/SQL, T-SQL,, HiveQL

Packages: VMWare, Oracle VM Virtual Box, SSIS(ETL), MS Office

PROFESSIONAL EXPERIENCE

Confidential, Dallas TX

Responsibilities:

• Extracted the data from MySQL into HDFS using Sqoop (version 1.4.6).
• Created and worked Sqoop jobs with incremental load to populate Hive External tables.
• DevelopedSparkSQL to load tables into HDFS to run select queries on top.
• Assisted in writing Scala Spark scripts for data cleansing. Data cleansing and data enrichment to remove duplicates, null values was also done using Pig Latin and HiveQL.
• Developed Hive (version 1.1) scripts for end user/analyst requirements to perform adhoc analysis.
• Very good understanding of Partitions, Bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance.
• Good experience in monitoring and managing the Hadoop cluster using Cloudera Manager.
• Worked with UDFs as and when necessary to use in Pig and Hive queries.
• Managed and reviewed Hadoop log files. Tested raw data and executed performance scripts.
• Loaded data into Hive Tables from Hadoop Distributed File System (HDFS) to provide SQL-like access on Hadoop data.
• Used Cloudera distribution for Hadoop ecosystem. Converted MapReduce jobs intoSpark transformations and actions usingSpark

Confidential, Dallas TX

SSIS/ETL DEVELOPER

Responsibilities:

• Extract, Transform and Load source data into respective target tables to build the required data marts.
• Designed and developed SSIS Packages to import and export data from MS Excel, SQL Server and Flat files.
• Involved in daily batch loads (Full & Incremental) into Staging and ODS areas, troubleshooting process, issues and errors using SQL Server Integration Services (SSIS).
• Developed and tested extraction, transformation, and load (ETL) processes.
• Used various Transformations in SSIS Dataflow, Control Flow using for loop Containers and Fuzzy Lookups etc.
• Extracted data from database and spreadsheets and staged into a single place and applied business logic to load them in the database
• Implemented Event Handlers and Error Handling in SSIS packages
• Developed, monitored and deployed SSIS packages.
• Created Complex ETL Packages using SSIS to extract data from staging tables to partitioned tables with incremental load.
• Created logging for ETL load at package level and task level to log number of records processed by each package and each task in a package using SSIS.

Confidential, Plano TX

SQL Developer

Responsibilities:

• Created and Implemented triggers in T-SQL to facilitate consistent data entry into the database.
• Created parameterized and drill through reports in SSRS
• Creating parameterized Stored Procedures, SSIS packages, triggers, cursors, tables, and views and other SQL joins for building various applications.
• Created stored procedures and triggers for data processing of huge volume of data.
• Used various transformations in SSIS to load data from flat files and FTP to the SQL databases.
• Designed and developed data load jobs using SSIS package and scheduled in SQL Agent.
• Created SSIS packages (.dtsx files) using SSIS to validate, extract, transform and load data to data warehouse databases and data mart databases.
• SSIS configuring the data flow, configuring the individual data flow elements and Monitoring the Performance of the Package.
• Wrote stored procedures and User Define Scalar Functions (UDFs) to be used in SQL scripts.
• Extensively used Joins and Sub-Queries to simplify complex queries involving multiple tables.
• Create and Modify database, Table creation, data manipulation and report generation.
• Used T-SQL Language and System Functions in querying database.
• SSRS report development according to the Business Requirement Document (BRD).