SUMMARY:

• 16+ years of experience in software development, architecture decisions and leading projects from concept through the release process
• 4+ years of experience in Hadoop Big Data solutions - (Architect, Data model, data pipeline for ingestion, development)
• 2+ years of experience in Scala / Spark
• Hands on Experience in Big Data technologies (Hive, Sqoop, Scala, Spark, Spark-Sql, Pig, Oozie, HDFS, Map Reduce)
• Cloudera Certified Developer for Apache Hadoop
• Proficient in Scala/Spark development and performance tuning
• Hands on in Hive, Sqoop, ORC and performance tuning (Tuning by using Vectorization, CBO, Bucketing and Partitioning)
• Good exposure in Hadoop Lambda Architecture
• Proficient in all Phases of SDLC (Analysis, Design, Development, Testing and Deployment) and gathering user requirements and converting them into software requirement specifications
• Work closely with Business customers and worked as liaison between the Customer and Off-shore
• Excellent Analytical, Programming and Logical skills
• Good exposure in OLAP
• Capable of handling multiple projects & teams at the same time
• Good Experience as a Tech / Project Lead

TECHNICAL SKILLS:

Big Data Eco System: Cloudera Distribution for Hadoop (CDH), MapReduce, HDFS, YARN, Hive, Pig, Sqoop, Storm, Impala, Spark, Parquet, Flume, AWS, Snappy, Avro, HBase

Programming Languages: Core Java, Scala

Scripting Languages: Shell Scripting

Operating Systems: LINUX, Windows

Database: Oracle, MySQL, Teradata, SQL DW

Tools: IntelliJ, Eclipse, Toad, ER Studio, Apache Ranger

Other Technologies: MS Azure, SSAS, SSRS, PowerBI, Blob, ADF, AWS S3, KMS

Methodologies: Waterfall, Agile

PROFESSIONAL EXPERIENCE:

Confidential, Houston, TX

Big Data Hadoop Architect / Lead Developer

Responsibilities:

• Performance Tuning (Vectorization, CBO, Partition & Bucking) large Hive reporting queries
• Optimize the load process during Partition & Bucketing
• Developing Scala/Sparks Jobs using various APIs (RDD, DataFrame, DataSet)
• Writing Spark Sql Queries using Analytical / Aggregate function
• Create data pipeline to ingest different data sources into Hadoop data lake using Spark/Scala
• Perform map side join in Scala/Spark programs using Broadcasting variables
• Ingested Structured / Semi-Structured data sources using Spark/Scala
• Create Hive and Scala/Spark queries using Analytical functions
• Perform historical and incremental loading of data into Hive Partitioned tables using Sqoop
• Interact with different stakeholders to get the requirement to bring the data into Enterprise Data Warehouse (EDL - HDP 2.4)
• Translate the requirements into architecture
• Architecture & Data Governance processes
• Interact with the Risk assessment team for the Cyber Security approval for the Fed LLC data
• Prepare application architecture diagrams, application blueprints, roadmaps, etc.,
• Define the Big Data - Hadoop guidelines and roll-out to the project team
• Manage the off-shore team to get the requirement done in Hadoop and MS Azure
• Design and create Data Model
• Review, interpret and respond to detailed business requirements specifications (BRS) to ensure alignment between customer expectations and current or future ICT capability
• Develop, test and implement technology solutions and report on delivery commitments to ensure solutions are implemented as expected and to agreed timeframes

Technologies: Horton Works, HDFS, Hive, Pig, Hue, Sqoop, Scala, Spark, Spark-Sql, Apache Ranger, Shell script, UNIX, Oracle, Toad, Talend, Amazon AWS, S3, KMS, Bucket Policies, MS Azure, DMG, Blob, ADF, SQL DW, SSAS, SSRS, Power BI, ER Studio, Load Balancer .

Big Data Architect / Lead Developer

• Performance Tuning (Vectorization, CBO, Partition & Bucking) large Hive reporting queries
• Optimize the load process during Partition & Bucketing
• Developing Scala/Sparks Jobs using various APIs (RDD, DataFrame, Dataset)
• Writing Spark Sql Queries using Analytical / Aggregate function
• Create data pipeline to ingest different data sources into Hadoop data lake using Spark/Scala
• Perform map side join in Scala/Spark programs using Broadcasting variables
• Ingested Structured / Semi-Structured data sources using Spark/Scala
• Create Hive and Scala/Spark queries using Analytical functions
• Perform historical and incremental loading of data into Hive Partitioned tables using Sqoop
• Provided design recommendations and thought leadership to sponsors /stakeholders that improved review processes and resolved technical problems.
• Co-coordinate between the Business and the Off-shore team
• Requirement gathering and prepare the Design
• Work with different Business and stake holders for each track
• Export and Import data into HDFS- HBase and Hive . creating Hive tables, loading with data and writing Hive queries
• Loading data into Hive partitioned tables

Technologies: Horton Works, HDFS, MapReduce, Hive, Pig, Apache Ranger, Flume, Storm, Hue, Sqoop, Shell script, UNIX, Oracle, Toad, Active MQ, Scala, Spark, Spark-Sql, .

Big Data Hadoop Architect / Lead Developer

• Provided design recommendations and thought leadership to sponsors /stakeholders that improved review processes and resolved technical problems.
• Co-coordinate between the Business and the Off-shore team
• Requirement gathering and prepare the Design
• Export and Import data into HDFS, HBase and Hive using Sqoop.
• Involved in creating Hive tables, loading with data and writing Hive queries
• Bulk loading HBase using Pig
• Implemented solutions using Hadoop, HBase, Hive, Sqoop, Java API, etc.
• Work closely with the business and analytics team in gathering the system requirements
• Load and transform large sets of structured and semi structured data.
• Loading data into HBase tables using Java MapReduce
• Loading data into Hive partitioned tables

Technologies: CDH, HDFS, Core Java, MapReduce, Hive, Pig, Flume, Storm, Elastic search, Shell scripting, UNIX.

Big Data Hadoop Architect / Lead Developer

• Worked on a Hadoop Cluster with current size of 56 Nodes and 896 Terabytes capacity.
• Written Map Reduce Jobs, HIVEQL, Pig.
• Imported data using Sqoop into Hive and Hbase from existing SQL Server.
• Supported code/design analysis, strategy development and project planning.
• Created reports for the BI team using Sqoop to export data into HDFS and Hive.
• Developed multiple MapReduce jobs in Java for data cleaning and preprocessing.
• Involved in Requirement Analysis, Design, and Development.
• Export and Import data into HDFS, HBase and Hive using Sqoop.
• Involved in creating Hive tables, loading with data and writing Hive queries which will run internally in Map Reduce way.
• Work closely with the business and analytics team in gathering the system requirements
• Load and transform large sets of structured and semi structured data.
• Loading data into HBase tables using Java MapReduce
• Loading data into Hive partitioned tables

Big Data Hadoop Lead Developer

• Imported data using Sqoop into Hive and Hbase from existing SQL Server.
• Supported code/design analysis, strategy development and project planning.
• Created reports for the BI team using Sqoop to export data into HDFS and Hive.
• Involved in Requirement Analysis, Design, and Development.
• Export and Import data into HDFS, HBase and Hive using Sqoop.
• Involved in creating Hive tables, loading with data and writing Hive queries which will run internally in Map Reduce way.
• Load and transform large sets of structured and semi structured data.
• Loading data into Hive partitioned tables

Technologies: HDFS, Core Java, MapReduce, Hive, Pig, Sqoop, Shell scripting, UNIX.

Big Data Hadoop Lead Developer

• Supported code/design analysis, strategy development and project planning.
• Created reports for the BI team using Sqoop to export data into HDFS and Hive.
• Involved in Requirement Analysis, Design, and Development.
• Export and Import data into HDFS, HBase and Hive using Sqoop.
• Involved in creating Hive tables, loading with data and writing Hive queries which will run internally in Map Reduce way.
• Work closely with the business and analytics team in gathering the system requirements
• Load and transform large sets of structured and semi structured data.
• Loading data into Hive partitioned tables

Technologies: CDH, HDFS, Core Java, MapReduce, Hive, Pig, Hbase, Sqoop, Shell scripting, UNIX.

Architect

• Understanding the ETL Specification Documents for mapping requirements.
• Extract Data from multiple sources like Flat files/Oracle/FTP site into staging database.
• Extensively worked on Informatica tools such as Source Analyzer, Data Warehouse Designer, Transformation Designer, Mapplet Designer and Mapping Designer to design, develop and test complex mappings and Mapplets to load data from external flat files and RDBMS
• Created mappings using the transformations such as the Source qualifier, Aggregator, Expression, lookup, Router, Filter, Joiner, Union, Sequence Generator, Rank, Normalizer, Transactional Control, Stored Procedure and Update Strategy.
• Involved in performance tuning of mappings, identifying source and target bottlenecks and worked with the sessions and workflow properties to improve the performance.

Technologies Informatica 9.1.0, Flat Files, Oracle, MySQL, Shell scripting, UNIX

Architect

Technologies COBOL, VSAM, DB2, ENDEVOR, INSYNC, TELON, EAZYTRIEVE, CONTROL-D, CONTROL-M, DUMP MASTER and TRACE MASTER

• Strong interpersonal skills, team oriented, and highly motivated, Counseling/ motivating team and addressing their concerns.
• Track record of customer satisfaction.
• Exposure to Software Development methodologies, Quality Assurance and Security & Control procedures.
• Lead Development, Testing & Production Support team with the max team size 35
• Worked with customers to gather business requirements and develop unit & system test plans.
• Creating FRD documents based on the request from business
• Knowledge of configuration management tools
• Leading a project team which is assigned module/ activity/ tasks/ deliverables.
• Work on estimates and allocate tasks to resources
• To prepare schedule for tracking of the tasks. Conduct regular/periodic team meetings to provide project specific plan and status update.
• To review work of team members, ensure that it is as per the laid down specifications by project/client, and meets the desired quality standards.
• Provide technical know-how, support, mentoring & coaching team members on technology, business & other project specific aspects.
• Provide project specific performance feedback to the team members.
• Work on training needs identification for the team members.
• To maintain different database such as of resource allocation sheet, checklist for testing and coding and prepare monthly Metrics.
• Responsible for project review, escalations, quality assurance, tasks completion, delivery of the tasks, customer interface & project status reporting for the project.
• Interpersonal skills, Counseling/ motivating team and addressing their concerns.
• To consult and review with the delivery manager to determine project deliverables, project plan, project staffing, scheduling, project time frame,
• Identification of risks, contingency plans, resource availability, quality process for accomplishing the project milestones during various phases of the project, if the project involved development or is a major enhancement. For other Support projects determine project deliverables, identification of risks, contingency plans, resource availability, and quality process.
• Coaching, mentoring team on technical/business specific aspects & Motivating team members to continuously deliver quality deliverables.
• To conduct regular/periodic team meetings to review project status and project specific plans.
• Provide continuous feedback to team members and plan appropriate training programs for their growth in terms of role/technology/domain expertise keeping project/client/organization specific requirements.
• Ensure basic project specific infrastructure is organized/arranged for its team members working on the project.
• Minimize exposure and risk across multiple projects.
• Ensure project documents are complete, current and stored appropriately.
• Coordinating with other departments for requirements.
• Good understanding of domain, executed several medium to large sized projects, and handled multiple projects/teams in onsite/offshore execution model including contracts and project specific service level agreements (SLAs).
• Liaison with Onsite Engagement Managers for Customer Relationship Management.
• Build strong client relationships and appropriately manage client expectations
• Responsible for meeting the current business plans as well as utilizing resources for meeting the challenges of the future.
• Share knowledge and best practices with team; facilitates project review sessions at the close of each project.
• Responsible for interviewing and approving candidates to be staffed on team.
• Escalation point of contact for the customers for various projects for immediate attention and resolution.
• Client meetings, employee retention, HR issues, performance management, customer satisfaction reviews,
• Handling production support activities
• Experience in Incident/Problem and Knowledge Management process areas.
• Working with global distributed teams
• Excellent oral and written communication skills, flexibility to accommodate working across different time zones and sensitivity to cultural and geographical differences.
• Experience in providing IT support in an onshore/offshore model is preferred

Senior Technical Lead

• Requirements Analysis
• Estimation
• Scheduling
• Preparing Business Requirement Document
• Preparing Business Design Document
• Preparing Work Break Structure
• Preparing Technical Design Document
• Estimation
• Provide Week-end support
• Preparing Abend report
• Review the deliverables
• Communicating with the clients
• Testing (Unit & Integration)
• UAT Support
• Implementation Support.
• Production support activities.

Technologies VS COBOL II, VSAM, DB2, ENDEVOR, FILE MANAGER, CONTROL-D, CONTROL-M, DUMP MASTER and TRACE MASTER

Module Lead

• Involved in Enhancement, Maintenance and production support of the Wholesaler system applications
• Responsible of assigning the support calls to the team members based on the priorities and complexity of the issues, review the tasks and ensure the quality of the deliverables is high
• Responsible for arranging the KT sections about the Wholesaler system applications and the Production Support process to new team members

Technologies VS COBOL II, VSAM, DB2, ENDEVOR, FILE MANAGER, BETA92 and QUIKJOB

Confidential

Software Engineer

• Aid to prepare data model Client Interaction
• Design of File Layout and Creation of Copy Books
• Preparation of Program Specification & review for Unit Test Plan and Unit Test Results
• Test Data
• Unit Testing
• Support from the offshore for System Integration Testing & User Acceptance Testing

Technologies VS COBOL II, VSAM, DB2, ENDEVOR, FILE MANAGER, BETA92 and QUIKJOB