SUMMARY

• 8+ years of experience in software development, Architecture decisions and leading projects from concept through the release process
• 3+ years of experience in Hadoop Big Data solutions - Architecting, Leading, development and testing
• Cloudera Certified Developer for Apache Hadoop
• Hands on in Elastic Search, Rest service, Spark 2.1, Hive, DataFrame, DataSet, MapR, M7
• Good understanding and experience with Cloudera Hadoop stack
• Hands on in NoSQL databases like Hbase
• Good working knowledge on Distributed data processing systems
• Expertise in Hadoop Lambda Architecture
• Capable of Designing and Architecting Hadoop Applications and recommending the right solutions and technologies for the application
• Proficient in all Phases of SDLC (Analysis, Design, Development, Testing and Deployment) and gathering user requirements and converting them into software requirement specifications
• Work closely with Business clients
• Worked as liaison between the Customer and the Off-shore & On-shore team
• Excellent Analytical, Programming and Logical skills
• Very good exposure in OLAP
• Capable of handling multiple projects & teams at the same time
• Good Experience as a Tech / Project Lead

TECHNICAL SKILLS

Big Data Eco System: Cloudera Distribution for Hadoop (CDH), MapR, MapReduce, HDFS, YARN, Hive, Pig, Sqoop, Storm, Impala, Elastic search, Scala, Spark, Spark SQL, DataFrame, DataSet, Parquet, AWS, Snappy, Avro, HBase, M7

Programming Languages: Core Java, Python, Scala

Scripting Languages: Shell Script

Operating Systems: LINUX, UNIX, Windows

Database: ORACLE, MySQL, Teradata, SQL DW

Tools: Eclipse, Toad, ER Studio, Apache Ranger

Other Technologies: MS Azure, SSAS, SSRS, PowerBI, Blob, ADF, Amazon Web Services S3, KMS, Rest Service, AWS Comprehend, Docker, Kubernetes, Wildfly 10.0, Boto3

Methodologies: Waterfall, Agile

PROFESSIONAL EXPERIENCE

Confidential, NY

Big Data Hadoop Architect

Responsibilities:

• Analyze requirement and do the impact analysis on the existing system (MapR cluster, Hive, HBase, Oozie, SQOOP, Java, MapReduce and Unix Shell Script).
• POC for the Voice Analytics for Sentiment Analysis using AWS Comprehend
• HBase to M7 migration back-end and rest services in JBoss
• Build Elastic Search Index, prepare the Json data and load into the Index
• Create Restful service to access the Elastic Search Index using Scala with Basic and Advanced Search
• Test and deploy rest services into Docker and Kubernetes
• Develop Spark/Spark-Sql program for various Arbitrations (Employee count, SIC, Year in Business, Hierarchy, etc.,)
• Develop a process to maintain and manager the potential customers from the Lead Management Team
• Create the low-level design and develop a prototype (proof of concept) based on the HLD. Review the prototype with business team.
• Based on the design document, develop an application/tool using the following technologies MapR cluster, Hive, HBase, spark, scala, Kafka, Elastic Search, SQOOP, Java, MapReduce 2, Python, Unix Shell Script and Restful Services.
• Improve the performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, Dataset, Broadcast variables, re-partition/ coalesce, Cache/Persist. Involve in various optimization like Memory optimization, Space optimization, Data Skewness optimization, Query tuning and Code optimization.
• Build the integrated application/tool and test all the connected components using Falcon, Python, UNIX/Perl script, wrapper script.
• Provide support/assist business team for final business use case testing for the developed application/tool/product and final signoff
• Maintain code base, create the final jar/war, and assist deployment team for final production deployment using Bitbucket repository and Jenkins.

Confidential, Houston, Tx

Big Data Hadoop Architect

Responsibilities:

• Interact with all the different stakeholders to get the requirement to bring the data into Enterprise Data Warehouse (EDL - HDP 2.4)
• Translate the requirements into architecture
• Architecture & Data Governance processes
• Interact with the various management team Directors, VP for different approvals
• Interact with the Risk assessment team for the Cyber Security approval for the Fed LLC data
• Produce application architecture diagrams, application interaction diagrams, application blueprints, roadmaps, etc.,
• Manage the offshore team to get the requirement done in Hadoop and MS Azure
• Preparing the Data Model
• Provide design recommendations and thought leadership to sponsors /stakeholders that improve review processes and resolve technical problems.
• Perform historical and incremental loading of data into Hive Partitioned tables using Sqoop
• Recommend and decide technology solutions and perform mapping of the business requirements to systems/technical requirements to ensure they are in line with the enterprise architectural plan.
• Lead in designing, specifying and selecting information system solutions
• Considering functionality, data, security, integration, infrastructure and performance.
• Understand the software architecture design and support development team in developing solutions accordingly
• Review, interpret and respond to detailed business requirements specifications (BRS) to ensure alignment between customer expectations and current or future ICT capability
• Develop, test and implement technology solutions and report on delivery commitments to ensure solutions are implemented as expected and to agreed timeframes

Technologies: Horton Works, HDFS, Hive, Pig, Hue, Sqoop, Scala, Spark, Apache Ranger, Shell script, UNIX, Oracle, Toad, Talend, Amazon AWS, S3, KMS, Bucket Policies, MS Azure, DMG, Blob, ADF, SQL DW, SSAS, PowerShell, Partition Builder, SSRS, Power BI, ER Studio, Load Balancer .

Confidential, Bellevue, WA

Big Data Architect

Responsibilities:

• Manage the BEAM Ingestion team for different tracks
• Provided design recommendations and thought leadership to sponsors /stakeholders that improved review processes and resolved technical problems.
• Co-coordinate between the Business and the Off-shore team
• Requirement gathering and prepare the Design
• Work with different Business and stake holders for each track
• Export and Import data into HDFS- HBase and Hive . creating Hive tables, loading with data and writing Hive queries
• Bulk loading HBase using Pig
• Initial load and incremental load data into HBase thru BEAM via GG
• Implemented solutions using Hadoop, HBase, Hive, Sqoop, Java API, etc.
• Work closely with the business and analytics team in gathering the system requirements
• Load and transform large sets of structured and semi structured data.
• Loading data into HBase tables using Java MapReduce
• Loading data into Hive partitioned tables

Technologies: Horton Works, HDFS, Core Java, MapReduce, Hive, Pig, Apache Ranger, Flume, Storm, Hue, Sqoop, Shell script, UNIX, Oracle, Toad, DMF, Active MQ.

Confidential, Greenville, SC

Big Data Architect

Responsibilities:

• Provided design recommendations and thought leadership to sponsors /stakeholders that improved review processes and resolved technical problems.
• Co-coordinate between the Business and the Off-shore team
• Requirement gathering and prepare the Design
• Export and Import data into HDFS, HBase and Hive using Sqoop.
• Involved in creating Hive tables, loading with data and writing Hive queries
• Bulk loading HBase using Pig
• Implemented solutions using Hadoop, HBase, Hive, Sqoop, Java API, etc.
• Work closely with the business and analytics team in gathering the system requirements
• Load and transform large sets of structured and semi structured data.
• Loading data into HBase tables using Java MapReduce
• Loading data into Hive partitioned tables

Technologies: CDH, HDFS, Core Java, MapReduce, Hive, Pig, Flume, Storm, Elastic search, Scala, Spark,, Shell scripting, UNIX.