SUMMARY:

• 11+ years of IT experience along with 4 years of Apache Hadoop and Spark experience
• Experienced Big Data Developer with good knowledge of HDFS architecture and Spark components such as Spark SQL, Spark Streaming and associated Spark API’s
• Sound knowledge in HADOOP ecosystem components such as MapReduce, Pig, Hive, Spark, Sqoop & Cloudera
• Experience in developing Pig Latin scripts, Hive scripts and using them in conjunction with simple user defined functions
• Experience in writing SQOOP commands to export data from Oracle/SQL server database into HDFS for batch processing
• Experience in analyzing high volume streaming data using Kafka
• Experience in configuring and utilizing 5 node Kafka cluster
• Knowledge in job/workflow scheduling and monitoring tools like Oozie
• Possess strong knowledge in processing high volume data using RDD’s, and advanced Spark API’s such as Dataframes and Datasets
• Hands on experience in developing business critical functions using SCALA
• Experience in using the editor Eclipse Neon to create Scala projects with the build tool: SBT
• Proficient knowledge of monitoring and managing Hadoop cluster using Cloudera
• Background with traditional databases such as Oracle, SQL Server, MySQL with good understanding of ETL processes
• Knowledge of NoSQL databases HBase
• Experience working in Agile Methodology that includes iterations plan proposal, assisting in developing user stories and unit test case design
• Apart from Big Data technologies, possess good experience in developing web based applications using JAVA/C#
• Actively involved in the life cycle of project implementing the Systems, Designing, Development, Testing and Documentation

TECHNICAL SKILLS:

Big Data Eco Systems: HDFS, MapReduce, Hive, Pig, HBase, Sqoop, Oozie, Spark, Spark - SQL, Python, Spark Streaming, Kafka and Scala

No-SQL Database: HBase

RDBMS: Sql Server, Oracle, MS Access and MYSQL

Internet Programming: Java, HTML, Visual C#, ASP.NET, Web Services, Windows Services, ADO.NET

Project Management Tools: TFS, Fortify Tool, Service Now, AZDoc, HP Quality Center and Remedy

Operating System: Windows 10, Linux

PROFESSIONAL EXPERIENCE:

Confidential, NJ

Confidential,BIG DATA DEVELOPER (SPARK/SCALA)

Responsibilities:

• Working with Business team and come up functional specification on data integration requirements
• Set up and configure a 5 node Kafka cluster and monitor the broker activities
• Configure Producers, consumers, set partitions and corresponding replication to handle effective fault tolerance
• Export data from RDBMS to HDFS using Sqoop import options
• Configured Oozie work flows to automate the data flow
• Implement Spark best practices by invoking advanced Spark API’s such as Dataframes and Datasets
• Monitor Spark tasks on each and every transformation and action via Spark UI and plan partition utilization accordingly for effective performance.
• Provided design recommendations and thought leadership to stakeholders that improved review processes and resolved technical problems

Environment: HDFS, Sqoop, Oozie, Scala, Spark and Kafka

Confidential, NJ

BIG DATA DEVELOPER (SPARK/SCALA)

Responsibilities:

• Working with Business team and come up functional specification on data integration requirements
• Export data from RDBMS to HDFS using Sqoop import options
• Configured Oozie work flows to automate the data flow
• Implement Spark best practices by invoking advanced Spark API’s such as Dataframes and Datasets
• Monitor Spark tasks on each and every transformation and action via Spark UI and plan partition utilization accordingly for effective performance.
• Provided design recommendations and thought leadership to stakeholders that improved review processes and resolved technical problems

Environment: HDFS, Sqoop, Oozie, Scala and Spark

Confidential, WILMINGTON, DE

Analytical Risk Mitigation and Observational Reporting

Responsibilities:

• Work closely with legacy system owners, gather requirements and propose an equivalent big data design
• Effective utilization of spark Dataframes and memory caching techniques for processing gigabytes of data
• Hands on experience in Scala programming involving the usage of case classes, key value pairs, application configurations and effective memory handling using sql context.
• Follow best coding practices to achieve spark process parallelization to achieve better performance compared to legacy Mainframe/Oracle systems.
• Monitor Spark tasks on each and every transformation and action via Spark UI and plan partition utilization accordingly for effective performance.
• Provide design recommendations, quality analysis and thorough review followed by technical documentation

Environment: HDFS, Sqoop, Oozie, Scala and Spark

Confidential, WILMINGTON, DE

BIG DATA CONSULTANT

Responsibilities:

• Involved in business requirement analysis. Working with Business Analysis team and users to understand data integration requirements
• Imported data from RDBMS to HDFS using Sqoop import/export options
• Create Hive Internal tables with appropriate partitioning and bucketing and write complex queries for data Analysis
• Develop simple user defined functions using Eclipse Neon to access Hive scripts
• Performing Data quality checks after data ingestion
• Configured Oozie work flows to automate data flow
• Developed Hadoop jobs using Java and Hive for data processing
• Implemented best practices like partitioner, combiner and caching

Environment: HDFS, Pig, Hive, Sqoop, Oozie

Confidential, PHILADELPHIA, PA

.NET DEVELOPER

Responsibilities:

• Involved in Knowledge transition and playback sessions with customer.
• Involved in requirement gathering and preparation of SRS.
• Involved in Detailed Design preparation.
• Involved in estimation and resource plan
• Involved in development and Unit testing, System testing.
• Involved in defect fixing and supporting the System testing phase.

Environment: ASP.NET, C#, SQL SERVER 2005, WINDOWS XP/2000

Confidential

.NET DEVELOPER

Responsibilities:

• Develop adhering to the application architecture following best coding practices
• Design and Develop base claases and methods to perform authentication and authorization based on the logged in user credentials across the application modules
• Develop base classes and methods to perform application to database connection followed by executing stored procedures
• Design and Develop Oracle packages containing stored procedures
• Perform Unit testing followed by logging the test results

Environment: ASP.NET, C#, SQL SERVER 2005, WINDOWS XP/2000

Confidential

.NET DEVELOPER

Responsibilities:

• Assist in developing survey and authorizer module
• Design and develop functions in C# to manage head count module
• Worked as a team player enhancing smooth delivery of project adhering to best practices followed by unit testing of modules.

Environment: ASP.NET, C#, SQL SERVER 2000, WINDOWS XP/2000