We provide IT Staff Augmentation Services!

Senior Hadoop Developer/big Data Solution Architect Resume

SUMMARY:

  • 10+ years of experience in IT industry as a Software developer, Technical lead, Solution Architecture and Project Manager in various domains
  • More than 2 years of client facing experience in Europe (Finland & Belgium)
  • 3+ years of experience on Big Data ecosystems Hadoop, MapReduce, HDFS, Yarn, Hbase, Hive, Pig, Sqoop, Flume, Spark, Kafka, Oozie, Hue, Zookeeper, Ambari, Hortonworks, MapR, AWS
  • Hands on experience in creating real - time data streaming solutions using Apache Spark, Kafka, HBase, Spark Streaming API(pySpark) & Kafka API
  • Experienced working with different file formats - Avro, Sequence and JSON
  • Good understanding on building Big Data/Hadoop applications using AWS Services like Amazon S3, EMRFS, EMR, RDS, Airflow etc...
  • Certified Scrum master
  • Hands on experience in Object oriented analysis, design and programming with C++, Java & Python
  • Strong domain knowledge in Telecom, GIS, Oil and Gas, Marine and Electrical
  • Hands on experience in developing the GIS applications & CAD application customization using GeoServer, HTML, CSS, JQuery, AngularJS, JavaScript, AutoCAD, ObjectARX & RealDWG.
  • Hands on experience in SQL and PL/SQL
  • Experience in using Custom distributions like Hortonworks, MapR and Cloudera
  • Excellent in technical & organization skills, communication skills & Collaboration skills.

TECHNICAL SKILLS:

Big Data Ecosystem: Hadoop, MapReduce, HDFS, Yarn, Hbase, Hive, Pig, Flume, Kafka, Oozie, Hue, Spark, Ambari, Zookeeper, Hortonworks, MapR

Programming Languages&IDEs: C++, VC++, Java, Python, Scala, Eclipse, Visual Studio

Web Technologies: HTML, XML, CSS, JavaScript, JQuery, AngularJS

Methodologies: Agile, UML, Design Patterns

APIs: ObjectARX (AutoCAD), Real DWG, Google Maps API, MFC

Operating System: Windows, Linux

Database: Oracle, SQL, PL/SQL

Software Applications: AutoCAD, Confidential NET Workhub & Dashboard, GeoServer, TribonM3

Versioning Control: VSS, StarTeam, TFS

Cloud Infrastructure: Amazon Web Services (AWS), EMR, S3, Dynamo DB etc...

Others: Enterprise Architect, Visio & MS Project

PROFESSIONAL EXPERIENCE:

Confidential

Senior Hadoop Developer/Big Data Solution Architect

Responsibilities:

  • Closely work with the customers to understand the Challenges and requirements
  • Analyze the customer data to provide a solutions
  • Installed and configured Apache Hadoop MapReduce, HDFS, developed multiple MapReduce jobs in Java for data cleaning and preprocessing
  • Loading the customer data into HDFS using Hadoop commands
  • Processing and analyze the data using MapReduce jobs
  • Configure Apache Kafka producer and consumer coding part in java to establish connection from data sources and loaded into HDFS
  • Created HIVE tables to store the processed results in tabular format
  • Written PIG scripts to process the HDFS data
  • Worked on pulling the data from MySQL database into HDFS using Sqoop
  • Developed Web application for Data Analysis in reporting format using HTML, CSS, JQuery
  • Responsible for Hadoop Admin related tasks like debugging, performance fine-tuning and monitoring for day-to-day activities
  • Designed and implemented Apache Spark streaming application using Python(pySpark) and Scala
  • Load and transform large sets(petabytes) of structured, semi-structured and unstructured data
  • Strong expertise in writing and implementing Pig/Hive scripts, UDFs
  • Experience in defining and coordination of job flows
  • Good experience in NoSQL database like Hbase
  • Responsibility to manage data coming from different sources
  • Responsibility to write and implement python & Scala coding for Spark implementations
  • Responsible to implement the Spark applications on AWS
  • Write a integration tool in python to connect to Confidential NET Workhub and Dashboard from Hadoop ecosystem
  • Involved in the loading data from Linux file system to HDFS
  • Responsible for managing systems on AWS platforms
  • Automated all jobs in Linux shell scripting
  • As a technical leader, ensure that on-time delivery of the agreed delivery
  • Ensuring that the development process is properly focused and controlled
  • Managing risks and issues at the development with in estimated time, escalating to project manager as required
  • Running the daily meetings ensuring they are timely, focused and brief

Environment: Java, Hadoop, HDFS, Hive, Pig, Flume, Sqoop, Hbase, Oozie, Spark, Kafka, Zookeeper, Ambari, Hue, Python, Linux, Eclipse, Hortonworks, AWS

Confidential

Hadoop Developer

Responsibilities:

  • Gathered requirements and design the application
  • Installation and Configuration of GeoServer and Hadoop ecosystem tools
  • Installation and Configuration of Tomcat for Development and Testing environment
  • Loading of Shape files into GeoServer
  • Loading the GIS data into HDFS
  • Used MapReduce framework to identify Delta updates and to refresh persistent files
  • PIG and HIVE extensively used for data analyses
  • Responsible to manage GIS data coming from different sources
  • Developed PIG scripts in the areas where extensive coding needs to be reduced
  • Experienced in MapReduce programs to load the data from application generated log file to Hbase database
  • Write a reusable and tested code in python for Spark Implementation
  • Extensively used Python in Big Data implementations
  • Created Hbase tables to store variable data formats of data coming from several data sources
  • Analyzing the data with Hive, Ping and Hadoop streaming
  • Responsible for monitoring Hadoop cluster, manage and review logs, performance tuning etc...
  • Developed Coverage Maps Web application on top of Google Maps using HTML, CSS, JQuery, JavaScript
  • Implemented RESTful scripting for Querying on Shape files
  • Deployed the application in Development and Testing environments
  • Coordinated the Onsite and Offshore teams for smooth implementation
  • Implemented Agile process for this project
  • Give training and mentorship to team members to make them better on the job
  • Review the completed tasks to ascertain compliance with standards
  • Coach all team members and provide necessary advice and guidance
  • Write and forward regular reports to the management
  • Perform regular appraisal of team member’s performance to help with improvement

Environment: HTML, CSS, JavaScript, JQuery, Apache Tomcat, Linux, GeoServer, Eclipse, Java, Hadoop, HDFS, Hive, Pig, Flume, Oozie, Zookeeper, Linux, Hue, Ambari, Hortonworks

Confidential

Team Lead & Techno Functional Architect

Responsibilities:

  • Requirements capturing and Analysis of the requirements
  • Functional design and Technical design of the requirements
  • Resource allocation and assigning the tasks to team members
  • Creating the required classes for the development
  • Responsibility of Quality of output to customer.
  • Developed use cases in VC++, ObjectARX
  • Implemented AutoCAD migration project from AutoCAD 2006 to AutoCAD 2012
  • Implemented AutoCAD 64-bit project from AutoCAD 32-bit to AutoCAD 64-bit
  • Embedded Google Maps into GDA application for interactive selection of area
  • Setting up of Development(Dev), Integration (ITT), acceptance(UAT) environments for application deployment and testing
  • Weekly meetings with customer regarding the health of application and current project activities.

Environment: VC++, ObjectARX, MFC, Oracle, SQL, PL/SQL, Enterprise Architect, VSS, AutoCAD Map 3D 2006/2012

Confidential

Object ARX Developer

Responsibilities:

  • Implemented use cases in VC++ and Object ARX.
  • Prepared User-friendly interfaces using MFC.
  • Implementation of various PL/SQL procedures.
  • Used Visual Source safe for doing proper Versioning Control.
  • Unit testing and integration testing.
  • Prepared technical design documentation for use cases.

Environment: VC++, ObjectARX, MFC, Oracle, SQL, PL/SQL, Enterprise Architect, VSS, AutoCAD Map 3D 2006

Confidential

VC++ Developer

Responsibilities:

  • Capturing requirements from client
  • Understanding the requirements and doing functional design
  • Implementation of Technical design for software protection
  • Developed the application using VC++
  • Implemented Software Protection using Aladdin HASP software’s
  • Module test design for the project
  • Unit testing and integration testing.
  • Prepared user-friendly interfaces using MFC.

Environment: VC++, Aladdin HASP, Codejack xtreme toolkit, Visual Studio, VSS

Hire Now