- 10+ years of experience in IT industry as a Software developer, Technical lead, Solution Architecture and Project Manager in various domains
- More than 2 years of client facing experience in Europe (Finland & Belgium)
- 3+ years of experience on Big Data ecosystems Hadoop, MapReduce, HDFS, Yarn, Hbase, Hive, Pig, Sqoop, Flume, Spark, Kafka, Oozie, Hue, Zookeeper, Ambari, Hortonworks, MapR, AWS
- Hands on experience in creating real - time data streaming solutions using Apache Spark, Kafka, HBase, Spark Streaming API(pySpark) & Kafka API
- Experienced working with different file formats - Avro, Sequence and JSON
- Good understanding on building Big Data/Hadoop applications using AWS Services like Amazon S3, EMRFS, EMR, RDS, Airflow etc...
- Certified Scrum master
- Hands on experience in Object oriented analysis, design and programming with C++, Java & Python
- Strong domain knowledge in Telecom, GIS, Oil and Gas, Marine and Electrical
- Hands on experience in SQL and PL/SQL
- Experience in using Custom distributions like Hortonworks, MapR and Cloudera
- Excellent in technical & organization skills, communication skills & Collaboration skills.
Big Data Ecosystem: Hadoop, MapReduce, HDFS, Yarn, Hbase, Hive, Pig, Flume, Kafka, Oozie, Hue, Spark, Ambari, Zookeeper, Hortonworks, MapR
Programming Languages&IDEs: C++, VC++, Java, Python, Scala, Eclipse, Visual Studio
Methodologies: Agile, UML, Design Patterns
APIs: ObjectARX (AutoCAD), Real DWG, Google Maps API, MFC
Operating System: Windows, Linux
Database: Oracle, SQL, PL/SQL
Software Applications: AutoCAD, Confidential NET Workhub & Dashboard, GeoServer, TribonM3
Versioning Control: VSS, StarTeam, TFS
Cloud Infrastructure: Amazon Web Services (AWS), EMR, S3, Dynamo DB etc...
Others: Enterprise Architect, Visio & MS Project
Senior Hadoop Developer/Big Data Solution Architect
- Closely work with the customers to understand the Challenges and requirements
- Analyze the customer data to provide a solutions
- Installed and configured Apache Hadoop MapReduce, HDFS, developed multiple MapReduce jobs in Java for data cleaning and preprocessing
- Loading the customer data into HDFS using Hadoop commands
- Processing and analyze the data using MapReduce jobs
- Configure Apache Kafka producer and consumer coding part in java to establish connection from data sources and loaded into HDFS
- Created HIVE tables to store the processed results in tabular format
- Written PIG scripts to process the HDFS data
- Worked on pulling the data from MySQL database into HDFS using Sqoop
- Developed Web application for Data Analysis in reporting format using HTML, CSS, JQuery
- Responsible for Hadoop Admin related tasks like debugging, performance fine-tuning and monitoring for day-to-day activities
- Designed and implemented Apache Spark streaming application using Python(pySpark) and Scala
- Load and transform large sets(petabytes) of structured, semi-structured and unstructured data
- Strong expertise in writing and implementing Pig/Hive scripts, UDFs
- Experience in defining and coordination of job flows
- Good experience in NoSQL database like Hbase
- Responsibility to manage data coming from different sources
- Responsibility to write and implement python & Scala coding for Spark implementations
- Responsible to implement the Spark applications on AWS
- Write a integration tool in python to connect to Confidential NET Workhub and Dashboard from Hadoop ecosystem
- Involved in the loading data from Linux file system to HDFS
- Responsible for managing systems on AWS platforms
- Automated all jobs in Linux shell scripting
- As a technical leader, ensure that on-time delivery of the agreed delivery
- Ensuring that the development process is properly focused and controlled
- Managing risks and issues at the development with in estimated time, escalating to project manager as required
- Running the daily meetings ensuring they are timely, focused and brief
Environment: Java, Hadoop, HDFS, Hive, Pig, Flume, Sqoop, Hbase, Oozie, Spark, Kafka, Zookeeper, Ambari, Hue, Python, Linux, Eclipse, Hortonworks, AWS
- Gathered requirements and design the application
- Installation and Configuration of GeoServer and Hadoop ecosystem tools
- Installation and Configuration of Tomcat for Development and Testing environment
- Loading of Shape files into GeoServer
- Loading the GIS data into HDFS
- Used MapReduce framework to identify Delta updates and to refresh persistent files
- PIG and HIVE extensively used for data analyses
- Responsible to manage GIS data coming from different sources
- Developed PIG scripts in the areas where extensive coding needs to be reduced
- Experienced in MapReduce programs to load the data from application generated log file to Hbase database
- Write a reusable and tested code in python for Spark Implementation
- Extensively used Python in Big Data implementations
- Created Hbase tables to store variable data formats of data coming from several data sources
- Analyzing the data with Hive, Ping and Hadoop streaming
- Responsible for monitoring Hadoop cluster, manage and review logs, performance tuning etc...
- Implemented RESTful scripting for Querying on Shape files
- Deployed the application in Development and Testing environments
- Coordinated the Onsite and Offshore teams for smooth implementation
- Implemented Agile process for this project
- Give training and mentorship to team members to make them better on the job
- Review the completed tasks to ascertain compliance with standards
- Coach all team members and provide necessary advice and guidance
- Write and forward regular reports to the management
- Perform regular appraisal of team member’s performance to help with improvement
Team Lead & Techno Functional Architect
- Requirements capturing and Analysis of the requirements
- Functional design and Technical design of the requirements
- Resource allocation and assigning the tasks to team members
- Creating the required classes for the development
- Responsibility of Quality of output to customer.
- Developed use cases in VC++, ObjectARX
- Implemented AutoCAD migration project from AutoCAD 2006 to AutoCAD 2012
- Implemented AutoCAD 64-bit project from AutoCAD 32-bit to AutoCAD 64-bit
- Embedded Google Maps into GDA application for interactive selection of area
- Setting up of Development(Dev), Integration (ITT), acceptance(UAT) environments for application deployment and testing
- Weekly meetings with customer regarding the health of application and current project activities.
Environment: VC++, ObjectARX, MFC, Oracle, SQL, PL/SQL, Enterprise Architect, VSS, AutoCAD Map 3D 2006/2012
Object ARX Developer
- Implemented use cases in VC++ and Object ARX.
- Prepared User-friendly interfaces using MFC.
- Implementation of various PL/SQL procedures.
- Used Visual Source safe for doing proper Versioning Control.
- Unit testing and integration testing.
- Prepared technical design documentation for use cases.
Environment: VC++, ObjectARX, MFC, Oracle, SQL, PL/SQL, Enterprise Architect, VSS, AutoCAD Map 3D 2006
- Capturing requirements from client
- Understanding the requirements and doing functional design
- Implementation of Technical design for software protection
- Developed the application using VC++
- Implemented Software Protection using Aladdin HASP software’s
- Module test design for the project
- Unit testing and integration testing.
- Prepared user-friendly interfaces using MFC.
Environment: VC++, Aladdin HASP, Codejack xtreme toolkit, Visual Studio, VSS