Bigdata Technical Lead Resume
RichmonD
PROFESSIONAL SUMMARY:
- Having 10 Years of IT experience in development of applications using Hadoop and Java Technologies.
- 4 years of experience in Bigdata technologies,Hive,Mapreduce,spark,kafka,storm,hbase
- Proficient in programming using J2SE/J2EE Technologies.
- Good Knowledge on AWS Kinesis,Lambda
- Knowledge on Microsoft Azure
- Working knowledge on Databricks cluster with SparkR(R package)
- Working experience on Python
- Working experience on SAS and R,Shiny
- Very good knowledge on Akka Streams
- Strong Knowledge on Nifi
- Good experience working with Hortonworks, MapR and Cloudera Distribution
- Having experience working in Amazon webservices(AWS)
- Strong knowledge on DOCKER and Vagrant
- Working experience in Chef
- Knowledge on Camel
- Having experience with Spark proceesingFramework such as Spark and Spark Sql.
- Experience in NoSQL databases like HBase, Cassandra, Redis and Mongodb.
- Procedural knowledge in cleansing and analyzing data using R,Spark, HiveQL, Pig Latin, and custom MapReduce programs in Java.
- Experienced in writing custom UDFs and UDAFs for extending Hive and Pig core functionalities.
- Ability to develop Pig UDF'S to pre - process the data for analysis.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems(RDBMS), Teradata and vice versa.
- Skilled in creating workflows using Oozie for cronjobs.
- Working experience on Javascript,Ajax,CSS,HTML,Angular JS,Node.js
- Experienced with Java API and REST to access HBase data.
- Hands-on experience with message brokers such as Apache Kafka
- Experience in Object Oriented Analysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns.
- Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
- Experience working with JAVA,PYTHON,SHELL, R,Spark,SQL
- Extensive experience with SQL, PL/SQL and database concepts.
- Having positive work attitude.
- Well Versed in Weblogic,PL/SQL.
- Good team player, effective communicator, self-motivated, problem solver, willing to take up new responsibilities.
- Certified in Java1.4
- Websphere message broker certified
TECHNICAL SKILLS:
Hadoop EcoSystem: HDFS, Map Reduce, Oozie, Hive, Pig, Sqoop, Flume, Zookeeper and Hbase
Machine Learning: Regression,Clustering and Classification
Streaming Analytics: Kafka,Spark Streaming,Kinesis
Programming Languages: Java, Python,R,SAS
Big Data ETL Tools: Talend,TOAD
Big Data Business Intelligence: Tableau and Tableau with R
NOSQL Databases: Cassandra, MongoDb, Hbase,Neo4j
Version Control Tools: SVN, CVS, VSS, PVCS
WORK EXPERIENCE:
Confidential, Richmond
BigData Technical Lead
Responsibilities:
- Involved in cluster setup CDH5.7 in AWS
- Integration of spark with Kafka
- Work with Line of busines people to implement bigdata solutions
- Involved in kafka broker design and implementation
- Implemented Hbase and Hive integration and exposed tables to Tableau application
- Involved in writing kafka producers and consumers
- Participate in code reviews and lead the team
Environment: Java,Cloudera,spark,scala,kafka,Chef,Jenkins,github,Hbase,SparkMlib,Tableau,Hive,Python
Confidential, Kansas
Big Data Lead
Responsibilities:
- Involved in cluster setup CDH5.2
- Integration of STORM with HBase for loading the Streaming data into the HBase Tables.
- Involved in writing Webservice using Rest API
- Work with Line of busines people to implement bigdata solutions
- Involved in kafka broker design
- Involved in integration of kafka and spark
- Involved in Hbase Data modelling
- K-Means clustering implemented to identify the patterns
- Involved in writing kafka producers and consumers
- Involved in writing code for spark engine
- Participate in code reviews and lead the team
- End to End Bigdata testing
Environment: Java,Cloudera,Spark,kafka,Hbase,Machine Learning,SparkR
Confidential
Big Data Developer and Admin
Responsibilities:
- Involved in coding to provision infrastructure in AWS using cloudera and hortonworks
- Involved in implementation of Hadoop security using Kerberos,Sentry, and LDAP
- Cluster maintenance as well as creation and removal of nodes using tools like Ganglia, Nagios, Cloudera Manager Enterprise.
- Performance tuning of Hadoop clusters and Hadoop MapReduce routines.
- Screen Hadoop cluster job performances and capacity planning
- Monitor Hadoop cluster connectivity and security
- Manage and review Hadoop log files.
- File system management and monitoring.
- HDFS support and maintenance
- Taking backup of data
- Infrastructure using AWS JAVA API
- Integration of Spark with HBase for loading the Streaming data into the HBase Tables.
- Written R scripts to clean the data in HDFS
- Used Tableau with R for predictions on the hive data
- Integration of impala with Hbase
- Develop new applications on Hadoop according to business needs, and convert existing applications to Hadoop
- Identify and implement new Hadoop technologies to mature platform and help research new big data and data warehousing technologies
- Work closely with other internal groups to optimize analytics and applications running in the Hadoop environment
- Delivering Detailed technical design and build components including data mapping, workflows and batch scripts
- Participate in code reviews and lead the team
Environment: Java,AWS,Teableau,Cloudera,Hortonworks,Hive,Cassandra,spark,STORM,kafka,Hbase,R,Shiny
Confidential
Big Data Lead
Responsibilities:
- Work with Line of busines and Data Scientists to design and implement bigdata solutions
- SQL migration and execution on Hive and Vertica environments.
- Source data loading into HDFS and Vertica environments.
- Written oozie scripts to load data into hdfs
- Result comparison between existing Db2 and Hive and Vertica by using SAS Enterprise guide
- Participate in code reviews and lead the team
Environment: Cloudera,Hive,Vertica,Db2,SAS
Confidential
Big Data Developer
Responsibilities:
- Analyzing the business requirements, design and implement a bigdata solution
- Participating in meetings with line of business
- Involved in Cluster setup and in configuration.
- Involved in Multithreading functionality which will read data from different sources at a time.
- Involved in developing Client application using Objective-C,IOS and in drawing charts using Sencha Framework.
- Involved in processing of large scale of data using Hadoop technologies like Mapreduce,Hive,Hbase and etc.
- Involved in creating Hive tables, loading with data and writing hive queries
- Involved in data ingestion into HDFS using Scoop and Flume from variety of sources
- Involved in code review.
Environment: MapReduce,Hive,Hbase,Java,IOS,Objective-C,sqoop,flume
Confidential
Big Data Developer
Responsibilities:
- Involved in Cluster setup and configuration
- Worked on data analytics using Pig and Hive on Hadoop.
- Evaluated Oozie for workflow orchestration in the automation of MapReduce jobs, Pig and Hive jobs.
- Involved in developing Client application using Objective-C,IOS and in drawing charts using Sencha Framework.
Environment: Java,Pentaho Mapreduce,Hive,Hbase,Mysql,ios,Flume.
Confidential
Java Developer
Responsibilities:
- Involved in Writing Webservice to enable communication with Peoplesoft.
- Involved in developing Client application using Objective-C,ios.
- Involved in UI Design.
- Code review
Environment: Mac iPad,ios,Java,peoplesoft,ExtJS,HTMl5,CSS,JAVASCRIPT
Confidential
Java Developer
Responsibilities:
- Design.
- Development.
- Code Review.
Environment: Java,Struts,Spring,Hibernate,Ajax
