Hadoop Developer/Spark Developer Resume MA-Boston - Hire IT People

PROFESSIONAL PROFILE:

Accomplished 5 years and high performing expert with deep expertise in the application of Bigdata technologies and JAVA/J2EE.
Strong exposure on streaming data processing using Kafka, Flume and Spark.
Hadoop Developer with good knowledge on ecosystems like Hive, Pig, sqoop, MapReduce, oozie, Spark Core, Spark SQL, Spark MLlib, Impala.
Good working knowledge of Amazon Web Service components like EC2, EMR, S3, EBS, ELB.
Experience in Apache Spark and Scala.
Experience in working with Linux Centos and windows operating systems.
Good knowledge in NOSQL databases like HBASE.
Good knowledge in Machine Learning Algorithms.
Good knowledge in ETL and BI.
Skilled in the conception and execution of strategic plans, and the architecture of associated tactics including
performance benchmarking against key operational targets/goals
Creative problem solver with a unique mix of technical, business, and research proficiency that lends itself to
Developing key strategies and world - class solutions with significant impact on revenue and ROI.
Excellent written, verbal, interpersonal communicator and negotiator.
Highly committed to a personal and management style embracing continuous learning and improvement.

COMPUTER SKILLS:

Operating System: Windows, Linux, UNIX.

Big data Technologies: HDFS, Yarn, Hive, Pig, Map Reduce, Spark, Sqoop, HBase, Zookeeper, Flume, Oozie, Kafka, Hue.

Languages: Java, Python, Scala, UNIX shell scripting, Hiveql, Pig scripts.

Database : MY SQL, NoSQL, Oracle, DB2.

SDLCs: Agile, Scrum.

Tools: Quality Center, SFTP, ECLIPSE, MAVEN, PUTTY, SSH, SQL ASSISTANT.

PROFESSIONAL EXPERIENCE

Hadoop Developer/Spark Developer

Confidential, MA-Boston

Responsibilities:

Worked on loading different kinds of data from different feeds.
Validate the data before loading to HDFS using checksum algorithms.
Developed SPARK applications to load files like XML and JSON to HDFS.
Developed preprocessor to load and convert the DAT file to CSV file.
Developed SPARK Applications in Scala and build them using MAVEN.
Worked on flattening the files for specific feeds using DATAFRAMES.
Used Spark-SQL to perform Transformations and Actions on data residing in HIVE.
Worked on creating UNIX shell scripts to modify the data before loading to HDFS.
Developed Scala scripts, UDAFs with Data frames in SPARK 2.0 for Data Aggregation queries.
Developed SPARK (SCALA) applications which includes Continuous Integration and Delivery (CIAD).
Handled large datasets using Partitions, SPARK in Memory capabilities, Broadcasts in SPARK.
Worked on getting Hadoop access from Windows to run application locally.
Configured Cygwin to run LINUX commands in windows.

Hadoop Developer

Confidential, TX-Dallas

Responsibilities:

Create the project using HIVE, PIG and SPARK.
Created Java UDFs in PIG and HIVE.
Implemented Partitioning, Bucketing in HIVE.
Involved in data modeling in Hadoop.
Creating Hive tables and working on them using Hiveql.
Written Apache PIG scripts to process the HDFS data.
Experience in using Sequence files, RCFile, Parquet, AVRO and ORC file formats.
Participated in backup and recovery of Hadoop file system.
Automated tasks using UNIX shell scripts.
Ingesting real time streaming data with KAFKA and SPARK streaming as a receiver.
Came up with estimations and Technical Design Specifications for projects.
Gathered the business requirements from the Business Partners and Subject Matter Experts.

JAVA Developer

Confidential

Responsibilities:

Developed the application by using the Spring MVC framework.
Developed proto-type test screens in HTML and JavaScript.
Developed user interfaces using JSP, HTML, XML and JavaScript.
Involved in developing JSP for client data presentation and data validation on the client side.
Prepare UNIX shell scripts for testing the application.
Created connection through JDBC.
Creating JDBC statements to call stored procedures.
Developed JUnit testing framework for Unit level testing.
Developed proto-type test screens in HTML and JavaScript.
Involved in various phases of Software Development Life Cycle (SDLC) as design development and unit testing.

JAVA Developer

Confidential

Responsibilities:

Used JavaScript, JQuery to perform client side validations.
Batch module to load data to tables from external sources.
Created Use case, Sequence diagrams, functional specifications and User Interface diagrams.
Created Data sources and Helper classes which will be utilized by all the interfaces to access the data and manipulate the data.
Involved in Bug fixing, Documentation, Deployments and Production Support.