SUMMARY

• Hadoop developer with 8 years of experience in IT industry including 4 years of experience in Hadoop Ecosystem and Big Data environments
• Extensive experience in development of Big Data projects using Hadoop, MapReduce, Spark, Java, Python, Hive, Impala and other open source tools/technologies
• Excellent understanding of Hadoop architecture and different daemons like Resource Manager, Node Manager, Name Node and Data Node.
• Implemented standards and processes for Hadoop based application design and implementation
• Responsible for writing MapReduce programs using Java
• Logical Implementation and interaction with HBase
• Developed MapReduce jobs to automate teh transfer of data from HBase
• Performed data analysis using Hive, Spark, and Impala
• Experience in converting SQL queries into SparkTransformations using SparkRDDs, Java, Scalaand Performed map - side joins on RDD's.
• Successfully loaded files to HDFS from Oracle and SQL Server using SQOOP
• Experience in design and development of ETL processes by using Talend, Apache NiFi
• Involved in Analyzing, designing, building &, testing highly useable de-normalized OLAP systems by using Apache NiFi
• Loaded streaming log data from various web servers into HDFS using Kafka and Flume
• Documented teh implemented processes and configurations in upgrades
• Evaluated and proposed new tools and technologies to meet teh needs of teh organization
• Experienced in using Sqoop, Zookeeper, Oozie, and Cloudera Manager
• Monitored teh cluster and has great Knowledge of Hadoop Cluster architecture
• Experience in application development using Java, JSP, JDBC, XML and front end development using HTML5, CSS and JavaScript
• Expertise in DOM manipulation using JavaScript, JQuery and AngularJS
• Expertise in Core Java, J2EE, Multithreading, JDBC, Shell Scripting and proficient in using Java API’s for application development
• Knowledge in teh concepts of Web Services, REST, JAX-RS, SOAP etc.
• Worked on Web application servers like Apache Tomcat.
• Hands on experience with NetBeans and Eclipse IDE tools.
• Excellent experience in using version control tools like GIT and GIT-GUI client tools like SourceTree and TortiseGit
• Experienced in working with AGILE and SCRUM methodology
• A good team player and a self-starter with great communication skills and proven abilities to finish tasks to meet deadlines

TECHNICAL SKILLS

Hadoop ecosystem: CDH 5.x/4.x Hadoop, MapReduce 2, Spark, Hive, Sqoop, Kafka, FlumeHDFS, Zookeeper, Solr, Impala

Databases: Oracle10g/11g, MS SQL server 2008R2/2012, NoSQL (HBase 0.94/1.x)

Programming Languages: Java, Python, PL/SQL, T-SQL

IDE’s & Utilities: Eclipse, NetBeans, PyCharm

Operating Systems: Linux, CentOS

PROFESSIONAL EXPERIENCE

Confidential, Bloomfield, CT

Spark/Hadoop Developer

Responsibilities:

• Worked on Hive and Spark to provide processed data for Business Intelligence and Machine learning
• Used teh RegEx, JSON and Avro SerDe for serialization and de-serialization packaged with Hive to parse teh contents of streamed log data
• Exported data from Relational Database Systems to HDFS and vice-versa using Sqoop
• Developed NiFiworkflow to pick up files from ftp location and move them to HDFS on daily basis
• Developed different process Workflows using Apache NiFi to Extract, Transform and Load raw data into HDFS and then process it to Hive tables
• Worked on Spark streaming using Apache Kafka for near real time data processing
• Experienced in creating Kafka producer and Kafka consumer for Spark streaming
• Wrote Spark programs in Python to read big data from HDFS using SQL Context
• Used Spark SQL to run queries and get results on data in Hive, with low latency
• Performed transformations and actions on RDDs to process teh data
• Proficient in developing data transformation and other analytical applications in Spark, Spark-SQL using Scala
• Experienced in working with DataFrames and Datasets in Spark
• Hands on experience creating and working with Spark UDFs and UDAFs using Java
• Optimized Spark functions by using broadcast variables, map partitions etc.
• Used Hive data warehouse tool to analyze teh data in HDFS and developed Hive queries
• Wrote queries in Impala to retrieve data in a faster and efficient manner
• Implemented daily workflow for extraction, processing and analysis of data with Oozie
• Worked on AGILE methodology and provided updates in daily SCRUM

Environment: CDH 5.x Hadoop, MapReduce 2, Spark, HDFS, Java, Scala, Python, Oozie, NiFi, Hive, Kafka, Sqoop, Flume, HBase 1.x

Confidential, Atlanta, GA

Hadoop Developer

Responsibilities:

• Developed Big Data Solutions dat halped teh teams to make data-driven decisions
• Involved in installing, configuring and managing Hadoop Ecosystem components like Spark, Hive, Sqoop, Kafka and Flume
• Loaded data into Hadoop from RDBMS (SQL Server and Oracle) using Sqoop for processing teh data
• Responsible for loading unstructured and semi-structured data into Hadoop cluster using Flume
• Worked on analyzing, writing Hadoop MapReduce jobs using Java API
• Responsible for troubleshooting MapReduce jobs by reviewing teh log files
• Developed MapReduce programs to cleanse and parse data in HDFS obtained from various data sources and to perform joins on teh Map side using distributed cache
• Experienced in handling Avro data files by passing schema into HDFS using Avro tools and MapReduce
• Developed Spark Programs for Batch and near Real Time Processing
• Created both managed and external tables in Hive
• Improved query performance by using partitioning and bucketing of Hive tables
• Performed bucketing on tables for which more sampling or similar queries are used
• Created internal and external tables with properly defined static and dynamic partitions for efficiency
• Implemented Hive custom UDFs using Java to do comprehensive data analysis
• Implemented authentication and authorization service using Kerberos authentication Protocol
• Exported teh business required information to RDBMS using Sqoop to make teh data available for BI team to generate reports
• Worked on Talend for Extracting, Transforming and Loading data from Oracle and SQL Server
• Experienced in using debug mode of Talend to debug a job to fix errors
• Monitored workload and job performance using Cloudera Hue

Environment: CDH 5.x Hadoop, MapReduce 2, Spark, HDFS, Java, Python, Hive, Pig, Kafka, Sqoop, Talend, Flume, HBase 1.x

Confidential

Java/J2EE Developer

Responsibilities:

• Involved in Design, Development, and Testing and Deployment phases.
• Used Spring IOC, AOP modules to integrate with application.
• Implemented MAVEN Build scripts for Build and deploy teh application and involved in deploying and shell scripting for teh system automation.
• Designed and developed several Servlets, JSPs, JAVA classes for presentation layer.
• Created stored procedures, prepared and maintain teh scripts for each custom service in Oracle10g database
• Used Hibernate as persistent layer by mapping to teh tables
• Involved in writing propagation scripts to move content from one environment to another environment
• Creating HTML Markups using responsive web design approach with HTML5, CSS3, jQuery and Angular JS
• Used AngularJS and jQuery for DOM manipulation and two-way binding
• Used MVC architecture and developed code using springs and JSP for teh view
• Used NetUI tags for better UI (User Interface) Implementation
• Used JSTL to remove scriplets in JSP’s
• Created tables, relationships, triggers and indexes for enforcing business rules
• Communicates with other internal applications via JMS messages, EJBs, and web services
• Implementing client Side validations using JavaScript and developed front end code with AJAX, HTML and CSS
• Struts Validation is used for server-side validation and struts internationalization
• Optimized teh performance of queries with modifications in T-SQL queries, removed unnecessary columns, and eliminated redundant and inconsistent data
• Testing teh code using teh JUNIT test scripts and supporting System Testing.
• Used SVN for Version Repository maintenance.

Environment: Java, J2EE, Servlets, JSP, Struts 2.0, Spring 2.0, Hibernate 3.0, JavaScript, AngularJS 1.x, Linux, AJAX, Beehive NetUI, SQL, AWS, MAVEN, Site Minder, SOAP, XML, UML, SVN, Oracle 10g, Eclipse 3.3

Confidential, Wellesley Hills, MA

Java Developer

Responsibilities:

• Responsible in gathering requirements from users and designing Use cases, Technical Design and Implementation
• Extensively worked on Spring and Hibernate Frameworks
• Worked on Front Controller, Dependency Injection, MVC, Data Access Objects and other J2EE core patterns
• Developed teh entire front end screens using AJAX, JSP, JSP Tag Libraries, CSS, HTML and JavaScript
• Used JavaScript and jQuery for client-side validations and functionalities
• Contributed significantly in applying teh MVC Design pattern using Spring
• Implemented action Form classes for data transfer and server side data validation
• Performed Unit Tests(using Junit), System Tests and Integration Tests
• Developed web services using SOAP and WSDL
• Application deployment is done in WebSphere, JBoss servers
• Used Eclipse as an IDE for developing application
• Involved in teh complete software development life cycle
• Involved in Maintenance and Bug Fixing
• Involved in unit testing and user documentation and used Log4j for creating teh logs

Environment: Java 1.5, Servlets, JSP, Java Mail API, JavaScript, HTML, Spring, XML, PL/SQL, JBoss, IBM Rational Rose.

Confidential

Java Developer

Responsibilities:

• Involved in teh Software Development Life Cycle of teh project development
• Gathered teh business requirements and converted them to technical specifications and use cases
• Created teh use cases and activity diagrams using STAR-UML
• Developed teh client side view using JavaScript, JQUERY, CSS, JSP and AJAX
• Performed client side validations by using JavaScript
• Worked on teh application development using java
• Developed JDBC commands to add and retrieve teh insurance records from teh database
• Responsible for writing SQL queries for storing and retrieving teh insurance records
• Used Eclipse for developing and debugging teh application
• Log4j was used for application logging and debugging
• Improved teh application performance significantly by 30%, using query tuning and indexing
• Wrote PL/SQL scripts to insert, modify and delete data by checking both pre-script and post-script conditions
• Created several database objects like Tables, Views, Stored Procedures, and Sequences etc.
• Generated ad hoc reports using SQL queries which consisted of complex joins to pull required data
• Wrote PL/SQL scripts to modify data by checking conditions before and after teh script is executed

Environment: Java, JavaScript, AJAX, JSP, HTML, CSS, XML, JDBC, JSON, Eclipse, PL/SQL, Apache Tomcat, STAR-UML