Sr. Big Data Developer Resume
Chicago, IL
PROFESSIONAL SUMMARY
- Around 9+ years of IT Experience in RDBMS, SQL and PL/SQL Programming and Big Data technologies.
- Around 5+ years of experience as a Designer & quality reviewer wif cross platform integration experience using Hadoop, Java and J2EE
- Qualified and certified Hadoop, Java Developer wif good experience in Analytics, Design and Development and maintaining applications for various Business enterprises.
- Experience using Cloudera and Horton Works platform and their eco systems. Hands on experience in installing, configuring and using ecosystem components like Hadoop MapReduce, HDFS, Pig, Hive, Sqoop and Flume.
- Extensive experience in data ingestion, big data storage planning, complex transformations, data integration, analysis for Pharmaceutical, Healthcare and Retail sectors.
- Experienced in Integrating Hadoop wif Apache Storm and Kafka. Expertise in uploading Click stream data from Kafka to Hdfs, Hbase and Hive by integrating wif Storm.
- Experience in Apache Spark, Spark Streaming, Spark SQL and No SQL databases like Cassandra and Hbase.
- Experience in Hadoop distributions like Amazon, Cloudera and Hortonworks and Cloud ecosystems components of Amazon like Redshift, Dynamo DB
- Hands - on experience in advanced Big-Data technologies like Spark Ecosystem (Spark SQL, SparkR and Spark Streaming, Yarn).
- Very good understanding of Hadoop ecosystems like Sqoop2, Spark and YARN.
- Extensive knowledge on Tableau reporting.
- Experienced in providing technical solutions to teh business on applications developed on Hadoop and its eco systems.
- Thrives on challenge and works well under pressure, wif technical expertise to learn new environments quickly, locate inefficiencies in code, and provide quick solutions.
- Expertise in developing web applications using JAVA, Spring MVC, vignette portal. Good experience to provide technical oversight for large complex projects and achieve desired customer satisfaction from inception to deployment.
- Experience in Data Analysis, Data Validation, Data Verification, Data Cleansing, Data Completeness and identifying data mismatch.
- Experience in working wif MR, PIG scripts &HIVE query language.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
- Extensive experience developing and deploying applications using Web Logic, Apache Tomcat and JBOSS.
- Experienced wif Upgrades, Migration and Backup, Disaster Recovery, Performance Monitoring and Fine-tuning of systems running various Linux platforms.
- Installed, monitored, and supported web and application servers in Linux environments.
- Extended Hive and Pig core functionality by writing custom UDF’s.
- Experienced analyzing data using Hive QL, Pig Latin, and custom MapReduce programs in Java.
- Extensive experience wif SQL, PL/SQL, PostgreSQL and database concepts.
- Knowledge of NoSQL and Mango DB such as HBase and Casandra.
- Knowledge of job workflow scheduling and monitoring tools like Oozie and Zookeeper
- Has advanced analytical, problem solving, negotiation and organizational skills wif demonstrated ability to multi-task, organize, prioritize and meet deadline. Ability to multitask and work multiple projects concurrently. Ability to work independently and as part of a team.
- Experience using waterfall project execution methodologies in all phases of an application development starting from Planning all teh way to teh delivery of teh Product
- Experience in implementing Spark using Scala and SparkSQL for faster analyzing and processing of data.
- Proficient in working wif Various IDE tools including Eclipse Galileo and IntelliJ IDE.
- Hands-on experience in Scala, Kafka and Strom.
- Experience in implementing Spark using Scala and SparkSQL for faster analyzing and processing of data.
- Java & J2EE Technologies, Core Java
- Knowledge of working SQL BI and Talend.
TECHNICAL SKILLS:
Big data: Hadoop, Map Reduce, HDFS, Hive, HBase, Pig, Sqoop, Flume, Oozie, Zookeeper, Flume, Netezza, Mahout, YARN, Storm, Spark, Kafka(0.8.2x, 0.9.0), Mongo DB, Cassandra, Tableau Reporting.
Hadoop Distributions: Cloudera, Hortonworks, MapR
Core Skills: Core Java (OOPs and collections), J2EE Framework, JSP, Servlets, Oracle ADF, JSF, Linux Shell Script, JDBC, Scala
Databases: Oracle, SQLServer
Design Patterns: Singleton, Factory, MVC
Build Tools: ANT, Maven
Browser Scripting: Java script, HTML DOM, DHTML, AJAX, AngularJS
IDE: Eclipse/My Eclipse, JDeveloper
Operating Systems: Red-hat Linux, Windows, UNIX
PROFESSIONAL EXPERIENCE
Confidential, Chicago, IL
Sr. Big Data Developer
Roles and Responsibilities:
- Worked directly wif teh applications that perform data loading and processing to HDMS internal tool “DART”. Develops and maintains teh process and procedures used to manage teh loading process or changes to that environment.
- Identifying and acting upon opportunities to improve teh reliability, efficiency, performance and overall service wifin teh environment.
- Worked wif different team in ETL, Data Integration and Migration to Hadoop
- Developed Talend jobs to do simple and complex enrichment of acquired data.
- Developed Pig scripts to do simple and complex enrichment of acquired data.
- Developed Sqoop framework to acquire and ingest data from SQL Server to MapR Hadoop distribution.
- Configured Spring to manage Action classes and set their dependencies in a Spring Context file and integrated middle tier wif Struts and also used EJB components for backend processing.
- Used Hibernate for mapping teh ORM objects to table using Hibernate annotations.
- Developed Web services component using XML, WSDL, and SOAP wif DOM parser to transfer and transform data between applications.
- Created and consumed RESTFul Web Services using JAX-RS(Jersey).
- Used AJAX framework for server communication and seamless user experience.
- Followed Agile Methodology (TDD, SCRUM) to satisfy teh customers and wrote Junit test cases for unit testing modules.
- Used client side Java scripting: JQUERY for designing TABS and DIALOGBOX.
- Created UNIX shell scripts to automate teh build process, to perform regular jobs like file transfers between different hosts.
- Used Log4j for logging teh output to teh files.
- As part of enriching teh data, developed custom UDFs and utilized them in Pig scripts.
- Created Hive external tables as part of teh requirements on enriched data.
- Implemented historical data load routine using Pig scripts and load teh updated data into Hive tables.
- Designing & Creating ETL Jobs through Talend to load huge volumes of data from flat files into Hadoop Ecosystem and relational databases.
- Performed Installation of Hadoop in fully and Pseudo Distributed Mode for POC in early stages of teh project.
- Processed large data sets utilizing Hadoop cluster. Teh data that are stored on HDFS were preprocessed/validated using TALEND tan teh processed data is stored into Hive warehouse, which enabled business analysts to get teh required data from Hive.
- Generated Data Quality reports for teh business users.
- Used different file formats like Text files, Sequence Files.
- Assisted in creating and maintaining Technical documentation to launching HADOOP Clusters and even for executing Hive queries and Pig Scripts.
- Extensively worked wif teh Admin team in job deployments and scheduling teh jobs.
- Worked on JENKINS for auto code build and deployments.
- Created Tech Specific documentation for all teh process and jobs developed.
- Implemented error handling and error reporting features in teh developed TALEND jobs.
- Created and developed ECG file Transfer Linux Shell Scripts and used them wif TALEND jobs.
- Worked on UNIX and Linux environment to work on teh data files received from various clients, developed shell scripts, and generated reports on top of it.
Environment: Hadoop, Sqoop, Hive, Oozie, Pig, Hbase, Scala, Zookeeper, Talend Open Studio, Oracle, Apache Cassandra, SQL Server 2008, MySQL, Java, SQL, PLSQL, Toad, Eclipse Kepler IDE, Dart reporting, Microsoft Office 2007, SharePoint Teamsite.
Confidential - Oak Brook, IL
Sr. Java/Hadoop Developer
Roles and Responsibilities:
- Worked on importing and exporting data from Oracle and DB2 into HDFS using Sqoop.
- Developed PIG scripts to extract teh data from teh webserver output files to load into HDFS.
- Worked on custom Pig Loaders and Storage classes to work wif a variety of data formats such as JSON, Compressed CSV etc.
- Monitored workload, job performance and capacity planning using Cloudera Manager.
- Worked wif Cloudera distribution of Hadoop.
- Deployed Hadoop Cluster worked on Managing and scheduling jobs on Hadoop cluster.
- Involved in analyzing system failures, identifying root causes and recommended course of actions.
- Adopted J2EE design patterns like Session Facade and Business Facade.
- Configuration of application using spring 2.6, Struts 1.3, Hibernate, DAO’s, Actions Classes, Java Server Pages.
- Configuring Hibernate Struts and Tiles related XML files.
- Developed teh application using Struts Framework that uses Model View, Controller (MVC) architecture wif JSP as teh view.
- Developed presentation layer using JSF, JSP, HTML and CSS, JQuery.
- Extensively used Spring IOC for Dependency Injection and worked on Custom
- MVC Frameworks loosely based on Struts.
- Worked on Hive for exposing data for further analysis and for generating transforming files from different analytical formats to text files.
- Handled importing of data from various data sources, performed transformations using Hive, Pig and Spark and loaded data into HDFS.
- Involved in NoSQL (Datastax Cassandra) database design, integration and implementation.
- Wrote queries to create, alter, insert and delete elements from lists, sets and maps in Datastax Cassandra.
- Created indices for conditioned search in Datastax Cassandra.
- Developed multiple MapReduce jobs in java for data cleaning and accessing.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Worked on teh core and SparkSQL modules of Spark extensively.
- Implemented name node backup using NFS for High availability.
- Created Hive External tables and loaded teh data in to tables and query data using HQL.
- Wrote shell scripts to automate document indexing to Cloud in production.
- Created Talend Mappings to populate teh data into dimensions and fact tables
- Developed jobs to move inbound files to vendor server location based on monthly, weekly and daily frequency in Talend.
- Created Hbase tables to store various data formats of PII data coming from different portfolios.
- Cluster co-ordination services through Zookeeper.
- Used Flume to collect, aggregate, and store teh web log data from different sources like web servers, mobile and network devices and pushed to HDFS.
- Worked for Amazon Elastic Cloud project using Agile methodology.
- Created reports for teh business requirements using Tableau.
- Analyzed teh web log data using teh HiveQL to extract number of unique visitors per day, page-views, visit duration, most purchased product on website.
- Converting teh Oracle table components to Teradata Table Components in Abilities Graphs.
- Used Ambary to manage provision and monitorHadoopcluster.
- Implemented Fair schedulers on teh Job tracker to share teh resources of teh Cluster for teh Map Reduce jobs given by teh users.
Environment:Hadoop, MapReduce, HDFS, Hive, Java, SQL, Cloudera Manager, Spark, AWS, Cassandra, Pig, Sqoop, Oozie, Zookeeper, Ambari, Storm, Teradata, Oracle, NoSQL, Elastic Search, Oozie, Hbase, Talend, Tableau.
Confidential
Java/Hadoop Developer
Roles and Responsibilities:
- Developer Hadoop ecosystem: Hadoop, MapReduce, Hbase, Sqoop, Amazon Elastic Map Reduce (EMR)
- Developed a scalable, cost effective, and fault tolerant data ware house system on Amazon EC2 Cloud.
- Developed MapReduce/EMR jobs to analyze teh data and provide heuristics and reports. Teh heuristics were used for improving campaign targeting and efficiency
- Response time for web services built on typical LAMP (php) stack was too slow developed a high performant / high volume / highly scalable platform for bidding in real-time understand from teh client teh extraction process and decide on teh load strategy me.e. whether they want historical data or teh current view
- Involved in multi-tiered J2EE design utilizing spring framework and JDBC.
- Worked on building a system using Model-View-Controller (MVC) architecture.
- Designed teh front end using HTML, CSS, Java Script, JSP, jQuery.
- Designed and implemented teh application using Spring MVC, JDBC, MYSQL.
- Written complex HSQL’s to generate data required in teh final reports and pass these HSQL’s to teh Ruby programs to convert these HSQL’s to map Reduce programs
- Importing, exporting data into HDFS and HIVE using Sqoop
- Responsible for loading unstructured data into Hadoop file system (HDFS)
- Created and scheduled jobs for maintenance
- Configured Database Mail
- Monitored File Growth
- Maintained Operators, Categories, Alerts, Notifications, Jobs and Schedules.
- Maintained database response times, proactively generated performance reports.
- Automated most of teh DBA Tasks and Monitoring stats
- Developed complex stored procedures, views, clustered/non-clustered indexes, triggers (DDL, DML, LOGON) and user defined functions
- Created a mirrored database using Database Mirroring wif High Performance Mode
- Created database snapshots and stored procedures to load data from teh snapshot database to teh report database
- Restore Development and Staging databases from production as per teh requirement
- Involved in resolving Dead lock issues and Performance issues
- Query Optimization and Performance Tuning for long running queries and created new indexes on tables for faster me/O
Environment: MS SQL Server 2005/2000, Windows 2000/2003 Server, DTS, Web Logic, Redhat Enterprise MS Access, XML, Hadoop, MapReduce, Hbase, Sqoop, Amazon Elastic Map Reduce CDH, Cassandra, NOSQL,Teradata
Confidential
Java/J2EE Developer
Roles and Responsibilities:
- Involved in multi-tiered J2EE design utilizing spring framework and JDBC.
- Worked on building a system using Model-View-Controller (MVC) architecture.
- Designed teh front end using HTML, CSS, Java Script, JSP, jQuery.
- Designed and implemented teh application using Spring MVC, JDBC, MYSQL.
- Used SVN version control tool.
- Automated teh build process by writing Maven build scripts
- Wrote SQL queries, stored procedures, modifications to existing database structure as required for addition of new features using MySQL database
- Involved in installing and configuring Eclipse for development
- Configured and customized logs using Log4J and unit testing using Junit.
- Developed JUnit Test cases and performed application testing for QC team.
- Used JavaScript for client side validations.
- Participated in weekly project meetings, updates and Provided Estimates for teh assigned Task.
Environment: Java, J2EE, JavaScript, JDBC, Spring, ASP.NET, VB.NET, AGILE - SCRUM, JSP, Servlet, XML, Design Patterns, Log4J, JUnit, SVN, MySQL, Eclipse.
