Big Data Developer Resume
San Bruno, CA
SUMMARY
- 7+ years of working experience and expertise in Analysis, Design, Development, Deployment and Implementation of Web and Agile Enterprise applicationsandClient/Server architecture.
- 2+ years of experience in Hadoop’s ecosystem implementation and development of Big Data applications.
- Experience in Hadoop ecosystem components like Map Reduce, HDFS, Hive, Sqoop, Pig,Kafka for scalability, distributed computing and high performance computing.
- Experience in developing of custom Map - Reduce programs.
- Strong experience in data analytics using Hive and Pig, including by writing custom UDFs.
- Expertise in working with Flumein configuring and working with Kafka to load the data from multiple sources directly into HDFS.
- Hands on experience on working with Hadoop Database, HBASE, Redis store and developing Storm topologies for real-time computation.
- In depth knowledge of Hadoop Architecture and various components such as HDFS, JobTracker, Task Tracker, Name Node, Data Node and MRv1 and MRv2 (YARN).
- Working knowledge of Spark features including Core Spark, SparkSQL, Spark Streaming.
- Knowledge of job workflow scheduling and monitoring tools like Oozie, UC4 andZookeeper.
- Extensive knowledge in Oracle PL/SQL, Teradata, MySQLandexperience inLinux/Unix shell script.
- Experience in developing Web and client-server applications using JAVA/J2EE technologies like JSP, Servletswith variousopen source framework like Struts, Spring, Hibernate.
- Experience in Core Java and multi-thread processing.
- Worked with creation and consumption of SOAP based &Restful web services using WSDL,SOAP, JAX-WS, JAX-RS, SOAP UI and Rest client.
- Experience with XML technologies like XML, XSD, XSLT and experience onSVN, Github.
- Strong knowledge of J2EE design patterns like MVC, Session Facade, Business Delegate, Front Controller, Service Locator, Data Transfer Objects and Data Access Objects etc.
- Good working knowledge on build tools like Maven,Ant for project build/test/deployment,Log4j forerror logging and Debugging, JUnit forunit and integration testing.
- Knowledge about SDLC and methodologieslike Agile, SCRUM.
- Developed and deployed applications onUnixand Windows platforms.
- Ability to perform at a high level, meet deadlines with quality delivery, adaptable to ever changing priorities.
- Have great motivation to learn new skills/technologies, excellent analytical/problem-solving skills, fast-learner, resourceful, committed, hard-worker, and self-initiative.
TECHNICAL SKILLS
Big Data Skills: HDFS, MapReduce,Hive,HBase,Spark,Kafka,Storm, Redis, Flume.
Programming Languages: Java, Linux shell scripts, Python, Scala
J2EE Technologies: Servlets,JSP,Web-Services
Web Technologies: JSP,HTML,CSS, Java Script
Frameworks: Struts,Spring, Hibernate
Web Services: SOAP, REST
IDE Tools: Eclipse, IBM Websphere, NetBeans.
Application Servers: IBM WebSphere, WebLogic,Tomcat,JBoss
Databases: Oracle 11g/10g/9i, MySQL, DB2, MS-SQL Server
Testing Tools: JUnit
Operating Systems: All Versions of Microsoft Windows, UNIX and LINUX.
PROFESSIONAL EXPERIENCE
Big Data Developer
Confidential, San Bruno, CA
Responsibilities:
- Involved in analyzing data to extract targeted customers required for specific campaigns using Hive based on transactions, user events like clicks/opens, browse information.
- Developed Map-Reduceprograms in both Python and Java for processing the extracted customer data.
- Developed MR jobs for bulk insertion of Walmart’s customer and item data from files to HBASE.
- Created shell scripts for automating the process of extracting and loading targeted customers into Hive tables on daily basis to whom various email campaigns are sent.
- Worked on various email campaigns like Back-In-Stock, Price-Drop, Post-Browse, Customer Ratings and Reviews, Shopping Cart Abandon etc.
- Developed and deployed HiveUDF’swritten in Java for encrypting customer-id’s, creating item-image-URL’s etc.
- Worked on StrongView tool for scheduling and monitoring Batch email campaigns.
- Extracted StrongView logs from servers using Flume and extracted information like open/click info of customers and loaded into Hive tables. Created reports for getting counts of emails sends, opens, clicks.
- Created marketing reports for various campaigns open/click info of customers.
- Written Shell scripts for automation of Hive query processing.
- Scheduled Map-Reduce and Hive workflows using Oozie.
- Developed HTMLtemplates for various trigger campaigns.
- Worked on Oracle PL/SQL and Teradata some extract and loading data.
- Analyzed transaction data and extracted category wise best-selling items info which is used by marketing team to come up with ideas for new campaigns.
- Developed complex Hive queries using Joinsand automated these jobs using Shell scripts.
- Monitored and debugged Map-Reducejobs using the Job-tracker administration page.
- Developed Storm Topologies for real time email campaigns where Kafka is used as source for getting customer’s website activity information and storing data into Redisserver.
- Involved in migrating existing Hive jobs to Spark SQL environment.
- Used Spark Streaming API for consuming data from Kafka source and processed data with core Spark functions written in Scala and then stored resultant data in HBase table which is later used for generating reports
- Developed data pipeline to ingest data from Kafka source into HDFSas sink using Flumewhich is used for analysis.
- Developed REST webservices for providing metadata information required for the campaigns.
Environment: MapReduce,Hive,HBASE,Java,Storm,Scala,SparkStreaming,SparkSQL,Redis,Oozie,Kafka,Flume,REST webservices.
Hadoop Developer
Confidential, Hoboken, NJ
Responsibilities:
- Installed and configured Hadoop, MapReduce, HDFS (Hadoop Distributed File System), developed multiple MapReduce jobs in java.
- Worked with the infrastructure and admin team in designing, modeling, sizing and configuring Hadoop cluster of 15 nodes.
- Developed Map Reduce programs in Java and Scalafor parsing the raw data and populating staging Tables.
- Created Hive queries to compare the raw data with EDW reference tables and performing aggregates
- Importing and exporting data into HDFS and Hive using Sqoop.
- Experienced in analyzing data with Hive and Pig.
- Experienced knowledge over the Restful API's like Elastic Search.
- Writing Pig scripts to process the data.
- Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.
- Integrating bulk data into Cassandra file system using MapReduce programs.
- Got good experience with NOSQL database.
- Involved in HBASE setup and storing data into HBASE, which will be used for further analysis.
- Experienced in managing and reviewing Hadoop log files.
- Experienced in defining job flows.
- Experienced in managing and reviewing Hadoop log files.
- Installed and configured Hive and also written Hive UDFs.
- Involved in creating Hive tables, loading with data and writing hive queries using the HiveQL which will run internally in map reduce way.
- Extracted the data from MySQL, AWS RedShift into HDFS using Sqoop.
- Used HiveQL to analyze the partitioned and bucketed data and compute various metrics for reporting.
- Deployed Hadoop Cluster in Fully Distributed and Pseudo-distributed modes.
- Experience in managing and monitoring Hadoop cluster using Cloudera Manager.
- Supported in setting up QA environment and updating configurations for implementing scripts with Pig, Hive and Sqoop.
- Unit tested a sample of raw data and improved performance and turned over to production.
Environment: JDK1.7, Java, Hadoop 2.6.0, MapReduce, HDFS, Hive, Sqoop, HBase, Pig, Oozie, Kerberos, Linux, Shell Scripting, Oracle 11g, PL/SQL, SQL*PLUS, HDInsight.
Java/J2EE Developer
Confidential, Washington, D.C.
Responsibilities:
- Worked as a Java Developer and involved in analysis of requirements, design, development, Unit and Integration testing.
- Interact with Business Analyst and Subject Matter Experts (SME) to understand the requirements and for any clarifications required by the team, followed agile methodology and SCRUM meetings to track, optimize and tailor features to customer needs.
- Developed the application using Struts Framework dat leverages classical Model View Layer (MVC) architecture.
- High level design of SOA components to complete end-to-end B2B integration
- Developed API's using JMS and IBM MQ Series for XML messaging between several applications in prudential. The received XML
- Worked on establishing communications with other applications usingWebsphere Message brokerwith JMS for asynchronous messaging.
- Designed and developed Middle-Tier components using EJB (Message Driven Bean)
- Implemented the GoF design patterns like Factory, Singleton and Command patterns.
- Designed and implemented the presentation layerusing Twitter Bootstrap, Java Server Pages, tag libraries, cascading style sheets, AJAX, HTML and DHTML.
- Worked ondevelopment,maintenance and bug fixingin graingere-commerceapplication.
- Implemented Java 1.5 new features like generics, autoboxing/unboxing, enhanced for loops in the application.
- Bulk user creation service uses the above MQ Series API's, which gets user information from a third party Single Sign on application.
- Developed new restful web services usingJersey, Spring frame works
- Designed and integrated the full scales hibernate / Struts.
- Developed Action forms, Action classes and struts-config.xml file of Struts framework Developed workstation web module using Struts MVC, JSTL, integrationwith Hibernate.
- Involved in development of Generic hibernate DAO framework
- Extensively involved in developing core persistence classes using Hibernateframework, writing HQL queries, creating hibernate mapping (.hbm) files, DB schema and PL SQL changes.
- Consumed Web Services to implement application search functionality.
- Used the Java Collections API extensively in the application.
- Worked and Modified the Database Schema according to the Client requirement.
- Fixed application issues and halped to mitigate defect damages.
- Used Clearcase as the version control.
- Used Clear Quest for bug tracking, issue tracking and project management.
Environment: JDK1.6,Struts framework1.2,Log4j, Hibernate3.3, JSP,JSTL, Servlets, JNDI, EJB, JMS, JDBC, SOAP UI, Web Services, Oracle 10g, SQL, SQL Developer, Clearcase, JavaBeans, CSS, TOAD,HTML, DHTML, JavaScript, Twitter bootstrap 2.3.2, RAD 7.X, MQ, WebSphere0
Java/J2EE Developer
Confidential, Austin, Texas
Responsibilities:
- Involved in analysis, design and development of e-bill payment system as well as account transfer system and developed specs dat include use cases, class diagrams, sequence diagrams and activity diagrams.
- Developed custom tags, JSTL to support custom user interfaces
- Involved in designing the user interfaces using JSPs and Servlets.
- Developed presentation layer using HTML, CSS and Java script.
- Used EXT-JS framework for building interactive web applications using techniques such as Ajax, DHTML and DOM scripting.
- Designed powerful JSF and JSP Tag libraries for reusable web interface components.
- Used XML wed services using SOAP to transfer the amount to transfer application dat is remote and global to different financial institutions.
- Involved in development of web services for business operations using various Web Services API and tools like SOAP, WSDL, JAX-WS, JDOM, XML and XSL.
- Developed XML schemas - XSD, DTD for validation of XML documents.
- Developed application using spring framework dat leverages MVC (model view layer architecture).
- Developed business domain layer using session and entity beans EJBs.
- Used Java Messaging Services (JMS) for reliable and asynchronous exchange of important information such as payment status report.
- Developed master JMS producer master, JMS Consumer, and notification manager to implement existing interfaces and hide JMS details from existing (legacy) notification producers and consumers.
- Worked with a variety of issues involving multi-threading, server connectivity and user interface.
- Made extensive use of java Naming and Directory interface (JNDI) for looking up of enterprise beans.
- Developed SQL, PL/SQL, stored procedures - database application scripts.
- Involved in Sprint meetings and followed agile software development methodologies.
- Deployed the application on Web logic Application Server.
- Developed JUnit test cases for all the developed modules.
Environment: Java, J2EE, JSP 2.0, PL/SQL, Spring 2.0, EJB 2.0, JMS, JNDI, Oracle, XML, UML, DOM, SOAP, Rationale Rose, my eclipse, BEA Web Logic 7.0, Hibernate 2.0, MS SQL Server 2008,Agile.
Programmer
Confidential
Responsibilities:
- Participated in Design meetings and technical discussions.
- Developed User Interface in JSP, JavaScript and HTML.
- Implemented web application with JSF MVC.
- Implemented web layer with Spring MVC.
- Created GUIs for applications and applets using SWING components and applets.
- Developed Java Servlets and Beans for Backend processes.
- Created database tables, data model with oracle 10g.
- Created JUnit test cases to test individual modules.
- Participated in status meetings to ensure the task updates.
- Involved in bug fixing and enhancements of application.
Environment: Spring, JSF, Oracle,JRE 1.4, Eclipse 3.2, My Eclipse 4.1, JBoss EJB 2.0, Subversion, JSP,HTML, Java Script, PL/SQL, Windows XP.
