We provide IT Staff Augmentation Services!

Bigdata Engineer Resume

New York, NY

SUMMARY

  • 10+ years of total experience in java/J2EE with 2+ years in Big Data.
  • Experience on BIG DATA using HADOOP and related technologies such as HDFS, YARN, Spark HBASE, MapReduce, HIVE, PIG, FLUME, OOZIE, SQOOP, and ZOOKEEPER.
  • Experience in deployment, configuration and management of CDH using Cloudera Manager and HDP using Ambari Cluster.
  • Hands - on experience in Linux administration - user management, storage management, network management and shell scripting.
  • Experience in integrating Kerberos to Hadoop Cluster
  • Experience in design, development of web-based and component architecture applications using Java / J2EE, EJB, Servlets, JSP, JDBC, Struts, Spring MVC, Hibernate, XML, Web Services, WSDL, SOAP. Also skilled in C, Bash, HTML, XHTML, Ajax, JavaScript, JQuery.
  • Experience as Software Engineer, with onsite and offshore teams, leading the development of Web applications using different application/web servers.
  • Experience with SQL and database schema design.
  • Good working knowledge on GIT, SVN, CVS for version control.
  • Proficient in OO Methodology, OO Analysis/Design Principles and OO design patterns.
  • Knowledge of J2EE Web Services and experience in GOF and J2EE design pattern.
  • Used SOAP (Simple Object Access Protocol) for web service by exchanging XML data between the applications.
  • Good understanding of algorithms and data structures.
  • Posses good communication and interpersonal skills and self-learning attitude.

TECHNICAL SKILLS

Languages: Java, SQL, Python,Scala, HTML,JavaScript,JQuery,Shell-Script

Hadoop Ecosystem: HDFS, YARN, MapReduce, Spark, Hive, Pig, Sqoop, Flume, Oozie, Hbase, Ranger, Sentry, Falcon, Kafka, Impala, Phoenix, Solr, Nifi, Splunk, HAWQ, Hue HDP Smart sense, Cloudera Manager, Ambari.

Java Technologies: Servlets, JSP, JSF, EJB, JDBC, JNDI, Ajax, JUnit, Struts, Spring, Hibernate, JDBC, Web Services, Maven

FrameWorks: Spring, Struts, Hibernate, MVC, Mesosphere

Databases: MySQL, Oracle, MSSQL,Postgres, Hbase, greenplum

Operating Systems: Windows, IOS, UNIX/LINUX, RedHat, Kali, Ubuntu, CentOS.

PROFESSIONAL EXPERIENCE

Confidential, New York, NY

BigData Engineer

Responsibilities:

  • Deploying multi-node hadoop cluster with LOB and application specific requirements.
  • Doing POC various big data solution and certifying with JPMC compliance policies.
  • Designing and implementing Hadoop security framework based on Kerberos catering to JPMC needs.
  • Working closely with the vendors to implement new big data solutions and troubleshooting.
  • Designed and implemented DR solution for Hortonworks environment using Falcon.
  • Designed and implemented Granular level access control to HDFS using Sentry and Ranger policies.
  • Worked on configuring HA solutions to various services on Hadoop cluster.
  • Worked on configure SSL/TLS encryption for CDH and HDP services.
  • Benchmarking of cluster using Teragen, Terasort, Teravalidate and Fine Tuning the cluster.
  • Commissioning and Decommissioning of data nodes on Hadoop Cluster.
  • Performing and certifying Upgrades to new versions of CDH and HDP distributions.
  • Provide engineering documentation, on various task such as cluster upgrade, benchmarking testing, configuration etc.
  • Automating upgrade process with Shell scripts.
  • Performed engineering certification for ETL tools like PENTAHO and PAXATA.
  • Engineering and implementation of MULTI TENANT Cluster model Cloudera.
  • Implemented setting quota and Access Control Lists on job queue on Hadoop Cluster.
  • Setting up local repositories for various big data components.
  • Install and configure notebooks like jupyter and Zeppelin for data scientist.
  • Onboarding and managing FIDs, user and user classes in Active Directory and Unix(Keon).
  • Engineered group mapping solution for multi region clusters.
  • Design, architecture and engineering of data movement framework NIFI for structured and unstructured data sources.
  • Engineering and implementation of security framework for APACHE NIFI with LDAP and SSL layers. Securing and encrypting nifi UI, Enabling LDAP.
  • Design and engineering spark for batch and stream processing. Tuning of spark with static and dynamic allocation.
  • Design, engineering and Implementation of R on spark using R-studio and SparkR
  • Engineering and implementation of YARN static and dynamic resource pools.
  • Engineering and implementing Kafka for stream processing, Kafka clusters for disaster recovery.

Environment: HDFS, YARN, MapReduce, Spark, Hive, Pig, Sqoop, Flume, Oozie, Hbase, Ranger, Sentry, Falcon, Kafka, Impala, Phoenix, Solr, Nifi, HAWQ, Hue HDP Smart sense., Cloudera, Hortonworks, BlueData, Mesosphere, Keon, RedHat, Windows, Active Directory, Ldap, Kerberos, greenplum, Java, Python, Scala, maven, eclipse

Confidential, Iowa, Des Moines

Hadoop Developer

Responsibilities:

  • Participated in requirements gathering and analysis.
  • Involved different phases in big data projects like data acquiring, data processing and data serving.
  • Import/export data from Oracle to/from HDFS using Sqoop.
  • Imported data from different data sources with Flume.
  • Implemented Custom Input formats that handles input files received from java applications to process in Map Reduce.
  • Created partitions, bucketed tables in Hive for batter performance.
  • Implemented Dash boards that handle HiveQL queries internally like Aggregation functions, basic hive operations, and different kinds of join operations.
  • Managing and scheduling batch Jobs on a Hadoop Cluster using Oozie.
  • Created production jobs using Oozie work flows that integrated different actions like Map Reduce, Sqoop, Hive.
  • Experience in managing and reviewing Hadoop Log files.
  • Experienced in configuring maven builds that integrated dependencies, check styles, test coverage's.
  • Designing Test Plans, Test Cases and performed System Testing.
  • Involved in daily SCRUM meetings to discuss the development/progress ofSprints and was active in making scrum meetings more productive.

Environment: Hadoop, MapReduce, Hive, Sqoop, Oozie, Linux, Oracle10g, Cloudier manager, Maven, MRUnit, Junit,Eclipse

Confidential, Arlington, VA

Sr. Application Developer

Responsibilities:

  • Developed widgets for Assessment Profile, Managing Favorite Projects, Message Feeds, Information Blocks, Login Authorization and authentication, Creating package for assessments including reports and tests.
  • Developing two factor Authentication service and implemented Captcha.
  • Worked extensively developing JSPX with Spring Framework, XHTML, CSS, JavaScript, Ajax, Java
  • Script for developing Flat design UI.
  • Implemented Spring security.
  • Worked closely with offshore teams in UK, India, who were involved developing different parts of application alongside gathering requirement and redesign database.
  • Worked ho hibernate and used query strings for backend Confidential repository level.
  • Developed and executed unit test plans using JUnit..
  • Resolved QA testing bug fixes.
  • Developed Spring Roo add-ons to facilitate widget Based development,
  • Implemented Apache Solr for search on widget level,
  • Localized application to support about 40 languages.
  • Implemented theme and layout options as per client preferences using JSF.
  • Created custom UI tag library,

Environment: Hadoop, MapReduce, HDFS, Hive, Sqoop, HiveQL, Oozie, Avro Data Serialization, Cloudera, Java, My SQL, SQL, Unix, Eclipse, maven, Junit.

Confidential, Jefferson City, MO

Sr. Application Developer

Responsibilities:

  • Developed modules General Ledger, Balance Sheet Reports, Invoice Report, Debt Collection, Check Report, and Account Payable, Account Receivable, Release Process of the application that manages the Correction center’s business.
  • Coding and unit testing of Profile, Admin Module, using JAVA, JSPs.
  • Responsible for the Development of Additional and Account Balance modules.
  • Involved in the fixing the issues.
  • Implemented Spring MVC architecture and Spring Bean Factory using IOC, AOP concepts.
  • Design and develop the various customized generic framework to load and Save business entity using the template design pattern.
  • Developed Role based user interface security with the preferences given to the user by their Organization Administrator.
  • Created JSP and freemarker templates with Struts Framework, DHTML, CSS, JQuery, Ajax, Java Script for developing User Interfaces.
  • Developed Struts action, interceptor, spring validator classes and restful Jersey web services
  • Worked with the cross functional teams in gathering requirement and finalizing the database design for the module.
  • Designed and developed jasper reports for Debt collection module.
  • Worked on HQL and Criteria Queries as part of backend development.
  • Developed and executed unit test plans using JUnit. Used PUMA SPI to store the user Profile Values in Tivoli LDAP.
  • Worked with JSR 168 Portlets Migration from Websphere Portal v6.1 to V7. 0.2
  • Worked with JSR 286 Portlets Migration from Websphere Portal v6.1 to V7.0.2
  • Worked with EJB's Migration from Websphere Portal v6.1 to V7.0.2
  • Developed the new Page Builder Theme for DOC ( Confidential, MO) Using WP V7.0.2
  • Involved in the Layout Development in WP V7.0.2 and created the pages configuration, assigned the PZN rules.
  • Develops Junits for various classes as this is a test driven application. Resolved UAT and QA testing bug fixes.
  • Facilitated the development team meeting and documented the new coding guidelines and standards.
  • Extensively worked on the SQL queries for the batch process for our module

Environment: Java, J2EE, Spring, Struts, Hibernate, JSP, JQuery, AJAX, HTML, WebSphere Application Server, JBoss, DB2, ISeries navigator, DB Visualizer, Log4J, Spring Source Tool Suite, IReport 3.0, CVS,RestFul web-services,Struts, Hibernate, EJB, JSP, Servlets, JDBC, Ajax, XML, DB2, HTML, CSS, JS, Tivoli LDAP Directory.

Confidential, Wilton, CT

Sr. Application Developer

Responsibilities:

  • Actively participated in the team meetings with business analysts to gather the business requirements.
  • Brainstormed with the team and created the scope and functional specifications to lay out the project phases.
  • Worked with the DBA team to develop the initial draft for the data model from the requirements.
  • Developed important modules (Projects, Reports, Contact Manager, Publisher, Email, and Administration) of the application that manages the firm’s business.
  • Designed the Role based user interface with the preferences given to the user by their Organization Administrator.
  • Designing new enhancement requests. This includes the use of various OOAD design Pattern to come up with the optimal system architecture
  • Designed use case diagrams, sequence diagrams and activity diagram using the UML diagrams and also prepared documentation to explain the flow.
  • Used JSP with Struts Framework, DHTML and for developing User Interfaces.
  • Developed Action Forms, Action Classes and modified Struts-config.xml for individual Action mappings to build the controller
  • Reports Module dynamically builds Queries based on the criteria selected by the users and returns thousands of records in less than 30ms. Customized the Report Module based on different client requirements.
  • Designed Java components using DAO and Singleton design patterns.
  • Used Data Transfer Objects (DTO) and Value Objects (VO) for transferring data from one layer to other.
  • Developed Session Beans to implement business logic, which uses Stateless session Beans to make database calls.
  • Used Java Mail concept in developing Email Module, which gets email from POP3 server into Confidential system. It reads the Email Header, Body and Subject and then routes to the respective Projects or Opportunities, Users and Contacts.
  • Publisher Module builds the dynamic .doc, .pdf, .html, .xml files on fly from the Confidential Database using the templates generated by the Users. Used itext in generating pdf files
  • Developed code to publish XML Content for user’s dynamic Flash Websites from Confidential application.
  • Coordinated and executed automated Load, Stress and Performance using WAPT 3.0.
  • Written SQL & PL/SQL code, functions for developing backend queries and procedures.
  • Used Eclipse IDE for development and Log4j for logging messages.
  • Managed a team of 4 developers between ongoing development projects and immediate defects reported in production.
  • Responsible for defect analysis and fixing System/UAT defects.
  • Involved in the production On-Call support team and responsible for production incidents and Change Orders.

Environment: Struts 1.x, JSP 1.1, Tomcat 4.x/5.x, Eclipse 2.x, MVC, JavaScript, DHTML, Oracle9i, XML, WAPT3.0,JavaMail,Log4j,VSS

Confidential, Middletown, NJ

Sr. Java Developer

Responsibilities:

  • agrams and preliminary class diagrams for the system using UML including VISIO.
  • Utilized Framework encouraging application architectures based on the Model View Controller (J2EE Design Patterns) design paradigm.
  • Performed server side programming using Servlets, JSP
  • Worked on event-driven messaging systems with point to point and publish / subscribe models.
  • Provided connectivity to Oracle 9i backend database using DAOs.
  • Configured controller Servlet that dispatches request to appropriate Action Classes.
  • Developed the Java Code using Eclipse as IDE.
  • Used Web Services to integrate our Applications with other Applications.
  • Developed Action Classes and configured them in struts-config.xml file.
  • Developed Form Beans and configured them in struts-config.xml file.
  • Developed JSP Custom Tag Libraries and used in JSP Pages.
  • Created Enterprise Java Beans (EJBs) and deployed them in the Web logic application server environment, to implement business logic and interface with back-end data base system (Oracle).
  • Implemented Session facade Design pattern to provide Low Network Overhead, Low Coupling, Good reusability, Good Maintainability in the EJB Layer.
  • Also implemented Business Delegate design, Data Access Objects Design, Data Transfer Objects Design patterns.
  • Used Hibernate as high performance object/relational persistence and query service to connect Database..
  • Created Hibernate configuration files and maintained the mapping files for the whole application.
  • Coded pages that use XSLT and display data to the user.
  • Parsed and created the XML content and updated the Oracle database using DOM & SAX Parsers.
  • Used and worked with SOAP with attachments API for Java
  • Formatted the XML Documents by using XSL-FO.
  • Star Team is being implemented as Version Control system.
  • Same Time is used as internal messenger.
  • Interfaces were developed to the database with the help of JDBC drivers for Oracle.
  • Written Test Cases using Junit.
  • Utilized ANT as a build system.

Environment: UML with Rational Rose, Clear Case, Clear Quest, J2EE, Java Beans, JSP, XML, JAXP, JAXM, JAX-RPC, JAXR, JSF, JavaScript, WSDL, SOAP, Struts Framework 1.1, JavaScript, Sybase 12, WebSphere 5.1, JSP Tag Libraries and Servlets, Rational Unified Process (RUP), Session facade, Service delegate, EJB Command Pattern, Composite View, Delegate Helper and Data transfer Object Factory, Oracle9i/10g.

Confidential

J2EE Developer and Production Support

Responsibilities:

  • Involved in the gathering the requirements for the Analysis of the system
  • Involved in preparing the use case diagrams, sequence diagrams and class diagrams using Rational Rose, UML
  • Developed JUnit test cases for regression testing and integrated with ANT build
  • Interacting with the client Confidential regular intervals
  • Responsible for the Development of Screens using JSP, HTML and Java Script
  • Worked on SOAP with attachments API for Java
  • Developed java interfaces to IBM FileNet P8 ECM and BPM
  • Developed Database access components using HIBERNATE
  • Developed Session Beans and Entity Bean components
  • Used JDBC 3.0 for Oracle database connection and written number of stored procedures for retrieving the data and validated that data against the business rules
  • Created and deployed EJB Beans (entity and session beans)
  • Installed and configured WebSphere 5.1 and applied the fix packs 5.1.1 and 5.1.1.1, JDK refresh 1.4.2.1 and DB2 V8.1
  • Automated different procedure executions as per the business needs
  • Designed and developed the business objects in RMI as well as Main Servlet and dynamically called Modules
  • Installed, implemented and maintained Apache Tomcat 5.0 for development environment responsibilities included deployment of WAR settings

Environment: Java, JSP, Servlets, JavaScript, WSDL, SOAP, IBM FileNet P8 ECM and BPM, Oracle 8i, Apache Tomcat 5.0, WebSphere 5.1, XML (JAXP, JAXM, JAX-RPC, JAXR), SAAJ, UML, UDDI Business Registry, Windows-NT and Linux

Hire Now