Big Data Architect / Big Data Engineer Resume
New York, NY
SUMMARY
- 10+ plus years of experience in analysis, design, development of web - based and component architecture applications using Big Data, Cloud, Java / J2EE, EJB, Servlets, JSP, JDBC, JNDI, Struts, Spring, Spring MVC, Hibernate, OIM, XML, XSL, XSLT, DTD, XQuery, Web Services, WSDL, SOAP, JAX-RPC, JAXB and Apache Axis. Also, skilled in C, Bash, HTML, Ajax, JavaScript, JQuery, .NET, BI tools.
- Experience on BIG DATA using HADOOP and related technologies such as HDFS, YARN, Spark HBASE, MapReduce, HIVE, PIG, FLUME, OOZIE, SQOOP, and ZOOKEEPER.
- Experience in deployment, configuration and management of CDH using Cloudera Manager and HDP using Ambari Cluster.
- Hands-on experience in Linux administration - user management, storage management, network management and shell scripting.
- Experience in integrating Kerberos to Hadoop Cluster
- Experience in design, development of web-based and component architecture applications using Java / J2EE, EJB, Servlets, JSP, JDBC, Struts, Spring MVC, Hibernate, XML, Web Services, WSDL, SOAP. Also, skilled in C, Bash, HTML, XHTML, Ajax, JavaScript, JQuery.
- Experience as Software Engineer, with onsite and offshore teams, leading the development of Web applications using different application/web servers.
- Experience with SQL and database schema design.
- Good working knowledge on GIT, SVN, CVS for version control.
- Proficient in OO Methodology, OO Analysis/Design Principles and OO design patterns.
- Knowledge of J2EE Web Services and experience in GOF and J2EE design pattern.
- Used SOAP (Simple Object Access Protocol) for web service by exchanging XML data between the applications.
- Good understanding of algorithms and data structures.
- Posses good communication and interpersonal skills and self-learning attitude.
TECHNICAL SKILLS
Languages: Java, SQL, Python, Scala, HTML, JavaScript, JQuery, Shell-Script
Hadoop Ecosystem: HDFS, YARN, MapReduce, Spark, Hive, Pig, Sqoop, Flume, Oozie, Hbase, Ranger, Sentry, Falcon, Kafka, Impala, Phoenix, Solr, Nifi, HAWQ, Hue HDP Smart sense, Cloudera Manager, Ambari.
Java Technologies: Servlets, JSP, JSF, EJB, JDBC, JNDI, Ajax, JUnit, Struts, Spring, Hibernate, JDBC, Web Services, Maven
FrameWorks: Spring, Struts, Hibernate, MVC, Mesosphere
Databases: MySQL, Oracle, MSSQL, Postgres, Hbase, greenplum
Operating Systems: Windows, IOS, UNIX/LINUX, RedHat, Kali, Ubuntu, CentOS.
PROFESSIONAL EXPERIENCE
Confidential, New York, NY
Big Data Architect / Big Data Engineer
Responsibilities:
- Deploying multi-node Hadoop cluster with LOB and application specific requirements.
- Doing POC various big data solution and certifying with JPMC compliance policies.
- Designing and implementing Hadoop security framework based on Kerberos catering to JPMC needs.
- Working closely with the vendors to implement new big data solutions and troubleshooting.
- Designed and implemented DR solution for Hortonworks environment using Falcon.
- Designed and implemented Granular level access control to HDFS using Sentry and Ranger policies.
- Worked on configuring HA solutions to various services on Hadoop cluster.
- Worked on configure SSL/TLS encryption for CDH and HDP services.
- Benchmarking of cluster using Teragen, Terasort, Teravalidate and Fine Tuning the cluster.
- Commissioning and Decommissioning of data nodes on Hadoop Cluster.
- Performing and certifying Upgrades to new versions of CDH and HDP distributions.
- Provide engineering documentation, on various task such as cluster upgrade, benchmarking testing, configuration etc.
- Automating upgrade process with Shell scripts.
- Performed engineering certification for ETL tools like PENTAHO and PAXATA.
- Engineering and implementation of MULTI TENANT Cluster model Cloudera.
- Implemented setting quota and Access Control Lists on job queue on Hadoop Cluster.
- Setting up local repositories for various big data components.
- Install and configure notebooks like Jupiter and Zeppelin for data scientist.
- Onboarding and managing FIDs, user and user classes in Active Directory and Unix(Keon).
- Engineered group mapping solution for multi region clusters.
- Design, architecture and engineering of data movement framework NIFI for structured and unstructured data sources.
- Engineering and implementation of security framework for APACHE NIFI with LDAP and SSL layers. Securing and encrypting nifi UI, Enabling LDAP.
- Design and engineering spark for batch and stream processing. Tuning of spark with static and dynamic allocation.
- Design, engineering and Implementation of R on spark using R-studio and SparkR
- Engineering and implementation of YARN static and dynamic resource pools.
- Engineering and implementing Kafka for stream processing, Kafka clusters for disaster recovery.
Environment: HDFS, YARN, MapReduce, Spark, Hive, Pig, Sqoop, Flume, Oozie, Hbase, Ranger, Sentry, Falcon, Kafka, Impala, Phoenix, Solr, Nifi, HAWQ, Hue HDP Smart sense., Cloudera, Hortonworks, BlueData, Mesosphere, Keon, RedHat, Windows, Active Directory, Ldap, Kerberos, greenplum, Java, Python, Scala, maven, eclipse
Confidential
Hadoop Developer
Responsibilities:
- Participated in requirements gathering and analysis.
- Involved different phases in big data projects like data acquiring, data processing and data serving.
- Import/export data from Oracle to/from HDFS using Sqoop.
- Imported data from different data sources with Flume.
- Implemented Custom Input formats that handles input files received from java applications to process in Map Reduce.
- Created partitions, bucketed tables in Hive for batter performance.
- Implemented Dash boards that handle HiveQL queries internally like Aggregation functions, basic hive operations, and different kinds of join operations.
- Managing and scheduling batch Jobs on a Hadoop Cluster using Oozie.
- Created production jobs using Oozie work flows that integrated different actions like Map Reduce, Sqoop, Hive.
- Experience in managing and reviewing Hadoop Log files.
- Experienced in configuring maven builds that integrated dependencies, check styles, test coverage's.
- Designing Test Plans, Test Cases and performed System Testing.
- Involved in daily SCRUM meetings to discuss the development/progress of Sprints and was active in making scrum meetings more productive.
Environment: Hadoop, MapReduce, Hive, Sqoop, Oozie, Linux, Oracle10g, Cloudier manager, Maven, MRUnit, Junit, Eclipse
Confidential
Sr. Unix/SAN Administrator (L3)
Responsibilities:
- Responsible to provide solution on Severity1/2 Problem /Incident tickets and worked on Change Request, and Work Order related to UNIX Servers & Storage.
- Performed Administration of Network Services (NIS / Ldap / DNS) on UNIX, Fixed architectural issues with the user authentication mechanisms.
- Configuring distributed file systems and administering NFS server and NFS clients and editing auto-mounting mapping as per system / user requirements.
- Implemented Virtualization using hardware nPars/ vPar / Solaris Zones (domains/partitions on Sun/HP).
- Planning, building & administering various High Availability clusters on HP-UX using Service Guard 11.17 in heterogeneous SAN environment.
- Performed project coordination function for evaluation, installation, monitoring, and maintenance of server software on variety of hardware platforms.
- Regular disk management like adding / replacing hard drives on existing servers / workstations, partitioning according to requirements, creating new file systems or growing existing one over the hard drives and managing file systems.
- Used shell/perl tools / scripts for various interests.
- Disaster recovery with the help of Ignite,vdump, Kick Start, PXE boot and other recovery tools.
- Performed security management SOX Audit, user administration (Secure shell /Sudoers) activities.
- Deliver IT solutions based on ITIL best practices (Change Management Process, Capacity Planning), Plan to deal with emergency response in such a way as to minimize rework.
- Planning, scheduling and implementing OS patches patch/package management tools such as HP swinstall, up2date, yum, Solaris patchadd, AIX install as a part of proactive Maintenance.
- Flexibly provided 24x7 production supports & executed project activities.
- Working with middleware/software teams for the deployment of the S/W & Web application.( e.g. Oracle10g,Oracle9i, Web Logic, Web Sphere and apache ).
- Train to new team member about the process & procedure & assist to other L1/L2 team member for any technical issue.
- As a part of SME (Subject Matter Expert ) responsible for preparing Technical Document & Shared on Hp/Delphi Security WIKI.
Environment: Over 600 UNIX Servers running on HPUX (11.11i, 11.23, 11.11i v3 ), Solaris10,Solaris (Ver. 9,8), Linux ( Redhat 3, 4 ), AIX (5.3) - Storage: - HP/ EMC Storage & Cisco/Brocade Switches Servers: - HP RP/Rx Server / SUN Fire E Series, Backup S/W (HP Data Protector & HP OPENVIEW monitoring S/W, Tape Library MSL 6030/ VLS 6000, Ultrim 960 Drives).
Confidential, Jefferson City, MO
Sr. Application Developer
Responsibilities:
- Developed modules General Ledger, Balance Sheet Reports, Invoice Report, Debt Collection, Check Report, and Account Payable, Account Receivable, Release Process of the application that manages the Correction center’s business.
- Coding and unit testing of Profile, Admin Module, using JAVA, JSPs.
- Responsible for the Development of Additional and Account Balance modules.
- Involved in the fixing the issues.
- Implemented Spring MVC architecture and Spring Bean Factory using IOC, AOP concepts.
- Design and develop the various customized generic framework to load and Save business entity using the template design pattern.
- Developed Role based user interface security with the preferences given to the user by their Organization Administrator.
- Created JSP and freemarker templates with Struts Framework, DHTML, CSS, JQuery, Ajax, Java Script for developing User Interfaces.
- Developed Struts action, interceptor, spring validator classes and restful Jersey web services
- Worked with the cross functional teams in gathering requirement and finalizing the database design for the module.
- Designed and developed jasper reports for Debt collection module.
- Worked on HQL and Criteria Queries as part of backend development.
- Developed and executed unit test plans using JUnit. Used PUMA SPI to store the user Profile Values in Tivoli LDAP.
- Worked with JSR 168 Portlets Migration from Websphere Portal v6.1 to V7. 0.2
- Worked with JSR 286 Portlets Migration from Websphere Portal v6.1 to V7.0.2
- Worked with EJB's Migration from Websphere Portal v6.1 to V7.0.2
- Developed the new Page Builder Theme for DOC ( Confidential, MO) Using WP V7.0.2
- Involved in the Layout Development in WP V7.0.2 and created the pages configuration, assigned the PZN rules.
- Develops Junits for various classes as this is a test-driven application. Resolved UAT and QA testing bug fixes.
- Facilitated the development team meeting and documented the new coding guidelines and standards.
- Extensively worked on the SQL queries for the batch process for our module
Environment: Java, J2EE, Spring, Struts, Hibernate, JSP, JQuery, AJAX, HTML, WebSphere Application Server, JBoss, DB2, ISeries navigator, DB Visualizer, Log4J, Spring Source Tool Suite, IReport 3.0, CVS, RestFul web-services, Struts, Hibernate, EJB, JSP, Servlets, JDBC, Ajax, XML, DB2, HTML, CSS, JS, Tivoli LDAP Directory.
Confidential, Wilton, CT
Sr. Application Developer
Responsibilities:
- Actively participated in the team meetings with business analysts to gather the business requirements.
- Brainstormed with the team and created the scope and functional specifications to lay out the project phases.
- Worked with the DBA team to develop the initial draft for the data model from the requirements.
- Developed important modules (Projects, Reports, Contact Manager, Publisher, Email, and Administration) of the application that manages the firm’s business.
- Designed the Role based user interface with the preferences given to the user by their Organization Administrator.
- Designing new enhancement requests. This includes the use of various OOAD design Pattern to come up with the optimal system architecture
- Designed use case diagrams, sequence diagrams and activity diagram using the UML diagrams and prepared documentation to explain the flow.
- Used JSP with Struts Framework, DHTML and for developing User Interfaces.
- Developed Action Forms, Action Classes and modified Struts-config.xml for individual Action mappings to build the controller
- Reports Module dynamically builds Queries based on the criteria selected by the users and returns thousands of records in less than 30ms. Customized the Report Module based on different client requirements.
- Designed Java components using DAO and Singleton design patterns.
- Used Data Transfer Objects (DTO) and Value Objects (VO) for transferring data from one layer to other.
- Developed Session Beans to implement business logic, which uses Stateless Session Beans to make database calls.
- Used Java Mail concept in developing Email Module, which gets email from POP3 server into Confidential system. It reads the Email Header, Body and Subject and then routes to the respective Projects or Opportunities, Users and Contacts.
- Publisher Module builds the dynamic .doc, .pdf, .html, .xml files on fly from the Confidential Database using the templates generated by the Users. Used itext in generating pdf files
- Developed code to publish XML Content for user’s dynamic Flash Websites from Confidential application.
- Coordinated and executed automated Load, Stress and Performance using WAPT 3.0.
- Written SQL & PL/SQL code, functions for developing backend queries and procedures.
- Used Eclipse IDE for development and Log4j for logging messages.
- Managed a team of 4 developers between ongoing development projects and immediate defects reported in production.
- Responsible for defect analysis and fixing System/UAT defects.
- Involved in the production On-Call support team and responsible for production incidents and Change Orders.
Environment: Struts 1.x, JSP 1.1, Tomcat 4.x/5.x, Eclipse 2.x, MVC, JavaScript, DHTML, Oracle9i, XML, WAPT3.0, JavaMail, Log4j, VSS
Confidential, Middletown, NJ
Sr. Java Developer
Responsibilities:
- Created use cases, sequence diagrams and preliminary class diagrams for the system using UML including VISIO.
- Utilized Framework encouraging application architectures based on the Model View Controller (J2EE Design Patterns) design paradigm.
- Performed server side programming using Servlets, JSP
- Worked on event-driven messaging systems with point to point and publish / subscribe models.
- Provided connectivity to Oracle 9i backend database using DAOs.
- Configured controller Servlet that dispatches request to appropriate Action Classes.
- Developed the Java Code using Eclipse as IDE.
- Involved in the client interactions and coordinating with on site / off shore team.
- Used Web Services to integrate our Applications with other Applications.
- Developed Action Classes and configured them in struts-config.xml file.
- Developed JSP Custom Tag Libraries and used in JSP Pages.
- Created Enterprise Java Beans (EJBs) and deployed them in the Web logic application server environment, to implement business logic and interface with back-end data base system (Oracle).
- Implemented Session facade Design pattern to provide Low Network Overhead, Low Coupling, Good reusability, Good Maintainability in the EJB Layer.
- Also, implemented Business Delegate design, Data Access Objects Design, Data Transfer Objects Design patterns.
- Used Hibernate as high performance object/relational persistence and query service to connect Database.
- Installed and configured Sunone webserver on local and configured access manager settings for Single Sign on (SSO).
- Created Hibernate configuration files and maintained the mapping files for the whole application.
- Coded pages that use XSLT and display data to the user.
- Parsed and created the XML content and updated the Oracle database using DOM & SAX Parsers.
- Used and worked with SOAP with attachments API for Java
- Formatted the XML Documents by using XSL-FO.
- Star Team is being implemented as Version Control system.
- Same Time is used as internal messenger.
- Interfaces were developed to the database with the help of JDBC drivers for Oracle.
- Used Rational Unified Process (RUP)
- Written Test Cases using Junit.
- Utilized ANT as a build system.
- Performed unit and System Integration Testing.
- Worked on automated regression tools
- Used Rational Clear Case to version control the code and test data files
- In the later part of the project, worked on the defect resolution phase. This included working with the Clear Quest tool for defects management
Environment: J2EE, Java Beans, JSP, XML, JAXP, JAXM, JAX-RPC, JAXR, JSF, JavaScript, JSP, Servlets, JDBC, Ajax, XML, RAD, DB2, HTML, CSS, JS, IBM Portal Express Server, Tivoli LDAP Directory, SOAP, Struts Framework 1.1, JavaScript, Sybase 12, JSP Tag Libraries and Servlets, Rational Unified Process (RUP), Session facade, Service delegate, EJB Command Pattern, Composite View, Delegate Helper and Data Transfer Object Factory, Oracle9i/10g.
Confidential, Denver, CO
J2EE Developer and Production Support
Responsibilities:
- Involved in the gathering the requirements for the Analysis of the system
- Involved in preparing the use case diagrams, sequence diagrams and class diagrams using Rational Rose, UML
- Developed JUnit test cases for regression testing and integrated with ANT build
- Interacting with the client Confidential regular intervals
- Responsible for the Development of Screens using JSP, HTML and Java Script
- Worked on SOAP with attachments API for Java
- Developed java interfaces to IBM FileNet P8 ECM and BPM
- Developed Database access components using HIBERNATE
- Developed Session Beans and Entity Bean components
- Used JDBC 3.0 for Oracle database connection and written number of stored procedures for retrieving the data and validated that data against the business rules
- Created and deployed EJB Beans (entity and session beans)
- Installed and configured WebSphere 5.1 and applied the fix packs 5.1.1 and 5.1.1.1, JDK refresh 1.4.2.1 and DB2 V8.1
- Automated different procedure executions as per the business needs
- Designed and developed the business objects in RMI as well as Main Servlet and dynamically called Modules
- Installed, implemented and maintained Apache Tomcat 5.0 for development environment responsibilities included deployment of WAR settings
- Involved in the gathering the requirements for the Analysis of the system
- Involved in preparing the use case diagrams, sequence diagrams and class diagrams using Rational Rose, UML
- Developed JUnit test cases for regression testing and integrated with ANT build
- Interacting with the client Confidential regular intervals
- Responsible for the Development of Screens using JSP, HTML and Java Script
- Worked on SOAP with attachments API for Java
- Developed java interfaces to IBM FileNet P8 ECM and BPM
- Developed Database access components using HIBERNATE
- Developed Session Beans and Entity Bean components
- Used JDBC 3.0 for Oracle database connection and written number of stored procedures for retrieving the data and validated that data against the business rules
- Created and deployed EJB Beans (entity and session beans)
- Installed and configured WebSphere 5.1 and applied the fix packs 5.1.1 and 5.1.1.1, JDK refresh 1.4.2.1 and DB2 V8.1
- Automated different procedure executions as per the business needs
- Designed and developed the business objects in RMI as well as Main Servlet and dynamically called Modules
- Installed, implemented and maintained Apache Tomcat 5.0 for development environment responsibilities included deployment of WAR settings
Environment: Java, J2EE architecture, Oracle, PL/SQL. Dvlp SOAP, Web Services, RESTful Web Svcs using FDML, PBEL, spring, iBatis, Hibernate, SQL queries, create hbm files, JSON, JQuery, Junit and EJB3.0, WSDL, SOAP, IBM FileNet P8 ECM and BPM, Oracle 8i, Apache Tomcat 5.0, WebSphere 5.1, XML (JAXP, JAXM, JAX-RPC, JAXR), SAAJ, UML, UDDI Business Registry, Windows-NT and Linux