Big Data Lead Resume
NyC
SUMMARY
- 13+ years of experience of developing Multi - threaded, low latency high volume Java Enterprise application with Application Integration, Designing and Modeling distributed enterprise architectures.
- 13+ years of experience in messaging infrastructure, Application servers with monitoring and performance tuning JAVA/JEE application in the field of Investment Banking and Manufacturing domain.
- 5+ years of expertise in developing, Big Data Enterprise application, Designing and Modeling for data analytics using most of the tools in Hadoop Echo System under Cloudera Infrastructure.
- 5+ years of experience in Writing and performance tuning Spark, Map Reduce and Kafka Streaming jobs.
- 3+ years of experience in implementing in memory data grids like GemFire, HazelCast for achieving high throughput on reference data access.
- Very good understanding and Professional Experience in Object Oriented Analysis & Design (OOAD), Design Patterns, Java EE Design Patterns.
- Implemented Microservices Architecture in DevOps structure with Agile methodologies to deliver on time.
- Extensive experience in Layered Architecture Spring framework. Involved in creating Spring IOC, Factory, and AOP, Transaction Management, Spring JMS, Spring MVC and Spring Quartz timers.
- Used Java Persistence API (JPA 2.0) for managing the object relational mapping (ORM) of Java objects to store persistent data in a database.
- Good Experience in implementing Microservices using Spring Boot with Spring Framework for independent and versatile applications.
- Good experience in JMS (Java Messaging Systems) applications. Used JBoss MQ and ActiveMQ services.
- 13+ years of expertise in writing and performance tuning core Java with collections, multi-threading and garbage collection by understanding internal implementation of each components.
- 13+ years of expertise in designing and relational mapping of RDBMS with normalizing the tables to achieve better performance for every transaction.
TECHNICAL SKILLS
Hadoop Eco System: Hadoop Eco System (Map Reduce, Spark., Hive, Sqoop, Impala, HDFS)
Programming Languages: Java (J2SE 1.8/1.7/1.6/1.5/1.4 ), JEE (JEE 1.8, 1.7, 1.6, 1.5), J2EE (J2EE 1.4/1.3)
Operating Systems: Windows XP/NT/200x, Sun Solaris, Linux, CentOS
Programming Technologies: JSP 2.1/2.0, Servlets 2.5/2.4/2.3, EJB 3.0/2.1/2.0/1.1, JPA2.0, JDBC 3.0/2/0, JMS, RMI, XML, XSD, JAXB, JIBX, Ajax, JavaScript, JQuery, Web Services, SOAP, LDAP
Open Sources: Struts, Hibernate 4.2.1/3.6.0/3.2.1/3.0, Spring 4.1.1/3.2.2/3.0.6/2.5, JSON Simple, JUnit, Find Bugs, Jakarta Ant, Maven
GUI Technologies: Java AWT, Swing, DHTML
Databases: Oracle 10g/9i, Sybase, MS Access, MySQL
Database Client Tools: Toad, Command Center, Oracle SQL Developer
Programming Tools: Eclipse, MyEclipse 10.6/8.6, IntelliJ Visual Café, WSAD 5.1.2, RAD 6.0.1, Axis
Configuration Management Tools: Visual Source Safe, ClearCase, CVS, Subversion, GIT
Issue Tracking Tool: JIRA, Mercury
Project Collaboration Tool: Confluence
Application Servers: BEA Weblogic Application Server 10.0/8.1/7.0/6.1/5.0, IBM WebSphere Application Server 6.0. JBoss Application Server 6.0/5.1.0/4.2.1, Apache Tomcat Server 6.0/7.0
SDLC Methodologies: Waterfall, and Agile methodologies
PROFESSIONAL EXPERIENCE
Confidential - NYC
Big Data Lead
Responsibilities:
- Designed and Implemented Microservices architecture using Spring Boot for independent modules for better maintenance and timely implementation.
- Designed and developed Spark batch jobs to process high volume data with low latency and fine-tuned existing jobs for quicker processing.
- Designed and tuned HDFS file partitioning for better performance of reporting tool.
- Worked on migrating code from Spark version 1.6 to 2.2 and also improved performance and tuning the code.
- Worked on implementing Spark Streaming with Kafka for receiving messages from upstream on a topic and process them in real time.
- Implemented Gemfire in Client server architecture for loading reference data to process real-time messages using Kafka/ Spark Streaming.
- Designed and implemented QA tool to trigger Spark jobs in Unix Yarn from windows. Redesigned the application structure to allow multiple instances to be used in one environment.
- Involved in all technical challenges application team faces.
Confidential - New York City
Big Data Lead
Responsibilities:
- Working as Server-side Lead Developer for seven-member team and involved in requirement gathering, requirement analysis, project design and development.
- Developed various complex capital calculations using Apache Spark / Map Reduce jobs.
- Caching the customer/product reference data for different clusters using Hazelcast.
- RETE algorithm has been used to maintain the rules for batch processing.
- Strategic batch and Tactical calculations are stored in HDFS distributed storage
- Used Apache Hive for structured data storage and Apache Impala for interactive query. Hue used for end user query access.
- Import/Export HDFS data into STAR database using Sqoop
- Involved in L3 Production Support for current application.
- Used Odyssey for Project collaboration, TeamCity for Continuous Integration (CI) server and JIRA for project tasks tracking.
- Fish Eye & Crucible used for code review and analysis.
- Developed and implemented new functionality in multi-threading environment to process data files in legacy application.
- Processed High volume data file using multithreading by splitting data file.
- Worked on implementing Inhouse rules engine which is used for implementing different functionality dynamically by defining rules.
- Used Lambda and functional programming features to simplify coding.
Confidential - New York City
Sr Java Lead
Responsibilities:
- Developed generic framework to process transactions from different type of files, MQ queues, Http systems, and database. All transactions are routed to different queues with built in workflow Engine. Output can be written to files, send to MQ queues, Http Systems or stores in database. Architecture of framework based on input Transporter (File Reader, MQ Reader and Database Reader) and Output Transporter (File Writer, MQ sender, Database writer) with build Workflow Manager to route the transactions to Different queues. Health Of system can monitor with Jconsole. This framework supports parallel processing and chunk processing. This framework developed using core java, Thread Pool Executor and Executor Completion Service used for parallel processing with Hibernate in persistence layer and spring for dependency injection.
- Developed generic component that pulls product information from ESM system in XML format and serialize to Object.
- Thread pool and blocking queues used for parallel processing.
- Developed generic component to send message and receive message from MQ queues using spring.
- Extensively used Spring JDBC and Spring Schedulers.
- Developed Rule based pluggable validation framework using composite design pattern.
- Developed generic enrichment framework to enrich transaction data with Product information from ESM and SDS counter party information.
- Hibernate associations and criteria API used for chunk processing (batch processing) and updating and inserting data in to MSSQL database.
- Worked on JSON responding RESTful web services to send and receive data on both upstream and downstream.
- Developed applications using spring integration Input adapters, Output Adapters, Flow components, Channels.
- Maven plug-in used for building applications.
Environment: Java1.6, java concurrent package, J2ee, Web services, JMS, Spring, Hibernate3.0, MQ, Sprig integration, Maven, XML, Eclipse 3.4, Windows, Linux, MSSQL, Sybase, Perforce.
Confidential - New Jersey
Sr Java Developer
Responsibilities:
- Reused the whole framework I developed for GFS in Java Spring and Hibernate and migrated the application from Visual Basic to Java/GWT application.
- Involved in challenging reverse engineering process of this legacy application, which is missing some source code. Black box calculation of this financial reporting and thoroughly understood and developed in spring and Hibernate technology.
- Migrated whole application from Sybase to MSSQL database server.
Environment: Java 1.6, spring, Hibernate, Eclipse 3.4, VB 6.0, MSSQL, Sybase, Rational ClearCase
