Big Data Developer - Hadoop Resume
SUMMARY
- Having 11+ years of IT experience in real time analysis, design, development, testing, and implementation of various IT applications in HealthCare, Telecom and Retail domains
- Responsible for guiding the full lifecycle of a Big Data Solution (Hadoop) solution, including requirements analysis, platform selection, technical architecture design, application design and development, testing, and deployment
- Involved in Hadoop Cluster Setup discussions / meetings and provided suggestions and recommendations and also involved in HDFS Cluster Setup Configurations
- End - to-end monitoring and troubleshooting of the Hadoop cluster, database and networks in addition to multi-cluster management using SPLUNK-Hadoop Connector
- Headline alerting for numerous threshold conditions, such as crashed disk and slow Map/Reduce jobs
- A centralized real-time view of Hadoop nodes using an intuitive heat map display
- Measuring current and historical Map/Reduce jobs to analyze failures or performance issues
- Developed many Map/Reduce programs
- Written User Defined Functions in Java language and implementing in Hadoop HDFS environment
- Created tables with Partition in HIVE tables and loaded data into HIVE & HBase tables
- Have done IMPORT / EXPORT operations using Hadoop Data Management tool SQOOP
- Have done Database Schema Evolution processing using Hadoop Data Schema Management tool AVRO
- Advanced SQL skills with ability to code complex queries to load, clean, format & store data
- Experience in requirement gathering, analysis, planning, designing, coding and unit testing
- Developed and maintained elaborate services based architecture utilizing open source technologies like Hibernate ORM and Spring Framework.
- Developed RESTful/SOAP web services using JAX-RS, JAXB, JAX RPC and JAX WS technologies.
- Hands on experience in Installation, Configuration, Administration, Migration, Integration, and support of web based applications in Regulatory environment.
- Experience in proven development methodologies - Agile, Scrum and Waterfall.
- Experienced with SOA and SCA architecture using IBM web sphere Integration developer WID 6.0, web sphere process server 6.1.2, Message broker, MULE ESB and IBM ESB.
- Full Lifecycle (SDLC) implementation experience, worked on all phases starting from Requirement gathering, Business Blueprint, Realization/Testing, Go-Live/Support, user training, documentation and developed many business functionalities using Java/JEE.
- Worked in financial, retail, sales, e-commerce, health insurance, health care, communication and media domains
TECHNICAL SKILLS
Hadoop Technologies: Hadoop, HDFS, Hadoop Map-Reduce, Hive, HBase, SQOOP, SPLUNK, Oozie, AVRO, Zookeeper & Pig-Latin
No SQL: MongoDB
Column Oriented DB: HBase
Frameworks: JSF, Spring, Spring MVC, Spring IOC / Spring AOP / DAO / ORM, Spring Webflows, Struts, facelets, RichFaces, Velocity
IDE/Tools: RAD, ECLIPSE, XML beans, Net Beans, Visual studio
ORM/Persistence and Rules: Hibernate, JPA, Fair Isaac Blaze Rules engine, EJB3
Web and Application Servers: Web sphere application server, JBOSS, Tomcat, Web logic
Core Competency Technologies: Java, OOPS, design patterns, Swings, JSP, JSTL, servlets, EJB, JDBC, JNDI, B2B, xml, OOA/OOD, java 5 / java 6 / java 7, C, C++
Web services: SOAP web services 1.1/1.2, RESTful web services, JAX-RPC, JAX-WS, JAX-RS, Apache AXIS 2, CXF, Jersey, GitHub, WCF
SOA products: IBM Web Sphere Process server 6.1.2, Message broker 6.1
Web presentation frameworks: Java Script, XHTML, AJAX, jQuery, CSS, DOJO, JSON
Messaging: JMS, MQ, MDB
Testing & Issue Log tools: JUnit 4, EasyMock, SOAP UI, Team Track, TDD
Xml: XSLT, XSD, SAX, DOM, JAXB, XPath, XQuery
SCM/Version control tools: Dimensions, Rational Clear Case, CVS, MSTFS, Sub Version
Operating Systems: Windows, HP-Unix, AIX, Sun Solaris, TANDEM
Modeling tools: Visio 2007, UML, OOAD, RSA
Build and continuous Integration: MAVEN, ANT, Jenkins, Hudson
Web and Photoshop editors: Fire Bug, IE Developer tool bar, Adobe Photoshop 10.1.1
Security Tools: IBM TAM, Site Minder, SSL
Data base: Oracle 9i/10g, DB2, PL/SQL, SSIS
Cloud computing: Sales force
OS: UNIX, LINUX
PROFESSIONAL EXPERIENCE
Confidential
Big Data Developer - Hadoop
Responsibilities:
- Involved in requirement gathering, analysis, design, development, documentation, testig and implementation
- Participated in Building Hadoop environment on 20 Node cluster
- Worked on tuning the hadoop cluster
- Written Map-Reduce programs in Hadoop environment and debugging the issues / errors
- Writing User Defined Functions in Java language and implementing in Hadoop HDFS environment
- Worked on file manipulation by matching, merging, cleaning, modifying and validating data from various data sources using BASE SAS, SAS/MACRO, SAS/ACCESS AND SAS/STAT
- Creating tables with Partition in HIVE tables and loading data into HIVE & HBase tables
- Writing IMPORT / EXPORT operations using Hadoop Data Management tool SQOOP
- Writing Database Schema in JASON format for AVRO Database Schema Evolution processing using Hadoop Data Schema Management tool AVRO
- Involved in Data Modeling discussions and provided recommendations for best data model strategic planning
- End-to-end monitoring and troubleshooting of the Hadoop cluster, database and networks in addition to multi-cluster management using SPLUNK-Hadoop Connector
- Headline alerting for numerous threshold conditions, such as crashed disk and slow Map/Reduce jobs
- A centralized real-time view of Hadoop nodes using an intuitive heat map display
- Measuring current and historical Map/Reduce jobs to analyze failures or performance issues
- Conducted market research for international clients and Performed statistical and financial analysisbased on the Market Capital
- Converted R Statistical models to SAS Statistical models
- Writing Map/Reduce programs for ETL processes, and HIVE Query language to extract the reports from HIVE tables
- Investigated product feasibility by performing analyses that include market sizing, competitive analysis and positioning
- Analyzed data using SAS procedures PROC CORR, PROC FREQ, PROC UNIVARIATE, PROC TAB, PROC REG, PROC ANOVA, PROC GLM
- Analyze reports of data duplicates or other errors to provide ongoing appropriate interdepartmental communication and monthly or daily data reports
- Writing Unix Shell Scripts to automate the processes for any data validations and job executions
- Create and present quality dashboards and generate routine and adhoc reports
- Based on provides Data Quality Standards, making sure the data integrity and working on performance tuning on existing SAS ETL processes
- Implementing ETL processes for daily, monthly, quarterly and annual market data using SPSS
- Worked on ETL processes like pull / push the Data from SAS datasets to databases using SAS, IBM Data Stage and Big Data Technologies
- Loaded Data into HIVE tables using Map/Reduce programs and written extensive SQLs in HIVE to verify the results
Environment: Hadoop, HDFS, Hive, HBase, SQOOP, SPLUNK, AVRO, Oozie, ZooKeeper, Pig-Latin, AutoSys, SQL, Oracle 11G, JAVA, JSP, Servlets, Spring, ECLIPSE, IBM Data Stage, Business Objects, LINUX, IBM Websphere, Maven, XML
Designer/Developer
Confidential
Responsibilities:
- Created new dynamic web JSR 286 portlets, JSF pages, URL mappings, labels, custom themes, custom skins, XML access based on the requirement.
- Written java script validations and JSF custom validations for all the fields.
- Implementation of client code to invoke the web service and registering the services in portlet service registry.
- Worked on mediation flows.
- Developed message flows for providing and consuming services using variety of nodes in message broker tool kit.
- Performed custom coding and transformation using ESQL and Java.
- Updated session summary using portlet session id using JSF AJAX.
- Developed delegate, façade layers, DAO layers (design patterns) to call ESB services.
- Triggered servlets and timer listeners to upload the images into cobalt server.
- Worked on Edit packages and saved packages using portlet session scope and DYNA CACHE concepts in JSF page beans and developed delegate, façade layer to invoke ESB operations.
- Worked on the preview functionality using Jquery, Java script. JSF session beans.
- Developed the Home page and created URL mappings for all the content HTMLs for the entire website.
- Created content, components, libraries using IBM Lotus WCM.
- Performed Syndication activity to move the static content across the environments.
- Worked on IBM portlet factory to customize portlets.
- Worked on WSRP (remote portlets over SOAP).
- Invoked the servlets directly from JSF pages and Converted static HTMLs to JSF pages.
- Created psd files and and edited images using adobe Photoshop.
- Monitored, edited HTMLS, CSS to fix various cross browser issues using mozilla fire bug inspect element and IE developer tool bar.
- Developed server side logic and faces context error messages for java script disabled browsers.
- Configured the DB connection from WAS console to call the stored procedures from servlets.
- Developed log in, registration, order creation, get items ESB services using message broker tool kit.
- Created message driven bean (MDB) using consumer bean and producer bean, and configured JNDI in WAS console to send/receive messages asynchronously using JMS.
- Worked on wps/portal non authenticated and wps/myportal authenticated user issues.
- Developed omniture java script tag for web analytics in all pages of the website.
- Performed WAR deployment and EAR deployment in various environments.
- Modified pom.xmls to add the path for dependency jars and allowed formats for maven compilation.
- Created favicon for website using adobe photoshop and fixed cross browser issues.
- Worked on SAP CRM smart forms to edit the HTMLs, style classes and image paths.
- Worked on load balancing, node sync up and clustering issues.
- Created web sphere adapter project to connect to the ESB from portal.
Environment: Windows-XP, SAP CRM, JDK 1.6, IBM SOAP engine, Oracle, Message broker, ESB, JDBC, Log4j, RAD 7.5.5, IBM Clear Case, IBM web sphere Portal server 6.1, IBM web sphere Application server 7.0, JSF 1.2, JBOSS, Omniture, Unix, LDAP
Senior Java Developer
Confidential, MINEAPOLIS
Responsibilities:
- Constructed the SOAP request using and response using SOAP 1.2.
- Came up with design plan for the requirement.
- Implementation of DAS Web service
- Written deployment scripts in shell
- Configuration management activities.
- Implementation of client code to invoke the web service.
- Performance improvement and timeout analysis.
- Log4j configuration.
- Security bindings and site minder configurations.
- Created IP filter xml file to prevent fraud requests to the servers.
- Converted sequence diagrams to java code using reverse engineering in RSA.
Environment: Windows-XP, JDK 1.5, Java bean, Axis2, Non-Stop SQL, JDBC, Log4j, WSAD, RAD 7.0, RSA, SUN UNIX, WBSD, Dimensions, SOAP UI.
Development Lead
Confidential, New Hampshire
Responsibilities:
- Developed JSR 168 portlets using IBM RAD 6.0.
- Added Spring for implementing aspects and container services, decoupling and separation of concerns, container independence, modernize web layer to Spring MVC
- Handled validation frame work, configured Event Listeners and developed java script for portal pages.
- Involved in implementation of Dyna Cache.
- Integrated claims module with rules using Fair Isaac Blaze advisory rules engine in WID.
- Created rule maintenance application, rule sets for Medicare, developed rule flows using Fair Isaac Blaze advisor rule engine.
- Worked specifically for Claims, billing systems and processing module using EJB and most concepts of WebSphere Integration Developer (WID) with SOA Architecture.
- Designed Use case model, Analysis model and Service model for Service Audit Logger framework using RSA 7.0.0.
- Worked on Category of Service (COS) which deals with the different types services provided falls in which category by considering different scenarios like, Provider details, Diagnosis details, Surgical Procedure Code details, Procedure Code details and Dates of service
- Setting up new communities and managing user roles implementing.
- Worked on Diagnosis Related Groups (DRG) use case by supporting Native libraries and integrating Third party COTS product with Claims Adjudication module using Mediation module concept of Web Sphere Integration Developer (WID).
- Extensively worked on Process and Application Layers of Programming.
- Involved in the implementation of Hibernate layer.
- Developed Rule Maintenance Application (RMA) Using Fairy Isaac Blaze Advisory Tool.
- Prepared the JUNIT test cases and Quality documents, executed JUNIT test cases.
- Prepared the Code coverage report using profiling mode.
- Extensively worked on Process and Application Layers of Programming.
- Integrated claims module with rules engine.
- Configuration management activities with static views of clear case.
Environment: Oracle Database 10g, IBM Rational Application Developer 7.0, Java EE, JSF, EJB (Session Beans), Hibernate, Clear Case, JUnit, Rational Clear Quest, Execute Query, Web Sphere Application server 6.0, Web Sphere Portal Server 5.1, web sphere process server 6.1
Onsite offshore coordinator / Team lead
Confidential, Atlanta
Responsibilities:
- Worked on the IBM Web Sphere Portal server all the configurations related the mapping of Portlets, cross wiring using Portal Admin Console.
- Pages were created in the IBM Web Sphere portal Server; portlets were mapped to the respective pages.
- Converted UI prototype into JSF 1.1 version pages.
- Worked on web sphere portal admin console for customized Themes and Skins.
- Security settings were done on the pages and portlets based on the user roles and groups as per the role matrix.
- Deployment of the projects, assigning roles to portlets and pages, URL Mapping of the pages was automated using the XML Access scripts. Documents related to Deployments, Applying Themes and Skins were done.
- IBM Portal configurations were done related the memory usage, logs and trace for good performance created the web pages using XML Access Script.
- Planned and implemented automation of portal scripts along with mapping completion of the build deployment.
- Created business processes using WID and WPS.
- Created custom jsp tag libraries.
Environment: Oracle Database 10g, IBM Rational Application Developer, JEE, JSF, EJB (Session Beans), Hibernate, Java, Clear Case, Eclipse, JUnit, Rational Clear Quest, TOAD, Execute Query, Web Sphere Application server and web sphere Portal server.
Designer/Developer
Confidential
Responsibilities:
- Developed UI using Swing (GUI) according to client specification.
- Introduced delegate flow at client side and EJB at server component and session management is established.
- Involved in finding the flow for Cache Management.
- Worked on deviation flows while hitting the data base in order to separate client component and server component
- Deployed application in JBoss server.
- Created JASPER reports by converting xmls to PDF, HTML, XLS and CSV.
- Developed on solution projects (.sln) in visual studio.
- Worked on Remote method Invocation (RMI) concepts.
- Performed debugging using break points.
Environment: JBOSS Server, Java script, EJB, My SQL, Java (Swing GUI), Eclipse, HTML