Bigdata Hadoop Admin Consultant Resume
San Ramon, CA
SUMMARY
- Around 10+ years of IT experience in requirement analysis, design, development, implementation, support and co - ordination of web based applications
- 3+ year experience working on Pivotal HD 2.x,1.x,HDP built on Apache Hadoop stack 2.x,1.x
- 4+ years of OracleSOA11g/10g development/administrator experience in SOA,BPEL,OSB,BAM
- 5+ years of Java/J2ee technologies expertise
- Expertise in the usage of Hadoop ecosystem’s core components like HDFS,MapReduce,Pig,hive,HCatalog,Hbase,ZooKeeper,Oozie,Sqoop,Flume
- Working experience in the usage of Hortonworks(HDP)2.x, Pivotal Hadoop distribution2.x,1.x(EMC GreenPlum) built on Apache stack 2.x
- Expertise in the usage of Pivotal Hawq which is parallel SQL processing engine rendering Hadoop queries faster than any Hadoop based query interface
- Expertise in the usage of SQL(RDBMS) and NoSQL(non-RDBMS)
- Expertise using Talend, Pentaho,Sqoop for ETL operations
- Expertise working on the use cases related to Data Ingestion, Data Streaming, Sensor Data using large data sets
- Expertise in the installation, configuration and management of Hadoop Clusters.
- Monitor Hadoop cluster job performance and capacity planning
- Worked on a POC for streaming and analyzing Twitter data using Apache Flume to Hadoop cluster
- Expertise using Apache Spark(fast engine for large-scale data processing) and Shark(fast hive SQL on Spark)
- Worked on a POC for performance evaluation of using Apache Spark/Shark against Apache Hive
- Experience in using Apache Ambari which is a framework for provisioning, managing and monitoring Apache Hadoop clusters
- Expertise in the usage of Kerberos Security deployed on Hadoop Cluster
- Experience in understanding of Ganglia and Nagios
- Expertise in using Sqoop to import and export data from external systems to HDFS
- Experience in using Hortonworks Sandbox 2.0 and Map R SandBox
- Used Beeswax tool with in HDP(Hortonwoks Data Platform) to interact with Apache Hive
- Experience in using Map R Control System which is a platform for providing the dashboard view of the health of the cluster Confidential a glance
- Expertise in developing and deploying OSB services and BPEL Processes using technology adapters (DB Adapter, File Adapter, JMS Adapter), Fault handling(catch and catchall), Messaging and Alerts, Transformations and Sensors
- Expertise in the use of WLST scripting to compile, deploy, list SOA composites and OSB services
- Expertise in installing Hadoop using Apache Ambari, enabling Kerberos Security and Namenode High Availability
- Expertise in taking Offline and Online backups for SOA runtime, application, metadata, Config data
- Expertise in SOA performance tuning by following Purging strategies
TECHNICAL SKILLS
Frameworks/DFS: Hadoop,Hbase,Hive,Pig,MapReduce,HDF2.0,Map R NFS, OracleSOASuite11g/10g,AIA2.5,O2B PIP, Oracle BPEL,ESB,OWSM,EDN,BusinessRules,ActiveVOS,Mediator,Jdeveloper,EM,BAM,OSB11
Servers: WebLogic 8.1/9.2/10.3 , Tomcat4
IDE: Eclipse, Jdeveloper
Messaging Middleware: JMS (Weblogic)
Databases: Oracle XE, Oracle Database 10g, DB2
Other Tools & Versioning systems: TOAD, CVS, VSS, Clear Case
Operating Systems: UNIX, Linux, Ubuntu,Windows NT 4.0, Windows 2000/XP
PROFESSIONAL EXPERIENCE
Confidential - San Ramon, CA
BigData Hadoop Admin Consultant
Responsibilities:
- Installation, Configuration and Management of Greenplum Pivotal Hadoop cluster
- Involved in the User Creation and Management in IBD platform for various Confidential ’s business units.
- Used Pivotal Hadoop for Data Ingestion into Hawq using Sqoop,Hive,Hive2,PXF,GemfireXD
- Expertise in the validation of Hive, Hbase, Sqoop, Flume, Oozie jobs
- Expertise in the usage of Kerberos Security deployed on Hadoop Cluster
- Enabled Namenode and Pivotal Cluster high availability in IBD environments.
- Expertise using ETL Systems as the Staging Systems for User Management, Data Loading, Data Analytics, Hadoop Cluster access for end users
- Implemented enforcing the web User authentication for Namenode, Oozie webconsoles
- Coordinate with operational Hadoop support team.
- Manage and Review Hadoop log files.
- Used GitLab UI with Puppet as the platform to manage the Hadoop users
- Defined and implemented the logging retention period for Hadoop components
- Underwent Greenplum database training
- Address the performance tuning of Hadoop ETL processes against very large data set work directly with statistically on implementing solutions involving predictive analytics.
- Develop Hadoop monitoring processes (capacity, performance, consistency) to assure processing issues are identified and resolved swiftly.
- Instrumental in setting the GemfireXD cluster environment in distributed mode
Environment: Greenplum PHD, ETL tools:Talend,HVR,Unix:RedHat,Centos,Java1.7,Sqoop
Confidential - SFO, CA
Hadoop Consultant
Responsibilities:
- Gathered the requirements from the management and from the target of each module and analyzed the requirements to develop the initial high-level design.
- Installing Hadoop Cluster
- Writing Map/Reduce programmes
- Load the data from Teradata tables into Hive Tables
- Testing the performance of Hadoop/Hive against Teradata
- Creation of Table schema in Hive
- Participating in client calls for Design, Code and Test Cases walkthrough
- Design and build robust Hadoop solutions for Big data problems.
- Deep understanding and related experience with Hadoop, HBase, Hive, Pig and Map/Reduce.
- Implement ETL processes with Hadoop, MapReduce, Java, Pig, Hive
- Works with application teams to install operating system and Hadoop updates,version upgrades as required.
- Coordinate with operational Hadoop support team.
- Manage and Review Hadoop log files.
- Address the performance tuning of Hadoop ETL processes against very large data set work directly with statistically on implementing solutions involving predictive analytics.
- Develop Hadoop monitoring processes (capacity, performance, consistency) to assure processing issues are identified and resolved swiftly.
Environment: Hadoop,Hbase,Hive,Windows 2003, MS SQL Server 2000, ASP.Net, Visual Basic 6.0, Business Objects and ETL tool (Informatica), UNIX, Linux, Ubuntu, eclipse, Java
Confidential - Houston, Texas
Java SOA consultant
Responsibilities:
- Involved in using OSB 11g, SOA 11g and BAM 11g components of Oracle SOA Suite 11g for creating SOA webservices for Pyxis Mobile
- Involved in usage of SOA 11g for building business logic and using DB, File, FTP,JMS adapters for invoking DB stored procedures in IMS back end application, reading and writing files, using FTP for inserting images in IMS backend application, using JMS for auditing messages between OSB and SOA
- Involved in defining Service Virtualization for Proxy Service in OSB
- Involved in using Direct Binding for OSB to SOA communication
- Involved in defining error handling in OSB and SOA by defining OSB and SOA specific error codes
- Involved in defining Auditing in OSB and SOA
- Involved in using test console in OSB for testing Proxy message flow and Business Service
- Extensively used SOAP UI for validating SOA services
- Defined and executed Configuration plan for environment specific properties
- Expertise in using WLST(Web Logic Scripting Tool) and Ant Scripts for SOA composites for automating SOA composite deployment
- Involved in designing and architecting SOA recovery steps in a Disaster
- Expertise working in a clustered SOA environment
- Expertise in developing and deploying BPEL Processes using technology adapters (DB Adapter, File Adapter, JMS Adapter), Fault handling(catch and catchall), Messaging and Alerts, Transformations and Sensors
- Expertise in taking Offline and Online backups for SOA runtime, application, metadata, Config data
- Expertise in SOA performance tuning by following Purging strategies
- Expertise in designing High Availability architecture for SOA and Webtier components in a Clustered environment
- Expertise in File, FTP,JMS,DB adapter configuration in Weblogic Server admin
- Extensively used XSLT transformation in SOA and Xquery transformation in OSB.
- Involved in creating Service Account for Proxy and Business service authentication in OSB11g project
- Expertise in the use of WLST scripting to compile, deploy, list and administer composites
- Expertise in use of node manager to manage SOA admin, managed servers
Environment: Oracle SOA Suite 11g, Web Logic Application Server10g, SOAP UI, Oracle, UNIX, EM,BAM 11g, Jdeveloper11g, OSB11g,Toad
Confidential, VA
Java SOA & AIA PIP Consultant
Responsibilities:
- Involved in creating delta Tech Designs based on the Functional Designs
- Involved in the usage of AIA SalesOrderEBO
- Involved in the usage of SalesOrderEBM, ProcessSalesOrderFulfillmentEBM and CreateShipmentRequestResponseEBM
- Involved in the usage of data mapping between the Siebel Payload and ProcessSalesOrderFulfillementEBM
- Involved in analyzing and datamapping between Siebel(Source),SOA and NOE(Target) systems
- Involved in designing and building the Contract Activation and DeviceUpdate flows
- Involved in integrating the Submit Order interfaces with other interfaces
- Involved in the usage of DVM lookup for referencing the Domain Value Maps
- Involved in the usage of configuring producer and consumer Queues in OC4J and Weblogic server
- Involved in the usage of creating JMS services in BPEL process for Writing and Listening to the messages in the Queues
- Involved in the usage of Hermes JMS tool to test the JMS Queues
- Involved in end to end testing to validate the order progression from Source to Target systems
- Involved in using OWSM for creating Users & Groups and attached/detached policies in EM Console for composite services using user-name token, saml token mechanisms and thus making SOA layered composite application more securable and less complex
- Used to Test the developed BPEL processes using WebService Client tools and SOAP UI
- Developed custom XPath functions to transform messages.
- Implemented Exception Handling for the BPEL Process models to handle System, Business exceptions and used Compensation handlers to handle the transaction if an exception occurs.
- Involved in creating Proxy and Business services using OSB for building SOA 11g interface
- Involved in using MFL(Machine Format Language) with Proxy and Business service for file transfer
- Involved in creating Service Account for Proxy and Business service authentication in OSB11g project
- Involved in designing and architecting SOA recovery steps in a Disaster
- Expertise working in a clustered SOA environment
- Expertise in developing and deploying BPEL Processes using technology adapters (DB Adapter, File Adapter, JMS Adapter), Fault handling(catch and catchall), Messaging and Alerts, Transformations and Sensors
- Expertise in taking Offline and Online backups for SOA runtime, application, metadata, Config data
- Expertise in SOA performance tuning by following Purging strategies
- Expertise in designing High Availability architecture for SOA and Webtier components in a Clustered environment
- Expertise in the use of WLST scripting to compile, deploy, list composites
- Expertise in use of node manager to manage SOA admin, managed servers
Environment: AIA 2.5,O2B PIP,Oracle SOA Suite 11g/10g, WebLogic Application Server10g, SOAP UI, Oracle, UNIX, Mediator, EM,BAM, BusinessRules, EDN,Jdeveloper10g/11g, OSB11g
Confidential, Atlanta
Java Consultant
Responsibilities:
- Responsible for Requirements gathering, Functional specs, Use case specs and System design.
- Developed the application using the Spring MVC frame work. Involved in creating or updating the servlet.xml, application-context files, writing HandlerMappings, Interceptors, controllers and handlers, view resolvers etc….
- .Installed and configured the high availability setup for Oracle ESB. This includes setting up Oracle application servers and Apache web servers in a cluster, installing the runtime and repository ESB components and configuring the web servers with a load balancer. The high-availability implementation is done on Linux servers..
- Leading the team of 4. It was the offshore and onsite project, Confidential offshore there were 4 people, and I was the lead for the team Confidential onsite. Guiding the team with changes, involving them in the meetings and discussions with the client.
- Build process with ANT framework to build and deploy the application. Used shell script and perl scripts for different batch jobs.
- Created and deployed EJB’s to process business logic and interfaced them with Servlets.
- Involved in Ajax implementation for the simple page updates of this application instead of complete page refresh.
- Used ActiveVOS tool from ActiveEndpoints for BPEL Workflows
- Involved in using ActiveVOS tool for creating, managing, deploying, auditing and monitoring processes
- Using ActiveVOS Designer as the development environment
- Used to Test the developed BPEL processes using Webservice Client tools and SOAP UI
- Involved in production support working on Telcordia’s Exact and IDIS systems
- Developed custom XPath functions to transform messages.
- Involved in the development of BAM dashboards for capturing various metrics of various applications
- Implemented Exception Handling for the BPEL Process models to handle System, Business exceptions and used Compensation handlers to handle the transaction if an exception occurs.
Environment: Java, Spring MVC, JSP, Servlets, JMS, Spring, Hibernate, SQL, LDAP, Oracle, Websphere6, Eclipse, Ant, CVS, Windows NT and UNIX.
Confidential
Oracle SOA Engineer
Responsibilities:
- Developed different SOAP services and Web Services using WSDL, SOAP, AXIS, and Oracle JDeveloper.
- Developed custom XPath functions to transform messages
- Used JDeveloper IDE for coding BPEL Process Models.
- Implemented Exception Handling for the BPEL Process models to handle System, Business exceptions and used Compensation handlers to handle the transactions if an exception occurs
- Analyzed the business needs for the processes and modules that needed migration
- Design and Implemented Adapters for backward compatibility with old modules
- Implement old processes into new modules and make sure all the business needs are addressed.
- Design, development and deployment of Web Services was done using Oracle BPEL, JAVA, and XML (WSDLs)
- Assisted in training others on BPEL and determining how to integrate BPEL into the Common Services projects.
- Extensively used BPEL to determine dynamic partner link calls, updating databases via BPEL, generating proxy stub classes to call BPEL process from a web application
Environment: Oracle SOA Suite (BPEL, BAM, ESB, Web service manager),Informatic ActiveVoS, Oracle Apps Server, JDeveloper, Linux, Siebel Adapter, ClearCase.
Confidential
TouchPoint Developer
Responsibilities:
- Involved in SDLC.
- Involved in defect fixeds with respect to TouchPoint application
- Involved in the SRD(Software Requirement Specification)reviews
- Involved in base lining(final draft) of SRD
- Involved in preparing and executing UT(Unit Test) and IT(Integration Test)plans
- Involved in executing the UT
- Involved in implementing the development by identifying the Use-Cases from SRD matching the Use-Cases with matching Prototypes from the Design document and finally identifying the fields and parameters from the API (Application Programming Interface).
Environment: TouchPoint14.0,Apache Tomcat 6.0.18, Apache Ant 1.7, Eclipse 3.2, JSP, JSF, JS, Struts1.0, Hibernate, XML, XSLT, AJAX My SQL 5.0 Windows XP, SVN, Tortoise SVN
Confidential
System Engineer
Responsibilities:
- Involved in SDLC.
- Involved in the SRD(Software Requirement Specification)reviews
- Involved in base lining(final draft) of SRD
- Involved in preparing and executing UT(Unit Test) and IT(Integration Test)plans
- Involved in executing the UT with respect to Simulator
- Involved in implementing the development by identifying the Use-Cases from SRD matching the Use-Cases with matching Prototypes from the Design document and finally identifying the fields and parameters from the API (Application Programming Interface).
- Involved in handling tickets with respect to Production issues.
- Worked on a production ticket on a Memory Leakage issue on a C App. For which used IBM Rational PurifyPlus7.0 as the Memory Leakage detection tool to detect the Memory Leakage in the C application
Environment: IBM Rational PurifyPlus7.0,Java,SpringMVC,JSP,Servlets,JMS,C,C++,IBMWebSphere,RAD, Ant, CVS, Windows NT& UNIX.
Confidential
Cúram Software Engineer
Responsibilities:
- Involved in business requirements analysis.
- Built the application using Struts framework with JSP as view part.
- Developed Dispatch Actions, Action Forms and Custom taglibs in Struts framework.
- Designed JSP pages as view in Struts for front-end templates.
- Developed Session Beans for handling the back business requirements.
- Used the IDE for development and Clear Case for the versioning.
- Involved in configuring the resources and administering the Web sphere application server 6.
- Built and deployed the application on Web sphere application server.
- Written stored procedures in DB2.
- Developed code to handle web requests involving Request Handlers, Business Objects, and Data Access Objects. Has coded different package structures based on the purpose and security issues handled by that particular package which assists developers in future enhancements or modifications of code.
- Involved in making the client side validations with JavaScript.
- Involved in code reviews, system integration and testing. Developed unit test cases using JUnit framework.
- Involved in deploying the application on UNIX (DEV, QA and Prod Environments) box.
- Used Change management tool Service Center for promoting the War file from one environment to other.
- Involved in user acceptance testing, fixing bugs and Production support.
Environment: Java, J2EE, Cúram Software Apache Struts, Websphere5 & 6, JNDI, JDBC, JSP, UNIX and Windows NT, DB2 and SQL Server 2000.
Confidential
J2EE Programmer
Responsibilities:
- As Team member I involved mainly in implementation and customization of tasks related to module
- Creating of the database tables, writing the SQL Queries and Stored Procedures.
- Coding Java, JSP, and Servlets using the extended Struts Framework.
- Involved in writing the programs for the XA transaction management on multiple databases of the application.
- Review, Deployment and JUnit Testing.
Environment: Java, Extended Struts Framework, JSP, Servlets, SQL, LDAP, Oracle RDBMS, JBOSS Server 4.0, Eclipse, Ant, CVS, Windows NT and Unix.