Big Data Architect Resume
NY
SUMMARY
- Big Data Architect with 2 ½ years experience in Hadoop and Big Data Development Framework and its ecosystem.
- Enterprise Content Management Solutions Architect and Lead with over 12 years of IT experience.
- Expertise in Apache Storm, Apache Spark, Kafka, Cassandra, Drools.
- Well experienced in analyzing the data using custom MapReduce programs in JAVA, HIVEQL and PIG Latin. Using custom UDSF’s to extend HIVE and PIG core functionality
- Sound knowledge and hands on experience on Hadoop Architecture and various other components such as HDFS, Name Node, Job Tracker, Data Node, Task Tracker, YARN, MapReduce, HIVE, PIG, Spark, Shark, Zookeeper, Avro and Oozie
- Experienced in using Sqoop and Flume to import and export data from HDFS to RDBMS and vice - versa
- Good understanding and versed on NoSQL databases like HBase, Cassandra and MongoDB.
- Experience with workflow engine like Oozie, used in order to run jobs of Hadoop MapReduce and Pig
- Experienced in building Case Management Applications using the Documentum xCP Platorm
- Proven experience in Internet, Intranet and stand-alone applications using J2EE (Java2, JSP, Java Beans, JDBC), UML, ASP, VBScript, Java Script, XML, XSL, HTML, HTTP, TCP/IP on various application servers like IIS, Tomcat, WebSphere, Apache and Weblogic.
- Experienced in implementing Client Server and N-Tier Web Based Enterprise solutions using J2EE as well Microsoft ActiveX technologies.
- Comprehensive knowledge of problem analysis, structured analysis & design, and lifecycle development.
- Lead overall project design, development, testing, implementation and training.
- Extensive experience in implementing Web Based Content Management Solutions using Documentum WebPublisher and Site Caching Services.
- Experienced in implementing validated solutions for Regulated Industries like Pharma (CFR Part 11), Finance and Education.
- Strong analytical and conceptual skills in database design and development using SQL Server, Oracle8i/9i/10, and other RDBMS.
- Strong experience in working on Linux/Unix based environments as well as Windows based Server environments.
- Possess good communication skills and excellent team worker qualities
TECHNICAL SKILLS
Tools: /APIs: J2EE (Servlets, JDBC, JSP, Java Beans, Taglibs), ASP, MVC, VBScript, JavaScript (client side), UML, XML, XSL, XSLT, Castor XML HTML 4.0, Apache, Tomcat, WebSphere, Weblogic
Hadoop/Big Data: HDFS, MapReduce, HBase, Spark, Shark, Pig, Hive, Sqoop, Flume, YARN and Oozie.
Documentum: Documentum 4i/5i/6, xCP(Taskspace, Process Builder, Application Builder), Developer Studio, Workflow Manager, Docbasic, Right Site Server, Documentum Administrator, iDQL, iAPI, WDK4/5/6, DFC4/5/6, Business Objects Framework, Documentum Server Methods, Web Publisher, WebCache, WebTop, eRoom 7.x, Captiva, Kofax, ADTS, DFS, xPlore, Brava.
Portlets: Liferay
IDE: Eclipse, Jbuilder, Composer
Languages: Java, Scala, C, C++, SQL, Python, PL/SQL, and Visual Basic.
Databases and Tools: Oracle 7.X/8.0/9i/10g, MS Access, MS-SQL Server 6.5/7.0, SQL, PL/SQL, TOAD.
Operating Systems: MS-Windows 95/98/ME/2000/XP/2003/2008, Windows NT, DOS, and UNIX.
PROFESSIONAL EXPERIENCE
Confidential, NY
Big Data Architect
Responsibilities:
- Implemented Lamda Architecture with Kafka, Storm, Spark and Cassandra
- Performance tuned Real-time alerting system to process trade messages at 250k/sec with latency of 10 ms
- Integrated Drools Rules engine with Apace Storm for real-time alerting
- Implemented Map Reduce jobs for data conversion
- Launched 100 plus nodes in AWS using vagrant, puppet and ruby
- Benchmarked Kafka, Storm and Cassandra for Low latency and High throughout.
- Evaluated different serialization based on the data size and format
- Reduced foot print of the servers by optimizing the deployment and technology stratergies.
- Developed POC’s for Monitoring and Metrics
- Involved in evaluating Apace Mesos for provisioning and scheduling
Confidential, NJ
Hadoop Consultant
Responsibilities:
- Involved in Requirement gathering, system architect and development of Design Document and Technical Design Document
- Developed data pipeline using Flume, Sqoop, Pig and Java MapReduce to ingest customer behavioral data and financial histories into HDFS for analysis.
- Developed free text search solution with Hadoop and Solr. Analyzing emails for compliance and eDiscovery.
- Worked on importing and exporting data from Oracle and SQL Server into HDFS and HIVE using Sqoop for analysis, visualization and to generate reports.
- Developed multiple MapReduce jobs in java for data cleaning.
- Integrated Spark and Shark with Hadoop.
- Redesigned MapReduce jobs with Scala and Spark to increase performance.
- Developed Hive UDF to parse the staged raw data to get the Hit Times of the claims from a specific branch for a particular insurance type code.
- Schedule these jobs with workflow engine like Oozie. Actions can be performed both sequentially and parallel using Oozie.
- Built wrapper shell scripts to hold Oozie workflow.
- Involved in collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
- Involved in creating Hadoop streaming jobs using Python.
- Provided ad-hoc queries and data metrics to the Business Users using Hive, Pig.
- Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.
- Used Pig as ETL tool to do transformations, event joins and some pre-aggregations before storing the data onto HDFS.
- Worked on MapReduce Joins in querying multiple semi-structured data as per analytic needs.
- Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
- Created many Java UDF and UDAFs in hive for functions that were not preexisting in Hive like the rank, Csum, etc.
- Used Hive and created Hive tables and involved in data loading and writing Hive UDFs.
- Developed POC for Apache Kafka and Apache Storm.
Confidential, NY
Hadoop Consultant
Responsibilities:
- Worked on Hadoop cluster which ranged from 4-8 nodes during pre-production stage and it was sometimes extended up to 24 nodes during production
- Used Sqoopto import the data from RDBMS to Hadoop Distributed File System (HDFS) and later analysed the imported data using Hadoop Components
- Established custom MapReduces programs in order to analyze data and used Pig Latin to clean unwanted data
- Did various performance optimizations like using distributed cache for small datasets, Partition, Bucketing in hive and Map Side joins.
- Involved in creating Hive tables, then applied HiveQL on those tables for data validation.
- Involved in loading and transforming large sets of Structured, Semi-Structured and Unstructured data and analyzed them by running Hive queries and Pig scripts
- Used Zookeeper to manage coordination among the clusters
- Experienced in analyzing HBase database and compare it with other open-source NoSQL databases to find which one of them better suites the current requirements.
- Gave assistance in exporting the analyzed data to RDBMS using Sqoop.
- Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts
- Installed Oozie workflow engine to run multiple Hive and Pig jobs which run independently with time and data availability
Confidential, NJ
Documentum/Java Consultant
Responsibilities:
- Involved in Requirement gathering, system architect and development of Design Document and Technical Design Document
- Involved in design and implementation of data extract for Legal holds and Audit reports
- Involved in upgrade of the system from 6.5to 6.7SP2.
- Migration of Workflows to xCP based workflows.
- Designed dashboard style reports in Task space using Business Activity Monitor.
- Developed custom widget and button for xCP Designer that will allow the user to export results in a grid to a CSV File from the front end application.
- Developed customized workflows, jobs and methods using Documentum tools.
- Fine-tuned existing workflow performance.
- Customized the various components of Webtop to meet the requirements.
- Involved in implementing Content Services for SAP and Archive Services for SAP
- Involved in Installing and implementing scanning based solutions using Captiva.
- Involved in design and development of search algorithm to highlight the content of Word documents.
- Involved in troubleshooting workflow issues.
- Involved in day to day maintenance of production systems.
- Integrated Documentum with third party applications like Life ray.
Confidential, FL
Documentum Consultant
Responsibilities:
- Reviewed the Architecture and Design of the system.
- Provided Gap Analysis of the system.
- Improved Performance of Application.
- Implemented DFS Services to integrate Xyleme with Documentum.
- Install and setup the Content server, xCP, DAMTop and DA on Cloud based servers.
- Upgraded the docbases from 6.0 to 6.7 SP1.
- Upgraded Digital Asset to 6.7
- Implemented xCP based web interface and workflows
- Integrated Documentum with Xyleme (LCMS)
- Implemented DFS Services to integrate Xyleme with Documentum.
Confidential, NJ
Documentum/Java Consultant
Responsibilities:
- Involved in Requirement gathering and development of Design Document and Technical Design Document
- Installed and configured xDB.
- Designed and implemented ARC workflow/case management solution using xCP bundle (Taskspace, process builder and process integrator).
- Involved in analysis of High availability Configuration of XML store.
- Involved in various xDB administration tasks like creating and restoring backups, Managing Libraries, Indexes and using xQuery.
- Involved in migration of the system from 5.3 sp4 to 6.5, 6.5 Oracle based CS to SQL server based CS.
- Developed customized workflows, jobs and methods using Documentum tools.
- Developed Methods for back end processing of tasks.
- Integrated DRS (Documentum Reporting Services) with Webtop and allowed the user to generate reports using single login.
- Installed, Configured and customized Advanced Document Transformation Services(ADTS)
- Designed and Implemented DFS bases RestFul services independent of client.
- Implemented DFS and RestFul Services to implement HCP portal.
- Implemented Custom Load Policy program to upload new policies into the system.
- Customized the various components of Webtop like create, new, import, search, advanced search and other components to meet the requirements.
- Installed, configured and fine tuned the Documentum Index Sever.
- Customized the LDAP Sync job in Documentum to retrieve the users and groups from Active Directory using custom filters.
- Involved in day to day maintenance of production systems.
- Integrated Documentum with third party applications like Life ray, Intranet and SAP.
Confidential, NJ
Documentun/Java Analyst
Responsibilities:
- Involved in Requirement gathering and development of Design Document and Technical Design Document
- Implemented prototype for UI Study and gathered user requirements based on feedback.
- Redesigned various components of WebTop like Inbox, Task Manager, Task Attachments, Task Comments to provide a one stop for processing the task items.
- Simulate the Invoice Posting to SAP using JCO BAPI calls.
- Involved in Workflow Optimizations for invoice posting to SAP.
Confidential, NJ
Documentum/Java Consultant
Responsibilities:
- Involved in evaluation of Documentum 6 products.
- Involved in installation and configuration of SharePoint for Pilot
- Involved in Requirement gathering and development of Design Document and Technical Design Document
- Involved in migration of the system from 5.2.5 sp4 to 5.3 sp2
- Developed customized workflows, jobs and methods using Documentum tools.
- Developed Methods for back end processing of tasks.
- Implemented Reporting component to display the reports based on user criteria.
- Implemented business objects to generate folder paths and auto numbering.
- Customized the various components of Webtop like create, new, import, browsertree, search, advanced search and other components to meet the requirements.
- Installed, configured and fine tuned the Documentum Index Server.
- Customized the LDAP Sync job in Documentum to retrieve the users and groups from Active Directory using custom filters.
- Implemented global search to search across different applications.
Confidential, NJ
Documentum Consultant
Responsibilities:
- Installed and configured eRoom Facilities, Communities and setup eRooms for Project Management.
- Configured and customized different document types, templates, and permission set templates, users and groups.
- Developed and implemented custom workflows, lifecycles.
- Developed Scripts to create users, groups and acls.
- Developed Methods and Jobs to Migrate documents from file system to docbase.
- Implemented Type Based Objects and Service Based Objects.
- Synchronized documentum user/groups to LDAP directory
- Customized various components of WebTop like Attributes, Import, Check-in, Search, Browse, Task Management and Advanced Search.
- Integrated eRoom with Documentum Extensions for Webtop and configured eRoom for content storage in the docbase.
- Integration involved creating a custom command extension in eRoom which when clicked by a user opens WDK components in a new browser window and logs in the user automatically using credentials from eRoom.
Confidential, NJ
Documentum Consultant
Responsibilities:
- Installed and configured eRoom Facilities, Communities and setup eRooms for Project Management. Integrated eRoom with Extensions.
- Configured and customized different document types, templates, and permission set templates, users and groups.
- Developed customized workflows, lifecycles, jobs and methods using Documentum tools.
- Developed Methods for back end processing of tasks.
Confidential, NJ
Documentum Consultant
Responsibilities:
- Installed, Configured Documentum Content Rendition Services, Kofax scanning software.
- Configured and customized different document types, templates, and permission set templates, users and groups.
- Developed customized workflows, lifecycles, jobs and methods using Documentum tools.
- Developed Methods for back end processing of tasks.
- Implemented a Messaging Service between Documentum and a web application.
- Developed custom Procedures, Methods and Jobs.
- Integrating scanning system Kofax with Documentum server
- Implemented Browse and Inbox portal’s using JSR 168 compliance and WDK for Pluto Portal. Configured Portal Pages, Users and Groups, Page Layouts on the Portal.
Confidential, NJ
Documentum/Java Consultant
Responsibilities:
- Modified the existing Workflow templates.
- Implemented the existing methods using java method server
- Developed JUnit test scripts for the server methods.
- Developed custom Methods and Jobs
- Installed and configured eRoom Facilities, Communities and setup eRooms for Project Management. Integrated eRoom with Documentum Extensions for Webtop and configured eRoom for content storage in the docbase.
Confidential, CA
Java/Documentum Consultant
Responsibilities:
- Installed, Configured Documentum e-Content Server 5.2, WebTop 5.2 and Web Logic 7.02
- Configured and customized different document types, templates, and permission set templates, users and groups.
- Customized various components of WebTop like Attributes, Import, Check-in, Search, and Advanced Search and others
- Customized look and feel of Webtop’s Classic View.
- Developed custom Actions based on roles.
- Developed Methods for back end processing of tasks.
- Implemented dynamic modification of attributes for document types with minimal changes.
- Developed custom Procedures, Methods and Jobs.
- Synchronized docbase users and groups to LDAP users
- Migrated WebTop 5.2 to WebTop 5.2.5
- Integrating scanning system KOFAX with Documentum server.