We provide IT Staff Augmentation Services!

Bigdata Engineer Resume

5.00/5 (Submit Your Rating)

SUMMARY:

  • Over 16 years of IT experience and 8 years as Sr. Data warehouse Engineer
  • Confidential OCP certified professional (7.x, 8, 8i, 9g,10g) and Expert in 11g, 12x.
  • Over 12 years of Confidential Database & Data warehouse Engineer ( OCP certified Professional)
  • Over 5 years as Sr. ETL developer using Informatica, Confidential OWB, Talend ETL
  • Over 5 years as Bigdata developer ( SPARK & SCALA certified professional ) / Bigdata Engineer (Hadoop, Spark, Scala, Kafka)
  • Over 14 years of experience working with data repository, data modelling and case tools: Confidential Case, Designer 2000, Informatica power cent. Talend ETL tools etc.,
  • Experienced in the Unix, Linux, VMS, Apache Server, NT and Windows environment.
  • Excellent verbal and written communication skills & highly detail oriented, proven to be highly effective in interfacing across business and technical groups.

TECHNICAL SKILLS:

Data warehouse Integration Systems & Tools: Informatica, Confidential Data warehouse Builder, OID, Talend Cluster Edition 5 + Installation, Administration & Deployment

Software: Confidential Database software, Confidential SOA ( 9x to 12x), Micro - soft office tools

Languages: HTML, DHTML, XML, JAVA, Java script, JSF, JSP, Linux shell, Unix shell scripting, SQL, PL/SQL, PL/SQL AGENT, Turbo C, c++, Perl, Python, SCALA

RDBMS: Confidential 12g, Confidential 11g.x, Confidential 10g.x, Confidential 9i. RAC/8i. (8.1.7) /8.0.5/7.x/6.0 ( Confidential OPS/OPFS) with databases size from 300GB to 160 TB

NoSQL: HBase, MongoDB, PostgreSQL

Big Data and Cloud Technologies: Hadoop, Map reduce, Spark, Scala, HBase, Pig, Hive & Amazon EC2, EC3, EMR, S3 (POC), Exposure to Amazon EBS, Glacier, Sage maker, Machine learning. Hadoop eco system and building data pipe lines

EXPERIENCE:

Confidential

Bigdata Engineer

Involved in developing fault tolerant engineered appliances using cluster technologies. As part of our exploration to adopt new technologies, I was involved in exploring the adoption of SPAK and Scala on top of Hadoop since 2012. The modular engineered appliance with 2 node cluster capable of scaling up to 128 nodes for bigdata, high volume and high velocity catering to special segment of private businesses and federal agencies mainly interested in co-location were architected. I was entrusted the deployment of Spark on top of Hadoop with Spark as a main data processing unit. The scope of the project involved showcasing the product for its speed and scalability at an affordable cost tailored to meet the client demands. Hence, I used multiple use cases such as earthquake detection, recommendation, twitter sentiment analysis using Spark streaming API and Scala as main programming language to showcase the product. Scala leverages the collaborative filtering and ALS for the recommendation feature that is not yet rated extensively using SPARK libraries. The use cases emphasized how Scala takes advantage of its key features such as 'ReducebyKey' helps in avoiding the network traffic allowing data engineers to avoid unnecessary infrastructure cost. Built multiple Spark applications extensively using SCALA and Spark APIs.

Confidential

Data Engineering Technical Lead / Informatica (Nike)

Worked as a Lead Data Integration &RAC consultant for a leading Healthcare provider designed, developed, implemented and supported software & systems necessary to integrate enterprise practice management system with enterprise insurance system. This project had a high level of complexity and a short timeline for completion. Over the last 12 months my responsibilities have included Bigdata data pipe line building, requirements definition of internal and external interfaces, assisting configuration of the Talend Data Integration software, software development of middleware components to complete the job working with other team members. The final solution leveraged Talend Data Integration 5+, Java, Spring Hibernate, REST web services, Tomcat 6, SQL Server 2008 and Confidential 11g RAC. Developed over 16 interfaces using Talend Data Integration Software. As Lead RAC consultant configured the RAC databases for dev, test & prod and supported the migration from legacy system and mentored non- Confidential DBAs throughout the SDLC of enterprise implementation. Further, tuned PROD database and supported until client was ready to take over the maintenance of all the systems

Worked as Technical Engineering Lead for a reputed retail brand for their key projects such as Bigdata "Data Integration Hub" and "Consumer 360-degree project". Worked closely with all the project stake holders & development teams to successfully steer the project eliminating all the technical hurdles. Successfully supported the program managers, project managers, solution architects, dba's in the implementation of both projects. Worked with the managers & development team members to solve the most complex data issues, story development, SCRUM, architectural issues, resource issues and "Informatica" Big data service issues. Developed a POC for "Consumer 360 degree" using Informatica based on SOA architecture and successfully showcased the capability of pulling data from various disparate sources based on the "consumer canonical data model". The "Consumer 360-degree project" POC helped to build a real time composite view of existing data across disparate systems in order to deliver personalized end user experiences. The POC was successfully built in a short span of time against "Amazon" EC2 environment and further with Enterprise Architecture against Confidential Exadata & Coherence.

Worked as a Lead consultant and architected HA solutions for a leading communication company and deployed MC (Media centre console & applications) on top of Confidential SOA suite against a 2 node RAC cluster. Involved in setting up the Confidential Fusion middle ware, 2 node RAC database on IBM storage system using IBM servers running Red Hat Enterprise Linux 5.5 implementing MC failover on Confidential database 11g R2 with Confidential Real Applications Clusters (RAC). The scope of the project involved providing architecture, POC solution and finally setting up the dev, test & production RAC databases exploiting Confidential middleware fusion SOA suite. The key tasks performed were planning the hardware for Confidential RAC and SOA suite implementation, configuring the servers and storage system, Implementing Confidential Fusion 11g SOA suite best practices, SOA suite installing and configuring Confidential Grid Infrastructure 11g R2 and Confidential database 11g R2 with RAC option, deploying MC application on 2 node RAC database with Confidential Fusion SOA suite and configuring the Confidential cluster ware High Availability frame work to deploy and enable MC application failover.

Worked as a Lead consultant and production support for very large RAC databases of various sizes from 4TB to 160 TB databases on multiple platforms such as Solaris, HP and Linux. Architected the Data warehouse ( Confidential 11g - 11.1.5) as well as installed, configured and built 2 node RAC clusters ( Confidential 11g) for DEV, TEST and PROD. Supported development team for database design, worked with team for logical model and implemented physical model. Further, supported Informatica, Confidential Data warehouse as well as OID tools at various phases for the development and production team. Performed performance tuning for OLTP and Data warehouse using AWR and ADDM, setup the RMAN backup and recovery strategies for all databases.

Confidential

Sr. Principal Engineer

  • Performed discovery phase, Assessment phase, Recommendation phase for Large Data warehouse - for an insurance company. Installed and configured an Confidential 11g (RAC) 2 node database on Solaris . Validated the shared disk array, configured interconnect and public network for RAC for Test database. Architected the SPA ( System Performance implementation) using Confidential Real application testing for 9i to 11g database upgrade, captured the performance impact using SQL impact performance analyser, fixed the regressed SQL tuning sets on the targeted databases. Lead DBA consultant on 9i RAC development project & Data guard for one of the largest Computer, Printer & Inkjet cartridge manufacturer. During this engagement, the RAC infrastructure was standardized, Transparent Application Fail over was implemented, and load balancing was benchmarked. Worked closely with the various client project leads in best practices, and system standardizations, and the testing of the HP cluster's ability to handle true 7X24 zero downtime commitments.
  • Project lead on migration/upgrade of Informix standalone database for large Computer manufacturer to 9i with addition of Data Guard fail over/hot standby database to minimize exposure in the event of primary server failure. Implemented Enterprise Manager, Confidential Management Server and Recovery Manager as part of overall lights out holistic solution to assure not only redundancy, but also recovery in a worst-case scenario.

We'd love your feedback!